Method for controlling multi-screen simultaneous display of multiple observation modes of endoscope system, and endoscope system

Abstract

The embodiment of the invention provides a control method for multi-picture simultaneous display of multiple observation modes of an endoscope system and the endoscope system. The control method comprises the steps that a control instruction for multi-picture simultaneous display of the multiple observation modes is received; responding to the control instruction, controlling the light filtering switching element to start, switching the target light filtering area of each sampling period in the stable stage, and controlling the light source to start to switch the light source emission combination corresponding to each sampling period after the light filtering switching element enters the stable stage; the image processing element is controlled to enter a standby mode, and after the filtering switching element enters the stable stage, the image processing element exits from the standby mode and starts to switch the image processing parameters corresponding to the sampling periods; receiving an image generated by the image processing element based on at least part of the sampling period, and correspondingly displaying the image in the corresponding picture area; the filtering switching element, the light source and the image processing element in the stable stage are switched synchronously. The method is beneficial to improving the stability of simultaneous display of multiple pictures.

Description

Technical Field

[0001] This application relates to the field of endoscopy technology, and in particular to a control method for simultaneous display of multiple viewing modes of an endoscopy system and an endoscopy system thereof. Background Technology

[0002] An endoscope system is a diagnostic instrument that integrates traditional optics, ergonomics, precision mechanics, modern electronics, mathematics, and software technologies. Images of the inspected area under different spectral illumination conditions can reflect different tissue characteristics. Therefore, in many applications, there is a need to display multispectral images of the same inspected area in real time and simultaneously for operators to compare and analyze. However, existing endoscope systems generally suffer from performance bottlenecks in displaying multiple images of the same inspected area in real time and simultaneously, often exhibiting problems such as image delay, making it difficult to meet the objective requirement of real-time and simultaneous display. Summary of the Invention

[0003] This application provides a control method and endoscope system for simultaneous display of multiple viewing modes in an endoscope system, which achieves high-precision synchronization between filter switching, light source modulation and image processing to reduce image delay, thereby displaying multispectral imaging images of the same detected area in more real time and simultaneously, and improving the display effect of real-time and simultaneous display of multiple images.

[0004] In a first aspect, embodiments of this application provide a control method for simultaneous display of multiple screens in multiple observation modes of an endoscope system. The control method is applied to an endoscope system, which includes a processor, a light source assembly, an image processing element, and a display device. The light source assembly includes multiple light sources and a filter switching element disposed on the light emission path of the light sources. The filter switching element includes at least two filter regions. The light emitted by the light sources is combined or passes through the corresponding filter regions individually to output illumination light of a corresponding spectrum. The image processing element is used to acquire the light signal after the illumination light illuminates the area being examined, and retrieves corresponding image processing parameters for processing to generate an image. The display device includes multiple screen areas. The control method includes:

[0005] Receive control commands for simultaneous display of multiple screens in the various observation modes;

[0006] In response to the control command, the filter switching element is activated, and the target filter region for each sampling period is switched during the stabilization phase, wherein the target filter region is the filter region correspondingly set on the main axis of the light source;

[0007] In response to the control command, the light source is controlled to start switching the light source emission combination corresponding to each sampling period after the filter switching element enters the stable phase;

[0008] In response to the control command, the image processing element is controlled to enter standby mode, and after the filter switching element enters the stabilization stage, it exits the standby mode and begins to switch the image processing parameters corresponding to each sampling period.

[0009] The image is received by the image processing element based on at least a portion of the sampling period and displayed accordingly in the corresponding screen area;

[0010] In the stable phase, the filter switching element, the light source, and the image processing element switch synchronously.

[0011] In one possible implementation, after controlling the filter switching element to start in response to the control command, the method further includes:

[0012] If it is determined that the filter switching element has entered the stable phase, a first start command and a second start command are sent to the image processing element and the light source, respectively, to instruct the filter switching element to enter the stable phase.

[0013] In one possible implementation, determining that the filter switching element has entered a stable phase includes:

[0014] After the filter switching element has been activated for a preset period of time, it is determined that the filter switching element has entered a stable phase.

[0015] Alternatively, monitor the rotational speed of the filter switching element, and determine that the filter switching element has entered a stable phase after the rotational speed of the filter switching element reaches and remains at a preset rotational speed.

[0016] Alternatively, the filter switching element may further include an optical coupler detection component and a body. An initial position is set on the body. The initial position is monitored by the optical coupler detection component. When the initial position is detected to repeat at a preset frequency, it is determined that the filter switching element has entered a stable phase.

[0017] In one possible implementation, the filter switching element further includes a rotation drive element and a body, wherein the at least two filter regions are respectively disposed in different regions of the body, and the rotation drive element is used to drive the body to rotate by rotation to switch the filter regions corresponding to the light source main axis with different sampling periods; the filter switching element maintains a uniform rotation speed during the stable phase.

[0018] In one possible implementation, after receiving the control command for simultaneous display of multiple viewing modes, the method further includes:

[0019] In response to the control command, the image processing element exits the standby mode and begins switching the image processing parameters corresponding to each sampling period after a first preset time.

[0020] In response to the control command, the light source is controlled to start switching the light source emission combination corresponding to different sampling periods after the second preset time.

[0021] In one possible implementation, in the standby mode, the image processing element is set not to output an image.

[0022] In one possible implementation, the method further includes:

[0023] In response to the user's parameter setting instructions, the light source information and the image processing parameters are determined.

[0024] In one possible implementation, the light source information includes at least the emission sequence and the light intensity ratio of each light source emission combination; the image processing parameters include at least color correction matrix parameters, which are used to convert the values ​​of the red, green, and blue channels.

[0025] In one possible implementation, the method further includes:

[0026] The image processing element acquires exposure data of an image generated based on at least a portion of the sampling period, and determines the brightness of the adjusted illumination light based on the exposure data;

[0027] The adjusted brightness of the illumination light is sent to the light source component so that when switching the light source emission combination corresponding to each sampling period, the illumination light is output with the adjusted brightness.

[0028] Secondly, embodiments of this application provide an endoscope system, which includes a processor, a light source assembly, an image processing element, and a display device. The light source assembly includes multiple light sources and a filter switching element disposed on the light emission path of the light sources. The filter switching element includes at least two filter regions. The light emitted by the light sources is combined or passes through the corresponding filter regions individually to output illumination light with a corresponding spectrum. The image processing element is used to acquire the light signal after the illumination light illuminates the area to be examined, and to retrieve corresponding image processing parameters for processing to generate an image. The display device includes at least multiple screen areas.

[0029] The processor is used to execute the first aspect and / or various possible control methods described above.

[0030] The control method and endoscope system for simultaneous display of multiple viewing modes in the embodiments of this application provide that, by coordinating and controlling the switching synchronization of filter switching elements, light sources and image processing elements, it is beneficial to achieve the real-time and synchronous display of multiple viewing modes in the simultaneous display of multiple images, thereby providing users with a real-time, consistent and ghost-free simultaneous observation experience. Attached Figure Description

[0031] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0032] Figure 1 This is a schematic diagram of the components of an endoscope system provided in an embodiment of this application;

[0033] Figure 2 A schematic diagram of a spectral curve under white light observation mode provided in an embodiment of this application;

[0034] Figure 3 A schematic diagram of the spectral curve under a special optical observation mode NEI provided in this application embodiment;

[0035] Figure 4 A schematic diagram of the spectral curve under another special optical observation mode REI provided in the embodiments of this application;

[0036] Figure 5 A comparative schematic diagram of spectral bands provided for an embodiment of this application;

[0037] Figure 6A A schematic diagram of the distribution of the filtering region on the filtering switching element provided in this application embodiment;

[0038] Figure 6B This is a schematic diagram of the structure of a color wheel provided in an embodiment of this application;

[0039] Figure 6C A schematic diagram showing the distribution of another filtering region on a filtering switching element, provided as an embodiment of this application;

[0040] Figure 7 A schematic diagram of a dual-light, dual-screen interface provided for an embodiment of this application;

[0041] Figure 8 A flowchart illustrating a control method for simultaneous display of multiple viewing modes of an endoscope system provided in an embodiment of this application;

[0042] Figure 9A schematic diagram illustrating the correspondence between a filter switching element, a light source, and an image processing element in each acquisition cycle, provided for an embodiment of this application;

[0043] Figure 10 A flowchart illustrating a control method for simultaneously displaying multiple viewing modes of an endoscope system, as provided in an embodiment of this application.

[0044] Figure 11 A flowchart illustrating a control method for simultaneously displaying multiple viewing modes of an endoscope system, as provided in an embodiment of this application.

[0045] Figure 12 This is an interactive schematic diagram illustrating the simultaneous display of multiple screens in an endoscope system, providing an embodiment of this application.

[0046] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0047] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0048] An endoscope system is a diagnostic instrument integrating traditional optics, ergonomics, precision mechanics, modern electronics, mathematics, and software technologies. It is used to acquire images of the examined area within the body to assist medical personnel in diagnosis. Images of the examined area under different spectral illumination conditions can reflect different tissue characteristics. For example, in white light observation mode, i.e., under white light illumination, the generated image colors are more consistent with human eye perception, better reproducing the colors of the observed object, facilitating doctors' observation and judgment of the overall tissue condition. However, white light does not sufficiently highlight blood vessels or lesions, making it difficult to detect specific tissues such as diseased tissues or bleeding points. In special light observation mode, i.e., if a narrow band of light with a specific spectrum is used to illuminate the observed object, the contrast between specific tissues and normal tissues can be enhanced, thus highlighting certain features. For example, under certain special light observation modes, superficial blood vessels can be highlighted; under other special light observation modes, bleeding points and deep blood vessels can be highlighted. However, prolonged use of special light observation modes may cause user discomfort and is not conducive to distinguishing the tissue colors of different observed objects.

[0049] To avoid discomfort caused by prolonged use of special light and to facilitate doctors' comparison of human or animal tissues under different light spectra for accurate identification of lesions, endoscopic systems can support multiple observation modes and simultaneously display images from various modes. For example, an endoscopic system can simultaneously support white light observation mode and a special light observation mode, or multiple special light observation modes, or both, and display multiple images simultaneously.

[0050] When simultaneously displaying images in white light observation mode and special light observation mode, to ensure there is no noticeable delay or visual tearing between the two images, it is necessary not only to rapidly switch the light source to illuminate the detected area according to a preset timing sequence, but also to synchronously switch the filter area in the filter switching element according to a preset timing sequence to filter the white light and special light emission combinations accordingly, forming white light and special light spectra. Finally, the image processing parameters of the image processing element must be synchronously switched according to a preset timing sequence. Similarly, simultaneously displaying images under multiple special light observation modes requires synchronous switching of the light source, filter switching element, and image processing element.

[0051] However, since filter switching elements are typically driven by rotational drive components (such as motors), given the characteristics of motor devices, it takes a certain amount of time for them to start rotating and stabilize at a specific speed. In other words, it also takes a certain amount of time for the rotational speed of the filter switching element to reach a stable state. When the filter switching element operates unstablely, the frequency at which it switches the filter area becomes unstable, making it difficult to control its synchronization with the light source and image processing components.

[0052] To this end, this application provides a control method for simultaneous display of multiple screens in multiple observation modes of an endoscope system. The control method includes: receiving a control command for simultaneous display of multiple screens in multiple observation modes; responding to the control command by activating a filter switching element and switching target filter regions for each sampling period during a stable phase, wherein the target filter region is a filter region correspondingly set on the main axis of the light source; responding to the control command by controlling the light source to switch the light source emission combination corresponding to each sampling period after the filter switching element enters the stable phase; responding to the control command by controlling an image processing element to enter a standby mode, and exiting the standby mode and starting to switch image processing parameters corresponding to each sampling period after the filter switching element enters the stable phase; receiving images generated by the image processing element based on at least a portion of the sampling periods and displaying them in corresponding screen areas; wherein the filter switching element, the light source, and the image processing element switch synchronously during the stable phase.

[0053] The control method for simultaneous display of multiple screens in multiple observation modes of an endoscope system provided in this application embodiment controls the filter switching element, the light source and the image processing element in precise and coordinated synchronous control. This ensures that the operation results of the three elements are completely matched in each sampling cycle, reducing the generation of unqualified sampling cycles. As a result, images can be captured and processed at a higher effective frame rate, thereby reducing the image update delay caused by the user moving the endoscope system and ensuring that the screen follows the operation in real time. At the same time, generating multiple screens based on mutually matched sampling cycles also eliminates the timing misalignment between screens, ensuring the synchronicity of simultaneous display and providing users with a real-time and consistent multi-screen simultaneous observation experience.

[0054] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0055] It should be noted that the control method for simultaneous display of multiple viewing modes of the endoscope system provided in this application embodiment is applied to the endoscope system, and more specifically, the control method is applied to the processor of the endoscope system. First, the structure of the endoscope system and its imaging principle will be explained.

[0056] Figure 1 This is a schematic diagram of the components of an endoscope system provided in an embodiment of this application, such as... Figure 1 As shown, the endoscope system 100 includes a processor 10, a light source assembly 20, an image processing element 30, and a display device 40.

[0057] The light source assembly includes multiple light sources and a filter switching element disposed on the light output path of the light source. The filter switching element includes at least two filter areas. The light emitted by the light source is combined or passes through the corresponding filter areas individually to output illumination light with a corresponding spectrum. The image processing element is used to acquire the light signal after the illumination light illuminates the area to be inspected, and retrieve the corresponding image processing parameters for processing to generate an image. The display device includes at least multiple screen areas.

[0058] For example, the light source component includes multiple light sources. In practical applications, multiple different types of light sources can be set according to actual needs. For instance, to output white light, a single white light source can be set, or multiple light sources of different colors can be used to synthesize white light; to output special light with different spectra, multiple light sources of different colors can be set to synthesize it. When each light source operates in different switching states, different light source emission combinations are formed.

[0059] For example, the light source of this application can be an LED lamp to quickly turn the LED on and off. Specifically, five LED lamps can be used, namely red (R), green (G), blue (B), amber (A), and blue-violet (UV) LED lamps. The five LED lamps are arranged in a set order, and combined with the corresponding collimating lens and converging lens, a specific spectrum can be synthesized. For example, when synthesizing white light, all LEDs need to be lit. In white light observation mode, the light source emission combination consists of red, green, blue, amber, and blue-violet LEDs. When synthesizing special light, only some LEDs need to be lit. For instance, to synthesize special light NEI, only green and blue-violet LEDs need to be lit. Similarly, to synthesize special light REI, only red, green, and amber LEDs need to be lit. This means the light source emission combination in REI is also red, green, and amber LEDs. It's understandable that the light source emission combination also includes the case where a single light source is activated, such as an independently set white light source that is activated alone to output white light.

[0060] For example, Figure 2 This is a schematic diagram of a spectral curve under white light observation mode provided in an embodiment of this application. Figure 3 This is a schematic diagram of the spectral curve under a special optical observation mode NEI provided in an embodiment of this application. Figure 4 This is a schematic diagram of the spectral curve under REI, another special optical observation mode provided in this application embodiment. Figures 2 to 4 In the schematic diagram of the spectral curve shown, the horizontal axis represents wavelength, and the vertical axis represents optical radiation power. Among these, Figure 2 The white light in the image is composed of red, amber, green, blue, and blue-violet light. Figure 3 The special light in the device is composed of blue-violet and green light, which can be used to highlight blood vessels. Blue-violet light is good for highlighting superficial blood vessels, while green light is good for highlighting deep blood vessels. Figure 4 The special light in the image is composed of green, amber, and red light, which can be used to highlight bleeding points and deep blood vessels.

[0061] To output illumination light of a specific spectrum, it depends not only on the combination of light sources emitted but also on the intensity ratio between different light sources. This is measured by the total luminous flux (brightness) of the synthesized white light. For example, the total luminous flux needs to be distributed across 5 LEDs so that the sum of the luminous flux of the 5 LEDs equals the total luminous flux of the white light. In some embodiments, the calculation process for the brightness proportion of each lamp is as follows: First, the chromaticity coordinates of the white light can be calculated based on the color temperature of the white light (which can be set) using approximate formulas such as theoretical integration or by looking up tables / online tools. , This application does not impose limitations on the embodiments thereof. For example, when the color temperature is 6500K, , When the color temperature is 5500K, , Then, calculate the brightness proportions of red, green, and blue light using the following formula:

[0062] (1)

[0063] Among them, in formula (1) , , These represent the proportions of luminous flux from red, green, and blue light to the total luminous flux, respectively. , Represents the chromaticity coordinates of the red light. , This represents the chromaticity coordinates of the green light. , This represents the chromaticity coordinates of the blue light. Similarly, the chromaticity coordinates of red, green, and blue light can also be calculated from the corresponding spectral curves, and this application does not impose any limitations on this.

[0064] Once the proportions of red, green, and blue light are obtained, the red and blue light can be further subdivided. The luminous flux of red light can be allocated to red lamps and amber lamps, and the luminous flux of blue light can be allocated to blue lamps and blue-violet lamps. Based on the above, assuming the required synthesized luminous flux is... If the light is white, then the luminous flux of red light is If red lights account for a certain percentage Then the proportion of amber lamps After further decomposition, the luminous flux of the red light is: The luminous flux of the amber lamp is Similarly, the luminous flux of blue light is If the proportion of blue lights The proportion of blue and purple lights After further decomposition, the luminous flux of the blue lamp is The luminous flux of the blue-violet lamp is .

[0065] In this embodiment, the luminous flux ratio of each light source can be calculated in advance. For example, the luminous flux allocation ratio for white light can be red:amber:green:blue:blue-violet = 20%:10%:40%:20%:10%; the luminous flux allocation ratio for special light NEI can be blue-violet:green = 1:99; and the luminous flux allocation ratio for special light REI can be green:amber:red = 38:51:11. In practical applications, the luminous flux ratio between each light source can also be adjusted according to actual needs, and this embodiment does not impose any limitations.

[0066] For example, Figure 5 This is a comparative schematic diagram of spectral bands provided for an embodiment of this application. For example... Figure 5 As shown, the green light spectrum curve A without a filter has a wider band, while the green light spectrum curve B with a filter has a narrower band.

[0067] In order to better highlight the tissue characteristics of the detected area under different spectral illumination conditions, the embodiments of this application generally do not deliberately limit its spectral range in white light observation mode in order to maintain a wide band coverage; while in special light observation mode, it is usually necessary to use a specific narrow band filter to limit the full-band light emitted by the LED light source to a specific narrow band spectral range, thereby highlighting the characteristics of the target tissue.

[0068] Based on this, in this embodiment, a filter switching element is also provided on the light emission path of the light source. A filter switching element is an optical device used to switch between different filter regions to allow light of different wavelengths to pass through at different times. The filter switching element includes at least two filter regions, such as two, three, four, or even more. Each filter region, depending on the filter it contains, is used to process light emitted by different combinations of light sources, thereby outputting illumination light with a corresponding spectrum. For example, the filter region for white light may have a broadband white light filter or no filter (empty); the filter region for special light NEI may have an NEI filter; and the filter region for special light REI may have a REI filter. In some embodiments, to ensure the rotational balance and lifespan of the filter switching element, the two filter regions are arranged symmetrically, and / or, the number of filter regions is arranged in pairs.

[0069] In some embodiments, the filter switching element includes two filter regions, which are respectively an empty filter region and a REI filter region, or an empty filter region and a NEI filter region, or a NEI filter region and a REI filter region. Similarly, the empty filter region in the above embodiments can be replaced with a region provided with a broadband white light filter. Preferably, the two filter regions are symmetrically arranged, so that the center of gravity of the filter switching element is balanced, which is beneficial to rotational stability and extended service life.

[0070] In some embodiments, the filter switching element includes three filter regions: an empty filter region, a REI filter region, and a NEI filter region. Similarly, the empty filter region in the above embodiments can be replaced with a region provided with a broadband white light filter.

[0071] In some embodiments, the filter switching element includes four filter regions: an empty filter region, a REI filter region, a NEI filter region, and an arbitrary filter region. The arbitrary filter region can be configured with filters corresponding to other special light observation modes; alternatively, it can be any one of the empty, REI, and NEI filter regions. Preferably, two identical filter regions are symmetrically arranged, which helps maintain the balance of the filter switching element's center of gravity, ensures stable rotation, and extends its service life. Similarly, the empty filter region in the above embodiment can be replaced with a region configured with a broadband white light filter.

[0072] For example, Figure 6A This is a schematic diagram illustrating the distribution of the filtering region on a filtering switching element, as provided in an embodiment of this application. Figure 6A As shown, the filter switching element can use a disk-shaped structure, with two filters arranged on either side of the disk at symmetrical positions, and the other two symmetrical positions left empty without filters. The empty positions 1 and 2 are used for two special light filters for white light filtering: one is the NEI filter, used to filter the light in the NEI special light observation mode, and the other is the REI filter, used to filter the light in the REI special light observation mode.

[0073] In some embodiments, the filter switching element may be a color wheel. For example, Figure 6B This is a schematic diagram of a color wheel provided in an embodiment of this application. Figure 6B As shown, the color wheel 11 includes two notches 112 arranged circumferentially along the color wheel 11, a first filter 111A, and a second filter 111B. The first filter 111A is located in a first light-transmitting region 114, and the second filter 111B is located in a second light-transmitting region 115. The notches 112 are used to filter white light. The first filter 111A can be a NEI filter, and the second filter 111B can be a REI filter. For example, Figure 6C This is a schematic diagram illustrating the distribution of another filtering region on a filtering switching element, as provided in an embodiment of this application. Figure 6C As shown, the filter switching element can also consist of only two filter areas, with the blank space and NEI filter arranged symmetrically.

[0074] For example, to achieve rapid switching between filters and blank positions, or switching between different filters, the filter switching element may further include components such as a rotation drive element, a synchronous pulley, and a rotating shaft. The rotation drive element is a device capable of providing rotational power, such as a stepper motor or servo motor, which receives electrical signals from the processor and converts them into precise angular displacement or continuous rotation. The body of the filter switching element is a mechanical structure that carries the filter, typically a disc-shaped component called a "filter wheel." Optionally, the rotation drive element and the filter wheel can be coaxially or non-coaxially arranged; this embodiment does not impose any limitations on this. For example, the body of the filter switching element can be combined with the synchronous pulley and fixed on a rotating shaft to form a synchronously rotating whole. The rotation drive element drives the synchronous pulley, which in turn drives the rotating shaft to rotate, ultimately causing the filter wheel to rotate, thereby quickly and accurately switching the specified filter or blank position into the optical path to meet the filtering requirements of different sampling periods.

[0075] For example, when the illumination light output from the light source passes through the filter area and illuminates the area to be inspected, the image processing element can acquire the light signal after the illumination light illuminates the area to be inspected, and retrieve the corresponding image processing parameters to process and generate an image, which is then displayed in the corresponding screen area of ​​the display device. The image processing element includes an image sensor, also known as a photosensitive element, which is a device that converts optical signals into electrical signals. It is used to acquire the light signal after the illumination light illuminates the area to be inspected; the light signal is also known as photosensitive data. The image processing element also includes an image processor (IP), also known as an image signal processor (ISP) or image processor, which is a chip integrated into a camera or other device with shooting capabilities. It is used for image signal processing. In this embodiment, the image processor is used to retrieve the corresponding image processing parameters (hereinafter referred to as: ISP parameters) to process the electrical signal output by the image sensor to generate an image.

[0076] Since directly using the light-sensing data of an image sensor to represent color will be very different from the human eye, in order to obtain the same RGB data for the same color even when using different image sensors, and to make it consistent with or close to the color perceived by the human eye, the light-sensing data of the image sensor needs to be transformed in some way to obtain a result that is similar to the color perceived by the human eye.

[0077] Understandably, the ISP parameters differ across observation modes. For example, the ISP parameters in white light observation mode aim to reproduce the true colors of the observed object as accurately as possible, while the ISP parameters in special light observation mode aim to highlight lesions or suspected lesions, requiring the use of different Color Correction Matrix (CCM) conversions. Therefore, the ISP parameters in white light observation mode differ from those in special light observation mode.

[0078] For example, for an image corresponding to white light, it can be achieved using the following color correction matrix formula (2), where, , , These are the values ​​of the red, green, and blue channels before conversion (i.e., the light-sensitive data from the image sensor). , , It is the converted value. These are the preset element values ​​of the transformation matrix. After transformation, the resulting colors will be closer to what the human eye sees.

[0079] (2)

[0080] For images corresponding to the special light NEI, since there is no red light in the special light NEI, the R channel (red channel) of each pixel after imaging is basically noise, and only the G channel (green channel) and B channel (blue channel) values ​​are meaningful. Therefore, the values ​​of G and B can be redistributed to the three channels R, G, and B by the following color correction matrix formula (3).

[0081] (3)

[0082] Among them, in formula (3) , These are pre-defined element values. After transformation using the above formula (3), the blood vessels will appear brown or dark gray, and the mucous membrane will appear light green.

[0083] For images corresponding to special light REI, since there is no blue light in the special light REI, the B channel (blue channel) of each pixel after imaging is basically noise, and only the G channel (green channel) and R channel (red channel) values ​​are meaningful. Therefore, the values ​​of G and R can be redistributed to the three channels R, G, and B by the following color correction matrix formula (4).

[0084] (4)

[0085] Among them, in formula (4) , , , These are pre-defined element values. After transformation using the above formula (4), the bleeding points or deep blood vessels appear dark red, while the mucosa appears yellowish-brown.

[0086] Typically, to quickly retrieve the corresponding image processing parameters for image generation, the corresponding image processing parameters (also known as default image processing parameters) can be pre-determined based on the user-selected observation mode before responding to control commands, so that subsequent parameters can be prioritized. In some embodiments, the image processing parameters can be adjusted according to the user's needs. For example, in a scenario where multiple observation modes are displayed simultaneously, the user can trigger parameter modification through the interactive interface of the endoscope system's application software. The adjusted image processing parameters (also known as custom image processing parameters) can be directly used in subsequent image processing steps in this scenario, which helps meet the observation needs of different users. In some embodiments, the user can adjust the default image processing parameters through the interactive interface of the endoscope system's application software.

[0087] The image generated by the image processing element is displayed on the corresponding screen area of ​​the display device. In the embodiments of this application, the display device includes at least multiple screen areas, such as two, three, four, or even more. For example, in a scenario where multiple screens are simultaneously displayed in various observation modes of an endoscope system, including white light observation mode and special light NEI observation mode, two screen areas will be displayed simultaneously on the display device, respectively displaying the corresponding images of the two observation modes. (See reference...) Figure 7 For example, in a scenario where an endoscope system simultaneously displays multiple viewing modes—white light observation mode, NEI special light observation mode, and REI special light observation mode—the display device will simultaneously show three screen areas, each displaying the corresponding image for one of the three observation modes. Similarly, in a scenario where multiple viewing modes are displayed simultaneously, the screen areas corresponding to each mode may not be displayed simultaneously. The display can be determined based on user commands or pre-defined display methods, such as automatic alternation. In essence, the aforementioned simultaneous display means that the endoscope system has the capability to display simultaneously, but the display can be adjusted according to actual needs.

[0088] Optionally, the display screen of the display device can be a touch screen, allowing users to directly input control commands through the display screen to achieve human-computer interaction. For example, when the user inputs the corresponding control command, the display screen can display menu switching or pop up parameter setting windows, so that the user can further set parameters and ultimately control various working processes of the endoscope system through the display screen.

[0089] Based on the structure and imaging principle of the aforementioned endoscope system, it is clear that achieving simultaneous display of multiple images across various observation modes requires synchronization between the filter switching element, the light source, and the image processing element. Specifically, within a given time period, the light source is adjusted to a specific emission combination, the filter switching element switches to the corresponding filter area, the image sensor acquires photosensitive data, and the image processor switches to the corresponding image processing parameters for image processing. This time period can be understood as one image sampling cycle, i.e., the reciprocal of the frame rate.

[0090] To ensure the real-time and synchronous nature of image display, the frame rate of image processing cannot be lower than a specific value, and a higher frame rate is more beneficial to image quality. Especially in scenarios where multiple screens are displayed simultaneously under various viewing modes, each viewing mode requires a portion of the sampling period, and the frame rate under each viewing mode must meet preset requirements to guarantee the display effect of the corresponding screen. Therefore, each sampling period needs to be utilized as effectively as possible. If, during at least a portion of a sampling period, the light source emission combination, filter, or image processing parameters are mismatched, the image generated in that sampling period will be unacceptable and must be discarded. Discarding too many sampling periods will correspondingly affect the frame rate, thus affecting the image display effect, especially in scenarios where multiple screens are displayed simultaneously under various viewing modes. Therefore, this application needs to ensure that the light source emission combination, filter, or image processing parameters are matched throughout the entire period of each sampling period. This requires that the three are matched for at least a portion of the sampling period (i.e., the effective sampling period for outputting images for display), and that the switching of the three is synchronized at the beginning of that sampling period.

[0091] In some embodiments, to facilitate processor switching control, the processor pre-sets the switching sequence for multiple viewing modes simultaneously displayed in multiple images. Based on the switching sequence of the filter switching element, the switching sequence of the light source, and the switching sequence of the image processor, the pairing relationships of the filter, light source emission combination, and image processing parameters are pre-set sequentially in each sampling cycle. Furthermore, the sampling cycle for the switching sequence of these three elements is the same; the processor only needs to synchronize the switching start points of the three elements. After the three elements switch synchronously at the same moment (i.e., the start point) in the first sampling cycle, they switch sequentially in each subsequent sampling cycle according to their switching sequence, thus achieving synchronous switching of the three elements throughout the entire scene. At this point, the switching frequencies of the three elements are consistent, which is beneficial for the processor's switching control.

[0092] In some embodiments, depending on the switching timing of the three components, the processor may switch only at least one of the filter switching element, the light source, and the image processor, not in every sampling period. That is, the switching frequencies of the three components are different, for example, they may be set in a multiple relationship. It is only necessary that the pairing relationship of the filter, the light source emission combination, and the image processing parameters conforms to the preset settings in at least a portion of the effective sampling periods in which the image processor needs to output the image.

[0093] The following section, in conjunction with the structure and imaging principle of the aforementioned endoscope system, provides a detailed explanation of the control method for simultaneous display of multiple screens in various observation modes of the endoscope system provided in this application embodiment.

[0094] For example, Figure 8 This is a flowchart illustrating a control method for simultaneously displaying multiple viewing modes of an endoscope system, as provided in an embodiment of this application. Figure 8 As shown in the embodiments of this application, the control method for simultaneous display of multiple screens in an endoscope system with multiple observation modes may include:

[0095] S801 receives control commands for simultaneous display of multiple screens in various viewing modes.

[0096] For example, control commands for simultaneous display of multiple viewing modes can be generated through a user interface (such as a button or touchscreen) or automatically triggered (such as a system timer), and received by the processor of the endoscope system. After parsing the commands, the processor coordinates the operation of subsequent components.

[0097] In some embodiments, the endoscope system supports any combination of three modes: white light observation mode, special light observation mode (NEI), and special light observation mode (REI). Users can freely choose the combination. By clicking the selected observation mode in the interactive interface and then clicking buttons such as "Enter" or "Start," control commands can be generated and sent to the processor of the endoscope system. For example, users can select from controls such as "white light + special light (NEI)," "white light + special light (REI)," "special light (NEI) + special light (REI)," or "white light + special light (NEI) + special light (REI)."

[0098] In some embodiments, the endoscope system may also support other observation modes, enabling more mode combinations; this application does not impose limitations. Through control commands, the processor of the endoscope system can flexibly activate a multi-screen simultaneous display mode for various observation modes, adapting to different observation needs and improving operational convenience and adaptability.

[0099] In some embodiments, the processor is further configured to: determine light source information and image processing parameters in response to a user's parameter setting instruction.

[0100] For example, the endoscopic system can also support users setting or adjusting light source information and / or image processing parameters through an interactive interface. Specifically, after the user manually sets the light source information and image processing parameters through the interactive interface, they send a parameter setting command to the processor. The processor can then determine the new light source information and image processing parameters based on the user's parameter setting command. This setting or adjustment can occur during the operation of a multi-screen simultaneous display of multiple observation modes, after exiting a mode, or before starting a new mode. In the case of adjustments made during mode operation, the adjusted light source information and / or image processing parameters can be used in subsequent sampling cycles.

[0101] The light source information may include the light source switching sequence and the light intensity ratio of each light source emission combination. In some embodiments, the light source information also includes light intensity information of the illuminating light. It is understood that at least part of the light source information is a default setting, or it may be sent in real time with control commands.

[0102] In this context, "light source switching timing" refers to the switching of light source emission combinations based on the corresponding light source emission combinations in a multi-screen simultaneous display mode with multiple observation modes. For example, switching to a white LED light combination in the first sampling cycle and switching to a special light NEI LED light combination in the second sampling cycle. Specifically, the processor can control the light source to switch in each sampling cycle, simplifying the switching control. Furthermore, the processor can control the light source to switch during sampling cycles where the light source emission combination needs to be adjusted.

[0103] In some embodiments, image processing parameters include color correction matrix parameters used to convert the values ​​of the red, green, and blue channels. In some embodiments, image processing parameters also include gain, white balance, contrast, sharpness, or noise filtering, etc., which are not limited in this application embodiment. It is understood that at least some of the image processing parameters are set by default, or they can be issued in real time according to user control commands.

[0104] S802 responds to control commands, controls the filter switching element to start, and switches the target filter area for each sampling period during the stabilization phase.

[0105] The target filter region is the filter region set on the main axis of the light source for each sampling period. Specifically, according to the preset switching sequence of the filter switching elements, the processor determines the filter for each sampling period, and the filter region where the corresponding filter is located needs to be moved to the main axis of the light source in each sampling period. At this time, the filter region is called the target filter region.

[0106] The filter switching element is positioned along the light output path of the light source. During each sampling cycle, the target filter area is moved to a position directly opposite the main axis of the light source, which helps to maximize light utilization and reduce light loss.

[0107] The sampling period refers to the time interval for acquiring light signals, also known as the frame interval or frame length. In some embodiments, the sampling period is determined based on the frame rate index within the operating range of components such as the graphics processor in the endoscope system. For example, to achieve a frame rate of 30 frames per second or higher for the image output of each observation mode, the sampling period needs to be determined by comprehensively considering factors such as the number of observation modes and the number of frames to be lost. Furthermore, the switching sequence of filters, light sources, and image sensing parameters is then set based on the sampling period.

[0108] In some embodiments, the switching timing of the light source, the switching timing of the filter switching element, and the switching timing of the image processor in all observation modes adopt a uniform sampling period. This is beneficial for the processor to control the filter switching element, the light source, and the image processor to operate synchronously at a fixed and mutually matched time rhythm.

[0109] After receiving the control command, the processor, in response, first sends a start command to the filter switching element to control its activation, causing it to begin rotating, and then enters the stabilization phase. It is understood that the starting point of the switching sequence of the filter switching element described above is set in the stabilization phase. During the period from the activation of the filter switching element to entering the stabilization phase, as the filter switching element rotates, the filtering area on the light source's main axis also changes. However, no image is output or displayed during this phase; therefore, the filter switching at this time has no substantial impact, and the filtering area is not the target filtering area defined in this paper.

[0110] The stabilization phase refers to the stage where the filter switching element reaches a stable operating state. During the stabilization phase, the filter switching element moves in a preset and fixed sequence and frequency, thereby stably switching the filter areas set on the main axis of the light source. According to the preset switching sequence, in each sampling period, the target filter area only allows illumination light of a specific wavelength to pass through, thereby achieving the filtering of the light emitted by the light source combination in that sampling period.

[0111] In some embodiments, the filter switching element may include a rotation drive element and a body, with at least two filter regions respectively disposed in different regions of the body. The rotation drive element is used to drive the body to rotate by rotation, so as to switch the target filter region corresponding to the light source main axis with different sampling periods. The filter switching element maintains a uniform rotation speed during the stable phase.

[0112] Specifically, the stabilization phase of the filter switching element can be understood as the phase in which the filter switching element maintains a uniform rotation speed.

[0113] When it is necessary to switch the target filter area, the processor sends a command to the rotation drive element, which then starts and rotates the main body. Due to the characteristics of the rotation drive element, it needs to accelerate first before it can enter a constant speed rotation state. Only then can the filter switching element enter the stable phase. In the stable phase, the main body rotates at a preset speed, which is beneficial for accurately switching the target filter area for each sampling cycle according to the preset switching sequence.

[0114] S803 responds to control commands and controls the light source to start switching the light source emission combination corresponding to each sampling period after the filter switching element enters the stable stage.

[0115] For example, the light source emission combinations are different for different observation modes. For instance, the light source emission combination in white light observation mode is red LED, green LED, blue LED, amber LED, and blue-violet LED; the light source emission combination in special light observation mode NEI is green LED and blue-violet LED; and the light source emission combination in special light observation mode REI is red LED, green LED, and amber LED.

[0116] In this embodiment, the processor needs to control the synchronous switching of the light source, filters, and image processing parameters to match the target filtering area, light source emission combination, and image processing parameters for each sampling period. Specifically, after detecting that the filter switching element has entered a stable phase, the processor determines the synchronous switching start point and controls the light source and image processor at the start point to start switching the light source emission combination and image processing parameters corresponding to each sampling period according to the preset light source switching sequence and image processing parameter switching sequence.

[0117] The switching sequence of filters, light sources, and image processing parameters can be characterized by a common switching sequence, which can be found in [reference needed]. Figure 9 , Figure 9 The relationship shown can be understood as the common switching timing of the three. In some embodiments, combined with Figure 6A and Figure 9As shown, specifically, the target filter area corresponding to the first sampling period is blank position 1, the target filter area corresponding to the second sampling period is the NEI filter, the target filter area corresponding to the third sampling period is blank position 2, the target filter area corresponding to the fourth sampling period is the REI filter, the target filter area corresponding to the fifth sampling period is blank position 1, and so on. Therefore, the light source emission combination that the light source should perform in the first to fifth sampling periods is, in order, white light, special light NEI, white light, special light REI, and white light, and so on.

[0118] In some embodiments, after the processor responds to a control command and controls the start-up of the filter switching element, it determines whether the filter switching element has entered a stable phase. After determining that the filter switching element has entered a stable phase, the processor can trigger a first start command to the light source to instruct the light source to begin entering a first sampling period and control the switching of the light source emission combination corresponding to the first sampling period. This facilitates the synchronization of switching between the light source and the filter switching element. Specifically, the processor can determine whether the filter switching element has entered a stable phase by monitoring the rotational speed of the filter switching element; for example, when the processor detects that the rotational speed of the filter switching element reaches and remains at a preset rotational speed, it determines that the filter switching element has entered a stable phase. Specifically, the processor can directly obtain the rotational speed using a rotational speed sensor installed on the filter switching element, or indirectly calculate the rotational speed using an optocoupler sensor installed on the filter switching element. Other sensors can also be used, and this application does not limit this.

[0119] In some embodiments, the processor can also automatically trigger a first start command to the light source after a preset time period following the start of the filter switching element. Specifically, the preset time period is a time value pre-set through experiments or calculations, which covers the entire transition process required from the moment the filter switching element receives the start command, to accelerating to the target stable rotational speed, and eliminating initial vibrations. Simultaneously or after issuing the start command to the filter switching element, the processor starts a timer. When the timer reaches the preset time period, the processor determines that the filter switching element has entered a stable phase and then triggers subsequent synchronous operations such as light source switching and the image processing element exiting standby mode.

[0120] In some embodiments, the optical coupler detection assembly consists of an optical emitter and an optical receiver facing each other, one of which is disposed on the main body. When the optical path is blocked, the output level changes. The processor can determine the speed of the filter switching element using this level signal. Additionally, the processor can also determine the initial position of the filter switching element using this level signal, which is beneficial for determining the starting point of switching based on the switching sequence after the filter switching element enters a stable phase. Specifically, during the rotation of the filter switching element's main body, a pulse signal is generated each time the initial position marker passes through the optical coupler detection assembly. The processor monitors the period or frequency of this pulse signal. When the time interval between multiple consecutive pulses is stable and its frequency equals the calculated preset frequency, it can be determined that the filter switching assembly has entered a stable phase of uniform rotation. Simultaneously, by setting and monitoring the initial position, not only is the rotational speed stable, but a precise physical position reference signal is also provided. This provides an absolute physical reference point for the starting point of each synchronous switching, which can be used to calibrate and clarify the position of the main body, and also prevent misalignment of the filter area due to accumulated errors, ensuring that each filter is precisely aligned with the optical path.

[0121] In some embodiments, the processor can directly control the light source to begin switching the light source emission combination corresponding to different sampling periods after a first preset time. The first preset time is the duration for which the control filter switching element reaches a stable state after activation, and can be obtained through pre-calibration; this embodiment does not impose limitations on this.

[0122] Specifically, the processor receives control commands for simultaneous display of multiple screens in various viewing modes. In response to these commands, after activating the filter switching element, it begins timing. When the timing reaches a first preset time, the processor immediately sends a command to the light source to initiate a sequence of synchronously switching the light source emission combinations according to the sampling period. This method features simpler control logic.

[0123] Understandably, during the period from the activation of the filter switching element to the start of the synchronous switching of the light source, there is no image output and display, and the light source can be in a turned-off state. In some embodiments, the light source can provide illumination according to the default light source emission combination, which is not limited here. Only after the start of the synchronous switching can the light source switch to emit light according to the preset switching sequence in the first sampling period, and perform the switching sequentially.

[0124] S804 responds to control commands, controls the image processing element to enter standby mode, and after the filter switching element enters a stable phase, exits standby mode and begins switching the image processing parameters corresponding to each sampling period.

[0125] In some embodiments, in standby mode, the image processing element does not acquire the light signal after the illumination light shines on the inspected area; for example, the image sensor does not output photosensitive data to the image processor. In some embodiments, the image processing element only acquires the light signal after the illumination light shines on the inspected area, but does not retrieve the corresponding image processing parameters for processing or generate an image; for example, after the image sensor receives the photosensitive data, it does not perform the image processing step; this step helps to save the computing power of the image sensor. In some embodiments, the image processing element acquires the light signal after the illumination light shines on the inspected area and retrieves the corresponding image processing parameters for processing to generate an image, but does not output the generated image; for example, after the image sensor receives the photosensitive data and performs image processing, it does not output the image to the display device.

[0126] The various embodiments of the image processing element described above in standby mode can prevent images from being displayed in standby mode, thereby affecting image display quality. This is because the filter switching element has not yet entered a stable stage at this time, and the filter and light source cannot switch at a stable frequency, and the corresponding switching sequence cannot be executed. Therefore, the processor controls the image processing element to be in standby mode, which helps to prevent unstable or erroneous images from being output and displayed.

[0127] In this embodiment, the processor, responding to a control command, first controls the image processing element to enter standby mode, where it waits for the filter switching element to enter a stable phase. Specifically, the processor, responding to the control command, sends a notification to the image processing element to enter standby mode, thus controlling the image processing element to enter standby mode; and when the filter switching element enters a stable phase and the starting time of synchronous switching is clearly defined, the processor sends a notification to the image processing element to exit standby mode, instructing the image processing element to exit standby mode and begin switching image processing parameters according to a preset switching sequence. For example, after receiving the notification, it begins switching the corresponding image processing parameters and performing image processing and output display in the first sampling period. For example, as... Figure 9 As shown, if the filter switching element enters the stable stage, the target filter areas corresponding to the first to fifth sampling periods are blank position 1, NEI filter, blank position 2, REI filter, blank position 1, and so on. Then the image processing parameters that the image processing element should use in the first to fifth sampling periods are the ISP parameters of white light, the ISP parameters of special light NEI, the ISP parameters of white light, the ISP parameters of special light REI, the ISP parameters of white light, and so on.

[0128] In this embodiment, the image processing element exits standby mode and begins synchronously switching image processing parameters. This facilitates the synchronous switching of the image processing element, the filter switching element, and the light source, thereby ensuring the matching of image processing parameters with the filter and light source emission combination in each sampling cycle, and guaranteeing the real-time performance and synchronization of multiple images in various observation modes. In some embodiments, the standby mode also reduces system power consumption and heat generation.

[0129] In some embodiments, the image processing parameters (ISP parameters) may include the color correction matrix parameter formulas (2) to (4) mentioned above, which require converting the photosensitive data collected by the image sensor and then generating an image based on the converted photosensitive data. In some embodiments, the image processing parameters may also include settings such as gain, white balance, and noise filtering.

[0130] In some embodiments, after the processor responds to the control command and controls the filter switching element to start, it can also determine whether the filter switching element has entered a stable phase. If it is determined that the filter switching element has entered a stable phase, it also sends a second start command to the image processing element to indicate that the filter switching element has entered a stable phase. After receiving the second start command sent by the processor, the image processing element exits the standby mode and begins to switch the image processing parameters corresponding to each sampling period.

[0131] In some embodiments, the processor controls the image processing element to enter a standby mode, and controls the image processing element to exit the standby mode after a first preset time, and to start switching the image processing parameters corresponding to each sampling period.

[0132] Specifically, when the processor receives a control command for simultaneous display of multiple screens in various observation modes, and responds to the command, it simultaneously activates the filter switching element and puts the image processing element into standby mode, then begins timing. Once the timing reaches a first preset time, the processor not only controls the light source to switch between light source emission combinations corresponding to different sampling periods, but also controls the image processing element to exit standby mode and begin switching image processing parameters corresponding to each sampling period. This method is very simple in its control logic and does not require a complex real-time communication link between the image processing element and the filter switching element. Stable and reliable synchronization can be achieved using only a fixed delay parameter, resulting in low processor requirements and good system robustness.

[0133] Understandably, the synchronous switching of the filter switching element, light source, and image processing element during the stable phase depends on the design of their switching timing. Within this timing, the allocation of the number of sampling periods per second (frames) for each observation mode must meet the basic frame rate requirements for image display in each observation mode to ensure image continuity. In some embodiments, the sampling periods of the two observation modes are alternated. This improves the real-time performance and synchronization of each screen in each dual-observation mode, preventing asynchrony between the two screens due to excessively long intervals between sampling periods corresponding to the same observation mode, and also preventing asynchrony with user actions caused by excessively long intervals between sampling periods corresponding to the same observation mode. Similarly, in the multi-screen simultaneous display mode of three observation modes, the sampling periods of the three observation modes are alternated. In some embodiments, to accommodate the needs of multiple observation modes and to match the design of the filter switching element, after meeting the basic frame rate requirements for image display in each observation mode, the sampling periods of the same observation mode can also be set adjacently. For example, the sampling period of the white light observation mode can be set in multiple adjacent sampling periods.

[0134] S805: Receive an image generated by the image processing element based on at least a portion of the sampling period, and display it in the corresponding screen area.

[0135] For example, the generated image refers to the image output by the image processing element after processing the raw data acquired in each sampling cycle. The screen area refers to multiple different areas allocated on the display device for displaying different images. Synchronous switching refers to the switching of the filter switching element, the light source, and the image processing element according to a preset switching sequence during the stabilization phase, ensuring data consistency.

[0136] After generating an image, the image processing element sends it to the processor. The processor receives the images generated by the image processing element and assigns each image to a corresponding screen area of ​​the display device. For example, combining... Figure 7 As shown, the image from the first sampling period is displayed in area 1, the image from the second sampling period in area 2, the image from the third sampling period in area 1, the image from the fourth sampling period in area 2, and so on, achieving continuous simultaneous display of multiple images. Throughout the process, the filter switching element, the light source, and the image processing element maintain synchronous switching, meaning that all three update their states simultaneously in each sampling period. Synchronous switching eliminates timing deviations between components, ensuring consistency in image acquisition, processing, and display, and avoiding image tearing or color distortion.

[0137] In some embodiments, such as Figure 6CAs shown, the filter switching element includes two filter areas. The processor outputs a white light image and a special light NEI image at a frame rate of 30 frames per second on the display device. Therefore, the image sensor needs to capture 60 frames per second of photosensitive data. The image processor then processes and outputs this photosensitive data, including 30 frames of white light images and 30 frames of special light NEI images. Figure 6C Taking the clockwise rotation of the filter switching element and the acquisition of the first frame image as an example of a white light image generated based on a blank position, the image output of the image processing element is sequentially a white light image, a special light NEI image, a white light image, and a special light NEI image, and so on. The display device updates the corresponding image area accordingly.

[0138] The specific processing flow is as follows: The user selects the white light observation mode and the special light NEI observation mode sequentially on the interactive interface. After clicking the "Enter Dual Light Dual Screen" button on the interactive interface, the processor receives the control command for simultaneous display of the two screens and then executes the following: First, it determines the light source information of the white light and the special light NEI, and determines the image processing parameters corresponding to the white light and the special light REI. Then, responding to the control command, it controls the filter switching element to start, and switches the filter area of ​​each sampling period during the stabilization phase. Simultaneously, it controls the image processing element to enter standby mode. After the filter switching element enters the stabilization phase, it controls the light source to begin alternately switching the light source emission combination of the white light and the special light NEI corresponding to each sampling period, and controls the image processing element to exit standby mode. It then begins alternately switching the image processing parameters of the white light and the special light NEI corresponding to each sampling period for image processing. The generated images are, in sequence, a white light image, a special light NEI image, a white light image, a special light NEI image… The white light image is displayed… Figure 7 In region 1 shown, the special light NEI image is displayed Figure 7 In region 2 shown.

[0139] In some embodiments, such as Figure 6A As shown, the filter switching element includes four filter areas. The processor needs to output a white light image and a special light REI image at a frame rate of 30 frames per second on the display device. Therefore, the image sensor needs to capture 60 frames per second of photosensitive data. The image processor then processes and outputs this photosensitive data, including 30 frames of white light images and 30 frames of special light REI images. However, because the filter switching element has four filter areas, during the sequential rotation, the image sensor still captures photosensitive data in some sampling cycles. This photosensitive data, or the corresponding images, is discarded because it is not needed for display. This means that these unnecessary images waste some sampling cycles, thus requiring a further increase in the frame rate. Figure 6AThe filter switching element shown is rotated clockwise. Taking the first frame image acquired as an example, which is a white light image generated based on blank position 1, the acquired images are, in sequence, a white light image, a special light NEI image, a white light image, a special light REI image, and so on. The rotation speed of the filter switching element needs to reach 1800 revolutions per minute (30 revolutions per second).

[0140] The specific processing flow is as follows: The user selects the white light observation mode and the special light observation mode (REI) sequentially on the interactive interface. After clicking the "Enter Dual Light Dual Screen" button on the interactive interface, the processor receives control commands for simultaneous display of multiple screens across various observation modes, and then executes the following: First, it determines the light source information for the white light and special light REI, and determines the corresponding image processing parameters for the white light and special light REI; then, responding to the control commands, it controls the filter switching element to start, and switches the filter area for each sampling period during the stabilization phase, while simultaneously controlling the image processing element to enter standby mode. After the filter switching element enters the stabilization phase, it controls the light source to begin processing according to... Figure 9 The corresponding relationship shown switches the light source emission combination for each sampling period and controls the image processing element to exit standby mode and begin processing according to... Figure 9 The corresponding relationship shown switches the image processing parameters for each sampling period to perform image processing. The generated images are, in sequence, white light image, special light NEI image, white light image, special light REI image, white light image, ... The white light image is displayed... Figure 7 In region 1 shown, the special light REI image is displayed Figure 7 In region 2 shown.

[0141] Specifically, after the filter switching element enters the stable phase, when the filter switching element switches to blank position 1, the light source starts to turn on the white light LED combination, and at the same time, the image processing element exits the standby mode. The image processing unit acquires the light signal of the first frame of white light image, calls the image processing parameters corresponding to the white light image for processing, and generates a white light image. Next, when the filter switching element switches to the REI filter, the light source switches to the LED combination that turns on the special light REI. The image processing unit acquires the light signal of the special light REI image and calls the image processing parameters corresponding to the white light image for processing, generating a special light REI image. When the filter switching element switches to blank position 2 and the NEI filter, the light source can turn on or off any LED, and the image processing unit can acquire or not acquire any light signal, and not call any image processing parameters for processing, or the image processing unit calls the image processing parameters and generates an image, but does not output an image. This embodiment does not impose any limitations. Finally, 30 frames / second of white light image and 30 frames / second of special light REI image are output on the display device.

[0142] Using the method of this application embodiment, after the filter switching element enters the stable stage, the switching starting points of the filter switching element, the light source, and the image processing element are aligned, and then they are controlled to run stably according to the preset switching sequence, with their respective switching order and running frequency. Stable synchronization can be achieved throughout the entire mode process, thereby ensuring the real-time performance and synchronization of multiple image frames in the scene mode where multiple viewing modes are displayed simultaneously.

[0143] In some embodiments, the processor can also calculate the exposure data of each frame of the output image in real time and perform exposure control. Specifically, the processor can acquire the exposure data of the image generated by the image processing element based on at least a portion of the sampling period, and adjust one or more of the following according to the exposure data: light source brightness, exposure time of the image sensor, and exposure gain. This application embodiment does not impose limitations. For example, the currently acquired exposure data can be used to adjust the image in any viewing mode or a specified viewing mode in subsequent frames, for example, in combination with... Figure 9 As shown, the processor uses the adjusted exposure data to simultaneously adjust the special light NEI image for the second sampling period and the white light image for the third sampling period. Alternatively, the processor may adjust the white light image for subsequent sampling periods using only the adjusted exposure data. By using the exposure data of the output image to provide real-time feedback and adjust subsequent images, image quality can be effectively improved, making it better suited to user needs.

[0144] The multi-screen simultaneous display control method for multiple observation modes in an endoscope system provided in this application embodiment, by coordinating the switching timing of the filter switching element, the light source, and the image processing element, and clarifying the starting point for synchronous switching of the three, can achieve efficient synchronization of multi-screen simultaneous display of multiple observation modes, realize the real-time and synchronous nature of multi-screen images, and solve the image delay problem. Specifically, when the processor receives the control command for simultaneous display of multiple observation modes, it first controls the filter switching element to start, and controls the image processing element to enter standby mode and wait for the filter switching element to enter the stable stage, providing a stable optical foundation for subsequent image acquisition; subsequently, after the filter switching element enters the stable stage, the switching starting point of the filter switching element is clarified, and the light source and image processing element are synchronously controlled to start switching the light source emission combination and image processing parameters corresponding to each sampling cycle, so that the operation of the three is completely matched in each sampling cycle. Based on this synchronous switching mechanism, it is beneficial to ensure an effective frame rate to output images in multiple observation modes, ensuring that the image update is not delayed due to the user moving the endoscope system, ensuring that the image follows the operation in real time, eliminating the timing misalignment between images, and ensuring the synchronicity of simultaneous display. The embodiments of this application provide users with a real-time, consistent and ghost-free simultaneous observation experience.

[0145] Figure 10This is a flowchart illustrating a control method for simultaneously displaying multiple viewing modes of an endoscope system, as provided in an embodiment of this application. Figure 10 As shown in the embodiments of this application, the control method for simultaneous display of multiple screens in an endoscope system with multiple observation modes may include:

[0146] S101: Receives control commands for simultaneous display of multiple screens in various viewing modes.

[0147] S102. Responding to the control command, control the filter switching element to start and control the image processing element to enter standby mode.

[0148] S103. Determine whether the filter switching element has entered the stable stage.

[0149] If yes, proceed to step S104; otherwise, continue with the judgment.

[0150] S104. Send a first start command to the light source to control the light source to start switching the light source emission combination corresponding to each sampling period; at the same time, send a second start command to the image processing element to control the image processing element to exit the standby mode and start switching the image processing parameters corresponding to each sampling period.

[0151] S105. Receive the image generated by the image processing element based on at least a portion of the sampling period, and display it accordingly in the corresponding screen area.

[0152] It should be noted that the specific implementation of steps S101 to S105 can be referred to other embodiments, and will not be elaborated here.

[0153] The control method for simultaneous display of multiple screens in multiple observation modes of the endoscope system provided in this embodiment first determines whether the filter switching element has entered the stable stage, and then controls the filter switching element, the light source and the image processing element to switch synchronously after the filter switching element enters the stable stage. This fundamentally avoids the problem of illumination and processing mismatch caused by the filter element not being completely stable. It can accurately control the synchronization of filter switching, light source and image processing, and significantly improve image quality and display effect.

[0154] Figure 11 This is a flowchart illustrating a control method for simultaneously displaying multiple viewing modes of an endoscope system, as provided in an embodiment of this application. Figure 11 As shown in the embodiments of this application, the control method for simultaneous display of multiple screens in an endoscope system with multiple observation modes may include:

[0155] S111: Receives control commands for simultaneous display of multiple screens in various viewing modes.

[0156] S112, responding to control commands, controlling the filter switching element to start and start timing, and controlling the image processing element to enter standby mode.

[0157] S113. During the stabilization phase, switch the target filter area for each sampling period.

[0158] S114. After the first preset time, control the light source to start switching the light source emission combination corresponding to different sampling periods; at the same time, control the image processing element to exit the standby mode and start switching the image processing parameters corresponding to each sampling period.

[0159] S115. Receive an image generated by the image processing element based on at least a portion of the sampling period, and display it in the corresponding screen area.

[0160] It should be noted that the specific implementation of steps S111 to S1115 can be referred to other embodiments, and will not be elaborated here.

[0161] The control method for simultaneous display of multiple viewing modes in an endoscope system provided in this embodiment introduces a centralized timing control based on a start time and a first preset time. Upon receiving a control command, the filter switching element is first activated and the image processing element is put into standby mode. Subsequently, based on precise time management, the switching of the light source emission combination and the standby exit and parameter switching of the image processing element are synchronously triggered at the first preset time point. This achieves high-precision synchronization between filter switching, light source modulation, and image processing, effectively avoiding problems such as image color distortion, screen flicker, and signal asynchrony caused by different response delays of various components, and significantly improving the stability of simultaneous display of multiple viewing modes.

[0162] Figure 12 This is an interactive schematic diagram illustrating the simultaneous display of multiple screens across various observation modes of an endoscope system, provided as an embodiment of this application. For example... Figure 12 As shown in the embodiments of this application, the control method for simultaneous display of multiple screens in an endoscope system with multiple observation modes may include:

[0163] It needs to be explained that, Figure 12 The “LED” in the diagram is located in the endoscope host and includes a filter switching element, a light source, and a processor; the image processor is also referred to as “FPGA” in the diagram, and the filter switching element is also referred to as “color wheel” in the diagram; “APP” includes its corresponding APP processor.

[0164] S1. The user sends control commands to the processor through the interactive APP (also referred to as "entering dual-light mode" in the diagram), and simultaneously sends the light source information corresponding to the various observation modes selected by the user to the processor, and simultaneously sends the image processing parameters corresponding to the various observation modes selected by the user to the image processor. The above sending of the image processing parameters corresponding to the two observation modes is also referred to as "sending dual-light ISP parameters" in the diagram; the above sending of the light source information corresponding to the two observation modes is also referred to as "sending LED information" in the diagram.

[0165] S2. After receiving the image processing parameters corresponding to the various viewing modes, the image processor responds to the operation and enters standby mode (also referred to as "entering standby mode" in the figure).

[0166] S3. After receiving the light source information corresponding to the multiple observation modes, the processor responds to the control operation and controls the filter switching element to start preparing. At this time, the filter switching element starts to rotate faster.

[0167] S4. After the filter switching element enters the stable stage, that is, after the rotation speed is stable, the processor sends a second start notification (also referred to as "ready notification" in the figure) to the image processor. At this time, the processor will simultaneously start the light source to switch the light source emission combination according to the preset switching sequence. In some embodiments, the processor also sends a ready notification to the APP processor to display the ready status on the interactive interface.

[0168] S5. The image processor responds to the ready notification, exits standby mode, and synchronously begins switching image processing parameters according to the preset switching sequence.

[0169] S6. The filter switching element, image processor, and light source switch synchronously, entering a multi-screen simultaneous display mode operation program with multiple observation modes. During operation, the figure shows one embodiment where the light source and filter switching element switch at a constant speed, and the image processor alternately switches between two image processing parameters.

[0170] In some embodiments, during the simultaneous display of multiple viewing modes, the user can send a dimming command (also referred to as "dimming - LED brightness" in the figure) to the processor through the interactive interface to adjust the brightness of the light source. In some embodiments, this dimming command may be automatically generated based on the inspection environment.

[0171] In some embodiments, during the simultaneous display of multiple viewing modes, the user can send a dimming command (also referred to as "dimming-exposure parameter" in the figure) to the image processor through an interactive interface to adjust the exposure parameters. In some embodiments, this dimming command may be automatically generated based on the inspection environment.

[0172] In some embodiments, during the simultaneous display of multiple screens in multiple viewing modes, the user can send adjustment commands to the image processor to adjust the ISP parameters (also referred to as "adjusting ISP parameters" in the figure) through the interactive interface.

[0173] S7. In the mode of simultaneous display of multiple screens in multiple observation modes, the user can send an exit command to the processor through the interactive interface (also referred to as "exit dual-light mode" and "exit dual-light mode notification" in the figure).

[0174] S8. The processor responds to the exit command by sending an exit notification to the image processor and shutting down the filter switching element and the light source. During the shutdown of the filter switching element, it gradually decelerates and returns to its default settings. In some embodiments, the processor sends a recovery notification to the APP processor to prompt the user to resume the normal check mode.

[0175] The control method for simultaneous display of multiple screens in an endoscope system provided in this application embodiment ensures coordinated operation of each component during the multi-screen display process by synchronously controlling the filter switching element, the light source, and the image processing element. This effectively reduces delay and interference, improves image clarity and real-time performance, and achieves the goal of simultaneously displaying multiple screens.

[0176] This application also provides an endoscope system, wherein the endoscope system includes a processor, a light source assembly, an image processing element, and a display device; the light source assembly includes multiple light sources and a filter switching element disposed on the light output path of the light sources, the filter switching element including at least two filter regions; the light emitted by the light sources is combined or passes through the corresponding filter regions individually to output illumination light of the corresponding spectrum; the image processing element is used to acquire the light signal after the illumination light illuminates the area to be examined, and to retrieve the corresponding image processing parameters for processing to generate an image; the display device includes at least multiple screen areas; the processor is used to execute the control method described above.

[0177] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0178] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0179] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0180] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0181] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0182] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0183] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0184] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0185] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0186] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0187] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A control method for simultaneous display of multiple screens in multiple observation modes of an endoscope system, characterized in that, The control method is applied to an endoscope system, which includes a processor, a light source assembly, an image processing element, and a display device. The light source assembly includes multiple light sources and a filter switching element disposed on the light output path of the light sources. The filter switching element includes at least two filter regions. The light emitted by the light sources is combined or passes through the corresponding filter regions individually to output illumination light with a corresponding spectrum. The image processing element is used to acquire the light signal after the illumination light illuminates the area being examined, and to retrieve the corresponding image processing parameters for processing to generate an image. The display device includes multiple screen areas; the control method includes: Receive control commands for simultaneous display of multiple screens in the various observation modes; In response to the control command, the filter switching element is activated, and the target filter region for each sampling period is switched during the stabilization phase, wherein the target filter region is the filter region correspondingly set on the main axis of the light source; In response to the control command, the light source is controlled to start switching the light source emission combination corresponding to each sampling period after the filter switching element enters the stable phase; In response to the control command, the image processing element is controlled to enter standby mode, and after the filter switching element enters the stabilization stage, it exits the standby mode and begins to switch the image processing parameters corresponding to each sampling period. The image is received by the image processing element based on at least a portion of the sampling period and displayed accordingly in the corresponding screen area; In the stable phase, the filter switching element, the light source, and the image processing element switch synchronously.

2. The method according to claim 1, characterized in that, After responding to the control command and controlling the filter switching element to start, the method further includes: If it is determined that the filter switching element has entered the stable phase, a first start command and a second start command are sent to the light source and the image processing element, respectively, to instruct the filter switching element to enter the stable phase.

3. The method according to claim 2, characterized in that, The step of determining that the filter switching element has entered a stable phase includes: After the filter switching element has been activated for a preset period of time, it is determined that the filter switching element has entered a stable phase. Alternatively, monitor the rotational speed of the filter switching element, and determine that the filter switching element has entered a stable phase after the rotational speed of the filter switching element reaches and remains at a preset rotational speed. Alternatively, the filter switching element may further include an optical coupler detection component and a body. An initial position is set on the body. The initial position is monitored by the optical coupler detection component. When the initial position is detected to repeat at a preset frequency, it is determined that the filter switching element has entered a stable phase.

4. The method according to claim 3, characterized in that, The filter switching element further includes a rotation drive element and a body. The at least two filter regions are respectively disposed in different regions of the body. The rotation drive element is used to drive the body to rotate by rotation, so as to switch the filter regions corresponding to the light source main axis with different sampling periods. The filter switching element maintains a uniform rotation speed during the stable phase.

5. The method according to claim 1, characterized in that, After receiving the control command for simultaneous display of multiple viewing modes, the method further includes: In response to the control command, the light source is controlled to start switching the light source emission combination corresponding to different sampling periods after a first preset time. In response to the control command, the image processing element exits the standby mode and begins switching the image processing parameters corresponding to each sampling period after the first preset time.

6. The method according to claim 1, characterized in that, In the standby mode, the image processing element is set not to output an image.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: In response to the user's parameter setting instructions, determine the light source information and image processing parameters.

8. The method according to claim 7, characterized in that, The light source information includes the light source switching sequence and the light intensity ratio of each light source emission combination; the image processing parameters include at least the color correction matrix parameters, which are used to convert the values ​​of the red, green, and blue channels.

9. The method according to any one of claims 1-6, characterized in that, The method further includes: The image processing element acquires exposure data of an image generated based on at least a portion of the sampling period, and determines the brightness of the adjusted illumination light based on the exposure data; The adjusted brightness of the illumination light is sent to the light source component so that when switching the light source emission combination corresponding to each sampling period, the illumination light is output with the adjusted brightness.

10. An endoscope system, characterized in that, The endoscope system includes a processor, a light source assembly, an image processing element, and a display device. The light source assembly includes multiple light sources and a filter switching element disposed on the light emission path of the light sources. The filter switching element includes at least two filter regions. The light emitted by the light sources is combined or passes through the corresponding filter regions individually to output illumination light with a corresponding spectrum. The image processing element is used to acquire the light signal after the illumination light illuminates the area being examined, and to retrieve corresponding image processing parameters for processing to generate an image. The display device includes at least multiple screen areas. The processor is used to execute the control method as described in any one of claims 1-9.

Images