A method and system for balancing RGB exposure brightness

Abstract

This invention provides a method and system for balancing RGB exposure brightness, belonging to the field of medical endoscopic imaging technology. The system includes an image output unit, an image processing platform, and a display. The image output unit decomposes incident light into blue light and red-green light using a beam splitter, generating a blue channel image and a red-green channel image. The image output unit sends the blue light image and the red-green light image to the exposure control unit of the image processing platform. Based on the brightness of the blue light image and bleeding-related indicators, the exposure parameters of the two images are dynamically adjusted. After de-mosaicing, the red-green light image is replaced with the blue pixel values ​​of the blue light image to generate a fused RGB image, which is then displayed after post-processing. This invention, based on adaptive exposure control and targeted image fusion using bleeding-related indicators, can significantly enhance the clarity and recognizability of bleeding areas, improving the accuracy of endoscopic bleeding point localization and diagnosis.

Description

Technical Field

[0001] This invention relates to the field of medical endoscopic imaging technology, and more specifically to a method and system for balancing RGB exposure brightness. Background Technology

[0002] Currently, bleeding is a common and difficult-to-avoid complication during endoscopic diagnosis and surgery. When bleeding occurs in the surgical area, the blood gushing from the wound quickly covers the tissue surface, severely affecting the visibility of the endoscope and obscuring the bleeding point, making it difficult to detect. In some ear, nose, and throat surgeries, bleeding can cause scattering, reducing image clarity and color resolution, increasing the difficulty of the surgeon's operation.

[0003] The physical mechanism by which hemorrhage degrades image quality is mainly due to the strong absorption of light in the 400-600nm wavelength range by hemoglobin in the blood. This results in a significant attenuation of incident light energy in the hemorrhage area, causing the image to darken. Simultaneously, light scattering caused by blood cells creates a cloudy medium, blurring tissue edges and causing loss of detail. Furthermore, the hemorrhage point is highly similar in tone to the surrounding tissue, with only slight differences in saturation, making it difficult for the human eye to accurately distinguish the boundaries of the hemorrhage point.

[0004] Most existing endoscopic imaging systems use single-color image sensors, where the red and green / blue channels share the same sensor pixels. Red information only accounts for 1 / 4 of all pixels, requiring interpolation algorithms for reconstruction. This results in a low signal-to-noise ratio and blurred details in the red channel, especially in bleeding areas where the red signal further attenuates, making accurate identification of bleeding points difficult. Furthermore, existing methods lack dedicated exposure strategies for bleeding scenarios and fail to dynamically adjust the exposure parameters of different spectral channels based on bleeding-related indicators, making it difficult to simultaneously capture bleeding areas and normal tissue areas.

[0005] To address the aforementioned issues, the industry has proposed several technical solutions. One approach is the time-series multi-light source switching solution, such as Olympus's RDI technology. This involves rapidly switching between different wavelength light sources to acquire multiple frames of images and then fusing them. However, this method is prone to color ghosting when dealing with fast-moving targets. Another solution is the three-image sensor imaging scheme. Light is split into red, green, and blue beams by a beam splitter, each received by three independent sensors with full pixel resolution. No color interpolation is required; each pixel possesses true RGB information. However, the size and weight of the camera significantly increase due to the need to accommodate the beam splitter assembly and three sensors. Furthermore, this solution is typically only suitable for rigid endoscopes (such as laparoscopes, arthroscopes, and hysteroscopes) or relatively thick flexible lens sections, making it difficult to apply to ultra-fine diameter endoscopes (such as neonatal gastroscopes and ultra-fine bronchoscopes). In addition, the three-color beam splitter assembly has extremely high requirements for shock resistance, waterproofing, and autoclaving, increasing manufacturing difficulty and maintenance costs.

[0006] For example, Chinese invention patent application CN117017166A discloses an image processing method, an endoscope system, an endoscope camera device, and a storage medium. It processes visible light scene images and near-infrared light scene images to obtain processed visible light scene images and near-infrared light scene images. The processed visible light scene images and near-infrared light scene images are then fused to obtain a fused image, improving the clarity and recognizability of images captured by the endoscope in complex scenes and / or poor imaging conditions during surgery. Although this invention can improve image clarity and recognizability by superimposing infrared light bands on white light images, it cannot solve the problem of poor color reproduction because it does not specifically process the red, blue, and green light in the visible light band, thus failing to improve color fidelity. Furthermore, the fusion of the infrared and visible light bands cannot be achieved naturally. Therefore, the existing technology still needs improvement. Summary of the Invention

[0007] The technical problem to be solved by this invention is that single-image sensor solutions have poor color reproduction and cannot solve the problem of drastic decline in endoscopic image quality in bleeding environments, while three-image sensor solutions are complex in process, expensive, bulky, and difficult to implement.

[0008] To address the aforementioned technical problems, this invention provides a method for balancing RGB exposure brightness, comprising the following steps:

[0009] S1. The image output unit sends a blue light image and a red-green light image to the image processing platform, wherein the blue light image contains blue channel information and the red-green light image contains green channel and red channel information;

[0010] S2. The image processing platform controls the exposure control unit to automatically control the exposure of the blue light image and the red-green light image, and adjusts the gain of the red-green light image according to the bleed-related index;

[0011] S3. The image preprocessing unit simultaneously preprocesses the blue light image and the red-green light image to obtain the preprocessed blue light image and the red-green light image.

[0012] S4. Merge the preprocessed blue light image and red-green light image to generate a color image;

[0013] S5. The color image is post-processed and then sent to the display for display.

[0014] Furthermore, the automatic exposure control in step S2 includes the following steps:

[0015] S21. Select a measurement area using the blue light image, measure the light, and calculate the average brightness and standard deviation of the current blue light image;

[0016] S22. Set the target brightness for exposure, and use the exposure strategy algorithm to calculate the exposure value and gain value of the blue light image based on the difference between the average brightness of the blue light image and the target brightness for exposure;

[0017] S23. Obtain bleeding-related indicators by statistically analyzing the proportion of pixels exceeding a preset brightness threshold within the photometric area;

[0018] S24. Calculate the exposure value and gain value of the red and green light image. The exposure value is obtained by using the exposure value of the blue light image, and the gain value is obtained by calculating the gain value of the blue light image and bleeding-related indicators.

[0019] S25. Write the calculated blue light image exposure value, red and green light image exposure value, blue light gain value, and red and green light gain value into the image output unit through the interface.

[0020] Furthermore, the exposure strategy algorithm in step S22 may employ at least one of PID control, lookup table method, and feedback loop method.

[0021] Furthermore, the preset brightness threshold is determined based on the average brightness and standard deviation.

[0022] Furthermore, the image fusion operation in step S4 includes the following specific steps:

[0023] S41. Generate an initial RGB image based on the preprocessed red and green light image;

[0024] S42. Use the blue channel information from the preprocessed blue light image to replace the blue channel of the initial RGB image to generate the final color image.

[0025] This invention also provides a system for balancing RGB exposure brightness. The system includes an image output unit, an image processing platform, and a display. The image output unit sends an image to the image processing platform via a high-speed transmission interface for algorithm processing, and then transmits the image to the display for display.

[0026] The image output unit includes a beam splitter, a first image sensor, and a second image sensor. The beam splitter is used to decompose the incident light into a first-band blue light and a second-band red-green light. The first image sensor is a monochrome sensor with a red filter, used to collect the first-band light to generate a blue channel image. The second image sensor is a color sensor, used to collect the second-band light to generate a green channel image and a red channel image. The image processing platform includes an image preprocessing unit, an image postprocessing unit, an exposure control unit, and a communication control unit. The image processing platform mainly receives the two images sent by the image output unit through the image preprocessing unit, performs preprocessing, and then connects the image data to the image postprocessing unit for further processing, thereby connecting to the display for display.

[0027] Furthermore, the exposure control unit implements the exposure algorithm control of the two image sensors, and the communication control unit, as the control center of the entire image processing algorithm platform, implements the control of each unit of the image processing platform. The image preprocessing unit mainly acquires the two images transmitted by the image output unit, performs a series of preprocessing on the two images, and then merges them into an RGB image, which is then transmitted to the image postprocessing unit. The image postprocessing unit mainly implements a series of algorithm processing on the RGB image, and finally transmits it to the display for image display.

[0028] The beneficial effects of this invention are as follows: A beam splitter is used to decompose the incident light into blue light and red-green light, which are independently acquired by a monochrome sensor and a color sensor equipped with filters, respectively. A control strategy is adopted that prioritizes the blue light image and correlates it with the red-green light image. The gain of the red-green light image is dynamically adjusted according to the severity of bleeding-related indicators, adaptively increasing as the bleeding-related indicators worsen. Both image sensors use the same exposure value, fundamentally avoiding the color ghosting problem of multi-frame fusion schemes. The blue light image pixel direct replacement fusion strategy fully utilizes the full resolution and high signal-to-noise ratio characteristics of the blue light image acquired by the monochrome sensor, avoiding the detail loss caused by the interpolation of the blue light image in traditional fusion schemes. This invention achieves an optimized balance of cost, size, and performance while ensuring high-quality imaging in bleeding scenarios, solving the long-standing but unresolved problem of endoscopic imaging in bleeding environments, demonstrating significant technological progress and broad clinical application value. Attached Figure Description

[0029] Figure 1 This is a flowchart illustrating an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of the exposure control process of the present invention;

[0031] Figure 3 This is a schematic diagram of the structure of an embodiment of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0033] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0034] To keep the following description of the embodiments of the present invention clear and concise, detailed descriptions of some known functions and components have been omitted.

[0035] This invention provides a method and system for balancing RGB exposure brightness, such as... Figure 3 As shown, the system includes an image output unit, an image processing platform, and a display. The image output unit sends images to the image processing platform via a high-speed transmission interface for algorithm processing, and then transmits the images to the display for display. Preferably, the high-speed transmission interface is at least one of PCIe, HDMI, and USB.

[0036] The image output unit includes a beam splitter, a first image sensor, and a second image sensor. The beam splitter is used to decompose the incident light into a first band of blue light and a second band of red and green light. The first image sensor is a black and white sensor with a blue filter, used to collect the first band of light to generate a blue channel image. The second image sensor is a color sensor, used to collect the second band of light to generate a green channel image and a red channel image.

[0037] The image processing platform includes an image preprocessing unit, an image postprocessing unit, an exposure control unit, and a communication control unit. The image processing platform mainly receives two images sent by the image output unit through the image preprocessing unit, performs preprocessing, and then connects the image data to the image postprocessing unit for further processing and display. The exposure control unit realizes the exposure control of the two image sensors, and the communication control unit realizes the control of the image preprocessing unit, the image postprocessing unit, and the exposure control unit.

[0038] Preferably, the image preprocessing unit includes at least one of Bayer noise reduction, white balance, black level correction, vignetting correction, bad pixel correction, and HDR, and the image postprocessing unit includes at least one of color enhancement, image sharpening, shadow enhancement, and color space conversion.

[0039] The exposure control unit, as a key module of the entire image processing platform, implements the exposure algorithm control for the two image sensors.

[0040] The communication control unit serves as the control center of the entire image processing algorithm platform, enabling control of each unit within the platform.

[0041] The image preprocessing unit mainly acquires two images transmitted from the image output unit, performs a series of preprocessing steps on the two images, merges them into an RGB image, and transmits it to the image postprocessing unit.

[0042] The image post-processing unit mainly implements a series of algorithmic processing of RGB images, and finally transmits them to the display for image display.

[0043] like Figure 1 The diagram shown is a flowchart illustrating a method for balancing RGB exposure brightness provided by the present invention. The specific steps of the method are as follows:

[0044] S1. The image output unit uses the PCIe 3.0 interface to send blue light and red-green light images to the image processing platform;

[0045] S2. The image processing platform controls the exposure control unit to perform automatic exposure control on the blue light image and the red-green light image at the same time;

[0046] Specifically, such as Figure 2 As shown, the automatic exposure control is explained through the following steps:

[0047] S21. Use the blue light image to select the metering area for metering, calculate the current image's average brightness μ and standard deviation σ, and the total number of pixels in the metering area is Np;

[0048] S22. Set the target brightness L, and use the exposure strategy algorithm to calculate the exposure value and gain value G of the blue light image based on the difference between μ and L. b ;

[0049] S23. The set H of pixels within the measurement area of ​​a blue light image that exceed the average brightness μ plus κ times the standard deviation σ is calculated using the following formula:

[0050]

[0051] H∈{ I i> μ+κσ}

[0052] In this embodiment, κ = 2.5, where I i α represents the pixel brightness value within the measurement area, and α is a parameter controlling the sensitivity of bleeding-related indicators to bright pixels;

[0053] S24. Calculate the exposure and gain values ​​of the red-green light image, where the exposure value is set to the blue light image exposure value, and the gain value is determined based on the hemorrhage-related index β, i.e.:

[0054]

[0055] in For the gain parameter, in this embodiment =0.5;

[0056] S25. Write the exposure and gain parameters of the blue light image and the red-green light image to the image output unit through the interface.

[0057] S3. The image preprocessing unit simultaneously performs black level correction, bad pixel correction, vignetting correction, and HDR image preprocessing operations on the blue light image and the red-green light image, and obtains the initial color image after the operation;

[0058] S4. Merge the preprocessed blue light image and red-green light image to generate the final color image;

[0059] Specifically, the image fusion operation is described through the following steps:

[0060] S41. Generate an initial RGB image from the blue-green light image obtained by the gradient-guided demosaic algorithm;

[0061] S42. Use the blue channel information from the preprocessed blue light image to replace the blue channel of the initial RGB image to generate the final color image;

[0062] S5. The color image is processed by color enhancement, image sharpening, shadow enhancement, color space conversion, etc., and then transmitted to the display for display.

[0063] The working principle of this invention is as follows: Addressing the technical problem of image quality degradation caused by bleeding during endoscopic surgery, this invention proposes a method and system for balancing RGB exposure brightness based on a beam splitter prism and a dual-sensor architecture. Its core working principle can be summarized as follows: Dedicated acquisition of the blue channel is achieved through spectral separation at the hardware layer, combined with dynamic exposure control and adaptive fusion algorithms at the software layer, achieving high signal-to-noise ratio and high contrast image enhancement in bleeding scenarios. This invention employs a beam splitter prism and a dual-sensor architecture to achieve multispectral imaging. This design utilizes the physical characteristic of strong absorption of red light by hemoglobin, maintaining a high signal-to-noise ratio signal even in bleeding areas. The exposure control unit adopts a strategy of blue light image channel dominance and adaptive following of red and green light images, dynamically increasing the gain of the red and green light images to compensate for signal loss caused by hemoglobin absorption as bleeding becomes more severe. Both sensors use the same shutter speed to ensure time consistency. After performing black level correction, bad pixel correction, white balance and other operations on the two images, the image preprocessing unit directly replaces the red channel in the interpolated red and green light image with the blue light image to generate a fused image for post-processing. The image post-processing unit finally outputs an enhanced image with high contrast and high signal-to-noise ratio.

[0064] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for balancing RGB exposure brightness, characterized in that, Includes the following steps: S1. The image output unit sends a blue light image and a red-green light image to the image processing platform, wherein the blue light image contains blue channel information and the red-green light image contains green channel and red channel information; S2. The image processing platform controls the exposure control unit to automatically control the exposure of the blue light image and the red-green light image, and adjusts the gain of the red-green light image according to the bleed-related index; S3. The image preprocessing unit simultaneously preprocesses the blue light image and the red-green light image to obtain the preprocessed blue light image and the red-green light image. S4. Merge the preprocessed blue light image and red-green light image to generate a color image; S5. The color image is post-processed and then sent to the display for display.

2. The method for balancing RGB exposure brightness according to claim 1, characterized in that, The automatic exposure control in step S2 includes the following steps: S21. Use the red light image to select the metering area for light measurement, and calculate the average brightness and standard deviation of the current blue light image; S22. Set the target brightness for exposure, and use the exposure strategy algorithm to calculate the exposure value and gain value of the blue light image based on the difference between the average brightness of the blue light image and the target brightness for exposure; S23. Obtain bleeding-related indicators by statistically analyzing the proportion of pixels exceeding a preset brightness threshold within the photometric area; S24. Calculate the exposure value and gain value of the red and green light image. The exposure value is obtained by using the exposure value of the blue light image, and the gain value is obtained by calculating the gain value of the blue light image and bleeding-related indicators. S25. Write the calculated blue light image exposure value, red and green light image exposure value, blue light gain value, and red and green light gain value into the image output unit through the interface.

3. The method for balancing RGB exposure brightness according to claim 2, characterized in that, The exposure strategy algorithm in step S22 can employ at least one of PID control, lookup table method, and feedback loop method.

4. The method for balancing RGB exposure brightness according to claim 2, characterized in that, The preset brightness threshold is determined based on the average brightness and standard deviation.

5. The method for balancing RGB exposure brightness according to claim 1, characterized in that, The image fusion operation in step S4 includes the following specific steps: S41. Generate an initial RGB image based on the preprocessed red and green light image; S42. Use the blue channel information from the preprocessed blue light image to replace the blue channel of the initial RGB image to generate the final color image.

6. A system for balancing RGB exposure brightness, the system implementing the steps of the method as described in any one of claims 1 to 5, the system comprising an image output unit, an image processing platform, and a display, characterized in that, The image output unit sends images to the image processing platform via a high-speed transmission interface for algorithm processing, and then transmits the images to the display for display. The image output unit includes a beam splitter, a first image sensor, and a second image sensor. The beam splitter separates the incident light into a first band of blue light and a second band of red and green light. The first image sensor is a monochrome sensor with a blue filter, used to collect the first band of light to generate a blue channel image. The second image sensor is a color sensor, used to collect the second band of light to generate green and red channel images. The image processing platform includes an image preprocessing unit, an image postprocessing unit, an exposure control unit, and a communication control unit. The image processing platform mainly receives two images sent by the image output unit through the image preprocessing unit, performs preprocessing, and then connects the image data to the image postprocessing unit for further processing, thereby connecting it to the display for display.

7. A system for balancing RGB exposure brightness according to claim 6, characterized in that, The exposure control unit controls the exposure algorithms of the two image sensors. The communication control unit, as the control center of the entire image processing algorithm platform, controls all units of the image processing platform. The image preprocessing unit mainly acquires the two images transmitted from the image output unit, performs a series of preprocessing steps on the two images, fuses them into an RGB image, and then transmits it to the image postprocessing unit. The image post-processing unit mainly implements a series of algorithmic processing of RGB images, and finally transmits them to the display for image display.

Images