Single-shot high-dynamic medical endoscope image acquisition system and method

Abstract

The invention discloses a single-shot high-dynamic medical endoscope image acquisition system and method, and belongs to the technical field of medical endoscopes. Two times of signal reading are completed through a DCG image sensor in a single-frame exposure period: for the first time of reading, under the condition that an LCG transistor is closed, an HCG signal is read; reading an LCG signal for the second time under the condition that the LCG transistor is opened; carrying out fusion processing on the image data read twice to obtain an image which gives consideration to the full well capacity and the light sensitivity of the sensor at the same time; meanwhile, a dynamic switching mechanism is introduced, and analog gain is applied to HCG reading. According to the method, the dynamic range and detail expressive force of the endoscope image can be effectively improved, and meanwhile the defect that smear is prone to occurring in a traditional alternate two-frame synthesis method is overcome. The image information amount and impression can be improved, and doctors can be helped to better observe the condition in the cavity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical endoscopes, in particular to a single-camera high dynamic medical endoscope image acquisition system and method. BACKGROUND

[0002] In real scenes, the human eye has a dynamic range far exceeding that of conventional image sensors. In the field of medical endoscopes, due to the limited space conditions of actual applications, the size of the image sensors used is small, and their dynamic range is more obviously inferior. At the same time, the internal environment of the human body is complex and variable, and the mucosa and other tissue regions near the light source are prone to overexposure. If the exposure parameters are reduced to avoid this problem, the dark details will be severely lost. These problems have a significant negative impact on the actual operation experience of doctors.

[0003] To solve the above problems, High Dynamic Range (HDR) technology has become the key to improving the overall picture experience. In order to more realistically display images with high dynamic range brightness and rich detail information in real scenes, traditional HDR technology usually takes multiple images with different exposure levels, synthesizes a high dynamic range image according to the camera response function, and then displays the HDR image using a professional display with HDR function. In the field of photography, the multiple exposure method is less restricted to implement. However, in the field of video, due to limitations such as frame rate, the photographing method cannot be directly used.

[0004] The common method in the industry is the two-frame alternation synthesis method: control the sensor to alternately capture high-exposure and low-exposure images, and then fuse the images in the ISP to achieve the HDR effect. However, this scheme has obvious defects: On the one hand, nearly half of the frame rate needs to be sacrificed, for example, the frame rate of a CMOS image sensor is 60 frames per second, and after two-frame synthesis, the final frame rate is reduced to 30 frames per second.

[0005] On the other hand, due to the large time difference between the two image captures, motion blur is likely to occur when moving. It is difficult to adapt to the demand for high frame rate and high definition in the medical imaging field.

[0006] Therefore, the present application designs a single-camera high dynamic medical endoscope image acquisition system and method to solve the above problems. SUMMARY

[0007] In view of the above shortcomings of the prior art, the present application provides a single-camera high dynamic medical endoscope image acquisition system and method.

[0008] To achieve the above purpose, the present application realizes the following technical solutions: The single-camera high dynamic medical endoscope image acquisition system comprises a DCG image sensor; The DCG image sensor completes two signal readouts within a single frame exposure cycle: the first readout reads the HCG signal when the LCG transistor is off; the second readout reads the LCG signal when the LCG transistor is on; by fusing the image data from the two readouts, an image that simultaneously takes into account the sensor's full-well capacity and photosensitivity is obtained. Introduce a dynamic switching mechanism: The DCG image sensor is operating in standard mode configuration, waiting for the mode switching trigger signal from the automatic exposure module; The automatic exposure module calculates the overall brightness distribution of the image, and separately calculates the brightness distribution of the highlight area based on the overall brightness distribution. Based on the statistical results and the debugging parameters set by the automatic exposure module, the highlight target and the safe target are obtained; the ratio of the safe target to the highlight target is defined as DRCgain. When the scene dynamically increases, and DRCgain exceeds the trigger threshold and meets the AE stabilization mechanism conditions, the automatic exposure module sends a mode switching trigger signal, and the DCG image sensor switches to HDR mode configuration; at the same time, the automatic exposure module reduces the overall exposure according to the bright target, and brightens the intermediate brightness area to the target brightness through gain and tone mapping; when the real-time DRCgain is lower than the trigger threshold, the DCG image sensor switches back to standard mode configuration. The ratio between HCG and LCG of the DCG image sensor is defined as Cgratio; when DRCgain ≤ CGratio for the current scene, the HCG analog gain remains at 1; when DRCgain > CGratio, the analog gain is increased on the HCG, with the specific formula as follows: CGratio*analoggain=DRCgain Analoggain is the maximum gain that can be achieved on HCG.

[0009] Furthermore, the value of Cgratio ranges from 2 to 4.

[0010] Furthermore, DRCgain should not exceed 16.

[0011] Furthermore, a protection zone exists when switching between the two modes.

[0012] Furthermore, the protection interval DRCgain value is [2.5, 3.5], meaning that when the system DRCgain > 3.5, it enters HDR mode, and when DRCgain < 2.5, it enters standard mode.

[0013] Furthermore, when switching modes, the state must remain unchanged for 6 consecutive frames to trigger the switch.

[0014] Furthermore, the DCG image sensor outputs HCG and LCG image data respectively, and the two image data are fused to generate high bit-count data. The specific fusion method is as follows: the LCG image data is left-shifted, and for pixel areas with a left-shifted number < 1024, the HCG image data is used for fusion or directly replaced with HCG image data according to the pixel value.

[0015] Furthermore, let the current fused pixel be (x, y). If HCG(x, y) is saturated, then: DCG(x,y)=LCG(x,y)*HCG / LCG, otherwise: DCG(x,y)=a*LCG(x,y)*HCG / LCG+(1-a)*HCG(x,y) Where 'a' is the weighting coefficient.

[0016] Furthermore, the DCG image sensor outputs raw data of HCG and LCG respectively, and the two 10-bit raw data are fused to generate high-bit data.

[0017] To better achieve the objectives of this invention, this invention also provides a method for acquiring high dynamic range images of a medical endoscope using single-frame dual-gain fusion, comprising the following: a. The DCG image sensor completes two signal readouts within a single frame exposure cycle: the first readout is performed with the LCG transistor off, reading the HCG signal; the second readout is performed with the LCG transistor on, reading the LCG signal; by fusing the image data from the two readouts, an image that simultaneously takes into account the sensor's full-well capacity and photosensitivity is obtained. b. Introduction of a dynamic switching mechanism: The DCG image sensor operates in standard mode configuration, waiting for the mode switching trigger signal from the automatic exposure module; the automatic exposure module calculates the overall brightness distribution of the image, and simultaneously separately calculates the brightness distribution of the highlight areas based on the overall brightness distribution. Based on the statistical results and the debugging parameters set by the automatic exposure module, the highlight target and the safe target are obtained; the ratio of the safe target to the highlight target is defined as DRCgain; when the scene dynamically increases, and DRCgain exceeds the trigger threshold and meets the AE stabilization mechanism conditions, the automatic exposure module sends a mode switching trigger signal, and the DCG image sensor switches to HDR mode configuration; simultaneously, the automatic exposure module reduces the overall exposure based on the highlight target, and brightens the intermediate brightness areas to the target brightness through gain and tone mapping; when the real-time DRCgain falls below the trigger threshold, the DCG image sensor switches back to standard mode configuration. c. Apply analog gain to HCG reading to increase Cgratio: The ratio between HCG and LCG of the DCG image sensor is defined as Cgratio; when DRCgain ≤ CGratio for the current scene, the HCG analog gain remains at 1; when DRCgain > CGratio, an analog gain is added to the HCG, using the following formula: CGratio*analoggain=DRCgain Analoggain is the maximum gain that can be achieved on HCG.

[0018] Compared to existing technologies, the advantages of this invention are as follows: This invention effectively improves the dynamic range and detail of endoscopic images, while avoiding defects such as motion blur that are common in traditional alternating two-frame synthesis methods. This enhances image information and visual appeal, helping doctors better observe the condition within the cavity. Figure 5 It can be seen that the HDR mode of the DCG image sensor can preserve details in dark areas while ensuring that highlights are not overexposed, and has a dynamic range that is significantly higher than that of the standard mode. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0020] Figure 1 This is a circuit diagram of the DCG image sensor of the present invention.

[0021] Figure 2 This is the "illuminance-output" curve under HCG (high conversion gain) in a DCG image sensor.

[0022] Figure 3 This is the "illuminance-output" curve for LCG (low conversion gain) in a DCG image sensor.

[0023] Figure 4 The output curve of HDR after fusing HCG and LCG data.

[0024] Figure 5 This is a comparison chart showing the effects of enabling HDR mode and standard mode in this invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0026] Example 1: A single-camera high dynamic range medical endoscope image acquisition system, including a DCG (Digital Correlation Double Sampling) based image sensor, referred to as a DCG image sensor; Real-time acquisition and output of HDR video is achieved through a DCG image sensor; Please refer to the accompanying drawings in the instruction manual. Figure 1 TX: Transmission transistor, connecting the pixel photosensitive area and the signal processing unit, controlling the transmission of photogenerated charge, and acting as the "switch" for pixel signal readout; RES: Reset transistor, used to reset the pixel unit's charge after signal readout, ensuring the initial state of the next photosensitive process is stable; SEL: Selection transistor, controlling whether the pixel signal is connected to the column signal line (V(Column)), enabling the selection of specific pixels, and working with row selection logic to achieve line-by-line scanning or window extraction of the image; CFD: Feedback capacitor, working with the SEL transistor to stabilize voltage fluctuations during signal transmission and improve signal readout accuracy.

[0027] The DCG image sensor performs two signal readouts within a single frame exposure cycle: the first readout, with the LCG transistor off, reads the HCG (high conversion gain) signal, characterized by a small full-well capacity (FWC) and low readout noise. Subsequently, with the LCG transistor on, the second readout reads the LCG (low conversion gain) signal; this time, the FWC is larger, but the readout noise is also larger. By fusing the image data (raw data) from the two readouts, an image that balances both the sensor's full-well capacity (FWC) and photosensitivity can be obtained. This approach effectively improves the dynamic range and detail of endoscopic images while avoiding the ghosting and other defects common in traditional alternating two-frame synthesis methods. It enhances image information and visual appeal, helping doctors better observe the internal cavity.

[0028] Since the high dynamic range mode configuration consumes relatively high power, a dynamic switching mechanism is introduced to reduce system heat generation, triggering HDR mode when the scene has a high dynamic range.

[0029] 1. The DCG image sensor is operating in standard mode configuration, waiting for the mode switching trigger signal from the automatic exposure (AE) module; 2. The automatic exposure module calculates the overall brightness distribution of the image, and at the same time, it separately calculates the brightness distribution of the bright parts based on the overall brightness distribution. Based on the statistical results and the debugging parameters set by the automatic exposure module, it obtains the bright target (the target that ensures the brightness of the bright parts is appropriate) and the safety target (the target that ensures the overall brightness range meets the requirements). Define the ratio of the safe target to the highlighted target as DRCgain; When the scene has a high dynamic range, meaning the highlights differ significantly from the overall image, DRCgain increases. If this increases and exceeds the trigger threshold while meeting the AE stabilization mechanism (where the brightness of N consecutive frames is less than a preset value, indicating stable image without drastic fluctuations), the automatic exposure module sends a mode switching trigger signal, and the DCG image sensor switches to HDR mode. The trigger threshold can be set by engineers based on actual hardware capabilities and application requirements. Generally, when DRCgain is greater than 1.5-2.5, it switches to HDR mode; when DRCgain is less than 1.5, it switches back to standard mode. Simultaneously, when the scene has a high dynamic range, the AE module reduces overall exposure based on the bright target to ensure the bright area signal is not overexposed. In this case, the mid-tone brightness area will be darker, requiring the gain and tone mapping modules to brighten the mid-tone brightness area to a safe target brightness, ensuring the main subject of the image has normal brightness.

[0030] The ratio between the HCG and LCG of the DCG image sensor is defined as CGratio, with a value ranging from 2 to 4. To handle situations with higher dynamic range, an analog gain is applied to the HCG readout to improve CGratio. When the current scene's DRCgain ≤ CGratio, the HCG analog gain remains at 1, resulting in a low-noise, high-dynamic-range image. When DRCgain > CGratio, it indicates a larger dynamic range in the actual scene, requiring a higher reduction in exposure factor. In this case, an analog gain is added to the HCG to further improve the output dynamic range. The specific formula is as follows: CGratio*analoggain=DRCgain Wherein, analoggain is the maximum gain that can be achieved on HCG; The maximum value of DRCgain depends on the maximum gain achievable on the HCG multiplied by CGratio. However, considering noise issues in actual use, DRCgain is usually limited based on actual performance, typically not exceeding 16.

[0031] When the real-time DRCgain falls below the trigger threshold, the DCG image sensor switches back to the standard mode configuration.

[0032] There is a protection interval when switching between the two modes. For example, the protection interval DRCgain value is [2.5, 3.5], that is, when the system DRCgain > 3.5, it enters HDR mode, and when DRCgain < 2.5, it enters standard mode. At the same time, in order to avoid the problem of frequent switching between modes, the state must remain unchanged for 6 consecutive frames before switching can take place.

[0033] The DCG image sensor outputs raw HCG and LCG data respectively. The two 10-bit raw data are fused to generate high-bit data. The specific fusion method is as follows: Perform a left shift operation on the LCG 10-bit raw data. For pixel areas where the number after left shift is less than 1024, segment them according to the pixel value and use HCG raw data for fusion or directly replace them with HCG raw data.

[0034] Let the current fused pixel be (x, y). If HCG(x, y) is saturated, then: DCG(x,y)=LCG(x,y)*HCG / LCG, otherwise: DCG(x,y)=a*LCG(x,y)*HCG / LCG+(1-a)*HCG(x,y) Where 'a' is the weighting coefficient, which ranges from 0 to 1, and is typically between 0.3 and 0.7. When CGratio is 16, the fusion result is as follows: Figure 4 As shown.

[0035] Through such Figure 5 It can be seen that the HDR mode of the DCG image sensor can preserve details in dark areas while ensuring that highlights are not overexposed, and has a dynamic range that is significantly higher than that of the standard mode.

[0036] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A single-camera high dynamic range medical endoscopic image acquisition system, characterized in that, Including DCG image sensors; Two signal readouts are completed within a single frame exposure cycle using a DCG image sensor: the first readout reads the HCG signal with the LCG transistor off; The second reading is performed with the LCG transistor turned on, reading the LCG signal; by fusing the image data from the two readings, an image that simultaneously takes into account the sensor's full-well capacity and photosensitivity is obtained; Introduce a dynamic switching mechanism: The DCG image sensor is operating in standard mode configuration, waiting for the mode switching trigger signal from the automatic exposure module; The automatic exposure module calculates the overall brightness distribution of the image, and separately calculates the brightness distribution of the highlight area based on the overall brightness distribution. Based on the statistical results and the debugging parameters set by the automatic exposure module, the highlight target and the safe target are obtained; the ratio of the safe target to the highlight target is defined as DRCgain. When the scene dynamically increases, and DRCgain exceeds the trigger threshold and meets the AE stabilization mechanism conditions, the automatic exposure module sends a mode switching trigger signal, and the DCG image sensor switches to HDR mode configuration; at the same time, the automatic exposure module reduces the overall exposure according to the bright target, and brightens the intermediate brightness area to the target brightness through gain and tone mapping; when the real-time DRCgain is lower than the trigger threshold, the DCG image sensor switches back to standard mode configuration. The ratio between HCG and LCG of the DCG image sensor is defined as Cgratio; when DRCgain ≤ CGratio for the current scene, the HCG analog gain remains at 1; when DRCgain > CGratio, the analog gain is increased on the HCG, with the specific formula as follows: CGratio*analoggain=DRCgain Analoggain is the maximum gain that can be achieved on HCG.

2. The single-camera high dynamic range medical endoscope image acquisition system according to claim 1, characterized in that, The value of Cgratio ranges from 2 to 4.

3. The single-camera high dynamic range medical endoscope image acquisition system according to claim 1, characterized in that, DRCgain should not exceed 16.

4. The single-camera high dynamic range medical endoscope image acquisition system according to claim 1, characterized in that, There is a protection zone when switching between the two modes.

5. The single-camera high dynamic range medical endoscope image acquisition system according to claim 4, characterized in that, The protection interval DRCgain value is [2.5, 3.5]. That is, when the system DRCgain > 3.5, it enters HDR mode, and when DRCgain < 2.5, it enters standard mode.

6. The single-camera high dynamic range medical endoscope image acquisition system according to claim 1, characterized in that, When switching modes, the state must remain unchanged for 6 consecutive frames to trigger the switch.

7. The single-camera high dynamic range medical endoscope image acquisition system according to claim 1, characterized in that, The DCG image sensor outputs HCG and LCG image data respectively. The two image data are fused to generate high bit-count data. The specific fusion method is as follows: the LCG image data is left-shifted. For pixel areas with a left-shifted value of <1024, the HCG image data is used for fusion or directly replaced with HCG image data according to the pixel value.

8. The single-camera high dynamic range medical endoscope image acquisition system according to claim 7, characterized in that, it is provided with The current fused pixel is (x, y). If HCG(x, y) is saturated, then: DCG(x,y)=LCG(x,y)*HCG / LCG, otherwise: DCG(x,y)=a*LCG(x,y)*HCG / LCG+(1-a)*HCG(x,y) Where 'a' is the weighting coefficient.

9. The single-camera high dynamic range medical endoscope image acquisition system according to claim 1, characterized in that, The DCG image sensor outputs raw HCG and LCG data respectively, and the two 10-bit raw data are fused to generate high-bit data.

10. A method for acquiring high dynamic range images of a medical endoscope using single-frame dual-gain fusion, characterized in that, Includes the following: a. The DCG image sensor completes two signal readouts within a single frame exposure cycle: the first readout is performed with the LCG transistor off, reading the HCG signal; the second readout is performed with the LCG transistor on, reading the LCG signal; by fusing the image data from the two readouts, an image that simultaneously takes into account the sensor's full-well capacity and photosensitivity is obtained. b. Introducing a dynamic switching mechanism: The DCG image sensor operates in standard mode configuration, waiting for the mode switching trigger signal from the automatic exposure module; the automatic exposure module calculates the overall brightness distribution of the image, and simultaneously calculates the brightness distribution of the bright parts separately based on the overall brightness distribution. Based on the statistical results and the debugging parameters set by the automatic exposure module, the bright target and the safe target are obtained; the ratio of the safe target to the bright target is defined as DRCgain. When the scene dynamically increases, and DRCgain exceeds the trigger threshold and meets the AE stabilization mechanism conditions, the automatic exposure module sends a mode switching trigger signal, and the DCG image sensor switches to HDR mode configuration; at the same time, the automatic exposure module reduces the overall exposure according to the bright target, and brightens the intermediate brightness area to the target brightness through gain and tone mapping; when the real-time DRCgain is lower than the trigger threshold, the DCG image sensor switches back to standard mode configuration. c. Apply analog gain to HCG reading to increase Cgratio: The ratio between HCG and LCG of the DCG image sensor is defined as Cgratio; when DRCgain ≤ CGratio for the current scene, the HCG analog gain remains at 1; when DRCgain > CGratio, an analog gain is added to the HCG, using the following formula: CGratio*analoggain=DRCgain Analoggain is the maximum gain that can be achieved on HCG.

Images