Endoscope device and application method based on characteristic spectral imaging of white light LED illumination

Abstract

The present invention relates to an endoscope device and an application method based on characteristic spectral imaging of white light LED illumination. The device includes an endoscope handle, a connecting wire, a host computer, and a display. The image collected by the endoscope handle is transmitted to the host computer through the connecting wire, and after being processed by the host computer, it is transmitted to the display. The endoscope handle includes a white light LED for illuminating the detection scene and a CMOS sensor for imaging the detection scene. Compared with the prior art, the present invention solves the problems of the traditional technology that requires a CCD sensor and a narrow-band illumination design. For the first time, under common hardware requirements, the characteristic spectral imaging technology is realized, so that the lesion recognition ability of this technology is extended to the disposable endoscope application that can avoid cross-infection, etc.

Description

Technical Field

[0001] The present invention relates to the field of endoscopes, and particularly to an endoscope device and an application method based on characteristic spectral imaging of white light LED illumination. Background Art

[0002] In traditional endoscope applications, using a narrow-band characteristic spectral light source to image tissues helps to detect lesions such as early cancer and polyps, and is an important means for endoscopic screening. Among many applications, the narrow-band imaging technology (NBI) proposed by Olympus and the BLI and LCI technologies proposed by Fujifilm Corporation are the most famous.

[0003] NBI is a technology that uses 415 and 540 nm blue-green narrow-band light generated by a filter to illuminate and image the detection scene. Since hemoglobin has a peak absorption effect on the light in these two wavelength bands, using this kind of illumination can identify the mucosal structure rich in vascular tissues and thus detect early lesions.

[0004] The BLI technology uses a narrow-band light source with a wavelength emphasizing blood vessels in the range of 440 - 460 nm to illuminate and image the scene. Using the BLI technology can achieve the same detection level as NBI, and the detection performance can be enhanced by increasing the illumination intensity.

[0005] The LCI technology is a technology that combines white light and narrow-band wavelength light to illuminate and image the scene. By adjusting and converting the white light and narrow-band light, the subtle color difference near the mucosa can be more easily identified.

[0006] Although the above technologies can enhance the lesion characteristics by using the characteristics of narrow-band light illumination. However, such systems all need to use a CCD as an image acquisition sensor and require a relatively complex light source design. This will significantly increase the complexity and cost of constructing the detection system, and reduce the application of characteristic spectral imaging in the field of endoscopes (especially disposable endoscopes). Therefore, the problem of the traditional characteristic spectral imaging technology requiring high-cost sensors and special narrow-band light illumination becomes the technical problem to be solved by the present invention. Summary of the Invention

[0007] The purpose of the present invention is to overcome the defects of the above-mentioned existing technologies and provide an endoscope device and an application method based on characteristic spectral imaging of white light LED illumination.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] According to one aspect of the present invention, an endoscope device based on white light LED illumination characteristic spectral imaging is provided, including an endoscope handle, a connecting wire, a host computer and a display. The image collected by the endoscope handle is transmitted to the host computer through the connecting wire, and after being processed by the host computer, it is transmitted to the display. The endoscope handle includes a white light LED for illuminating the detection scene and a CMOS sensor for imaging the detection scene.

[0010] As a preferred technical solution, a light source controller for periodically controlling the illumination brightness of the white light LED is built in the connecting wire.

[0011] As a preferred technical solution, the host computer includes a video input, an FPGA and a video output;

[0012] The video input is used to receive the image collected from the CMOS sensor transmitted by the connecting wire; the FPGA is used to provide instructions for the light source controller to guide it to complete the white light LED control, and convert the collected image according to the white light LED spectrum and sensor characteristics; the video output is used to transmit the image processed by the FPGA to the display.

[0013] As a preferred technical solution, a micro filter is integrated on the surface of the pixels of the CMOS sensor, so that each pixel of the sensor can image light of different wavelengths respectively.

[0014] According to another aspect of the present invention, an application method of the endoscope device based on white light LED illumination characteristic spectral imaging is provided, and this method includes the following steps:

[0015] Step 1, image acquisition: The CMOS sensor acquires the original image of the detection scene;

[0016] Step 2, original image: The original image is transmitted to the FPGA by the connector;

[0017] Step 3, image processing: The FPGA performs preliminary image processing operations;

[0018] Step 4, color transformation: The FPGA performs color transformation of characteristic spectral imaging on the processed image;

[0019] Step 5, result image: The FPGA generates the result image.

[0020] As a preferred technical solution, the specific content of the said step 3 is:

[0021] Perform denoising, structure enhancement, dynamic range correction and enhancement, color correction and enhancement processing on the original image collected by the CMOS sensor.

[0022] As a preferred technical solution, step 4 specifically includes:

[0023] Step 4.1: The gray values obtained by different color pixels of the CMOS sensor are expressed as follows:

[0024]

[0025] where GV c is the gray value obtained by the CMOS pixel, λ represents the wavelength, λ l , λ h are the highest and lowest wavelengths of the light source, Q e is the relative quantum efficiency of the current pixel for light of a specific wavelength, φ s is the relative luminous flux of light of a specific wavelength in the light source, V λ is the flux power conversion coefficient of light of a specific wavelength, P is the luminous power of the light source, and I is the proportional relationship between the light power received by the sensor and the pixel gray value;

[0026] Step 4.2: Convert the formula in step 4.1:

[0027]

[0028] where GV g and GV b represent the gray values of the green and blue pixel points recovered from the CMOS sensor after being processed in step 4.1, and represent the green and blue pixel values recovered from the above formula;

[0029] Step 4.3: After obtaining the recovered green and blue gray values, perform color transformation using the following formula:

[0030]

[0031] where and represent the gray values after color transformation of blue and green, and represent the quantum conversion efficiencies of the sensor for light at 540 nm and 415 nm respectively, and represent the relative luminous fluxes of the light source at 540 nm and 415 nm respectively, V 54 0 and V 415 represent the flux power conversion ratios of light at 540 nm and 415 nm; thus, the color transformation of blue and green light is completed;

[0032] Step 4.4: The conversion results of the above blue and green light are summarized into the following two formulas:

[0033]

[0034] Using the above formula, the color conversion of cyan light can be completed;

[0035] Step 4.5: To ensure the color balance of the image, the following processing is performed on the red channel:

[0036]

[0037] In the above formula, represents the pixel value of the R channel after conversion, and respectively represent the cyan and green pixel values after color conversion.

[0038] As a preferred technical solution, step 4.2 is used to eliminate the components in the gray value that do not belong to green and blue, that is, to obtain the ratio of the responses of the green and blue wavelengths to the full-band light range sensor and the light source distribution, so as to restore the original gray value in the green and blue bands.

[0039] As a preferred technical solution, step 4.3 can convert the conversion ratio of the light power gray value of green light from 505nm to 566nm into the conversion ratio of 540nm light; similarly, blue light from 350nm to 480nm is also converted into the conversion ratio of 415nm light.

[0040] As a preferred technical solution, in practical applications, the following discrete form is also applicable to step 4.4:

[0041]

[0042] where Δλ is the difference in discrete light wavelengths.

[0043] Compared with the prior art, the present invention has the following advantages:

[0044] 1. Based on the traditional characteristic spectral imaging technology, the present invention proposes a characteristic spectral imaging technology composed of a CMOS sensor and a white LED.

[0045] 2. The present invention successfully solves the problem that the traditional technology requires a CCD sensor and a narrow-band illumination design. For the first time, under common hardware requirements, the characteristic spectral imaging technology is realized, so that the lesion recognition ability of the technology is extended to the disposable endoscope application that can avoid cross-infection.

[0046] 3. The present invention improves the problem that the traditional characteristic spectral imaging technology requires a high-cost sensor and special narrow-band light illumination, making the method proposed by the invention more suitable for the disposable endoscope application scenario. Description of the Drawings

[0047] Figure 1Schematic diagram of the device of the present invention;

[0048] Figure 2 Flow chart of the method of the present invention;

[0049] Figure 3 Spectral response curve of the Micron MT9T031 - C sensor;

[0050] Figure 4 Illumination spectral distribution diagram of the LED.

[0051] Where 1 is the endoscope handle, 11 is the white - light LED, 12 is the CMOS sensor, 2 is the connecting wire, 21 is the light source controller, 3 is the host computer, 31 is the video input, 32 is the FPGA, 33 is the video output, and 4 is the display. Specific embodiments

[0052] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0053] As Figure 1 shown, an endoscope device based on white - light LED illumination characteristic spectral imaging includes an endoscope handle 1, a connecting wire 2, a host computer 3, and a display 4. The image collected by the endoscope handle 1 is transmitted to the host computer 3 through the connecting wire 2 and then transmitted to the display 4 after being processed by the host computer 3. The endoscope handle 11 includes a white - light LED 11 and a CMOS sensor 12. The white - light LED 11 is used to illuminate the detection scene, and the CMOS sensor 12 is used to image the detection scene.

[0054] The connecting wire 2 includes a light source controller 21, and the light source controller 21 is used to periodically control the illumination brightness of the white - light LED. The host computer 3 includes a video input 31, an FPGA 32, and a video output 33. The video input 31 is used to receive the image collected from the CMOS sensor 12 transmitted by the connecting wire; the FPGA 32 is used to provide instructions for the light source controller to guide it to complete the white - light LED control and convert the collected image according to the white - light LED spectrum and Sensor characteristics; the video output 33 is used to transmit the image processed by the FPGA to the display 4.

[0055] As Figure 2 shown, a method using the endoscope device based on white - light LED illumination characteristic spectral imaging includes the following steps:

[0056] Step 1, Image Acquisition: The CMOS sensor acquires the original image of the detection scene;

[0057] Step 2, Original Image: The original image is transmitted to the FPGA by the connector;

[0058] Step 3, Image Processing: The FPGA performs preliminary image processing operations;

[0059] Step 4, Color Transformation: The FPGA performs color transformation for feature spectral imaging on the processed image;

[0060] Step 5, Resultant Image: The FPGA generates the resultant image;

[0061] Traditional Feature Spectral Imaging Principle:

[0062] Generally, a micro filter is integrated on the surface of each pixel of the CMOS sensor, enabling each pixel of the sensor to image light of different wavelengths respectively. The imaging ability of the pixel can be reflected by the spectral response curve of the sensor. As Figure 3 shown, taking the Micron MT9T031-C sensor as an example, in the figure, the abscissa is the wavelength of the absorbed light, and the ordinate is the photoelectric conversion efficiency. This figure can illustrate that pixels loaded with different filters have different absorption effects on light of different wavelengths, resulting in different photoelectric conversion efficiencies of the sensor for light of different wavelengths.

[0063] In addition, the distribution of the LED light source power at each wavelength is also different. As Figure 4 shown, taking the Osram GWVJLPE1.EM light source as an example, the lighting spectral distribution of this LED is shown, where the abscissa is the wavelength of the emitted light, and the ordinate is the relative luminous flux. It can be seen from the figure that the relative luminous flux of this light source is low in the blue light band and high in the red light band.

[0064] Based on the traditional feature spectral imaging principle and combining the characteristics of the CMOS sensor and the lighting source, the present invention proposes the following color conversion method:

[0065] Step 4.1, The gray value obtained by CMOS pixels of different colors is related to the following formula:

[0066]

[0067] In the above formula, GV c is the gray value obtained by the CMOS pixel, λ l λ h is the maximum and minimum wavelengths of the light source (generally 350nm - 800nm), Q e is the relative quantum efficiency of the current pixel for light of a specific wavelength, φ sis the relative luminous flux of light with a specific wavelength in the light source, V λ is the flux power conversion coefficient of light with a specific wavelength, P is the luminous power of the light source, and I is the proportional relationship between the light power received by the sensor and the pixel gray value.

[0068] Step 4.2: In order to make the values of each pixel similar to those in traditional characteristic spectral imaging applications, the following conversion formula is proposed:

[0069]

[0070] In the above formula, GV g and GV b represent the gray values of the green and blue pixel points recovered from the CMOS sensor in Step 4.1 (i.e., the blue and green gray values of the image obtained after interpolation of the original image data). and represent the green and blue pixel values recovered by the color conversion performed by the above formula. Since the sensor will receive components outside the green and blue spectra during the acquisition process in the blue and green channels, this step is used to eliminate the components in the gray values that do not belong to green and blue, that is, to obtain the ratio of the responses of the green and blue wavelengths to the full-band light range sensor and the light source distribution, so as to recover the original gray values within the green and blue bands.

[0071] Step 4.3: After obtaining the recovered green and blue gray values, in order to make the green and blue spectra able to image the illumination performance of the 540 and 415 nm characteristic spectra, the following formula is used for color transformation:

[0072]

[0073] In the above formula, and represent the gray values after the blue and green color transformations, and represent the quantum conversion efficiencies of the sensor for light at 540 nm and 415 nm respectively, and represent the relative luminous fluxes of the light source at 540 nm and 415 nm respectively, V 540 and V 415 represent the flux power conversion ratios of light at 540 nm and 415 nm. Using the above formula, the conversion ratio of the light power gray value of green light from 505 nm to 566 nm can be converted into the conversion ratio of light at 540 nm. Similarly, blue light from 350 nm to 480 nm can also be converted into the conversion ratio of light at 415 nm. Thus, the color transformation of blue and green light is completed.

[0074] Step 4.4: The conversion results of the above blue and green light are summarized into the following two formulas:

[0075]

[0076] In practical applications, the following discrete form is also applicable:

[0077]

[0078] In the above formula, Δλ is the difference in discrete optical wavelengths. By applying the above formula, the color conversion of blue-green light can be completed.

[0079] Step 4.5: To ensure the color balance of the image, the following processing is performed on the red channel:

[0080]

[0081] In the above formula, represents the pixel value of the converted R channel, and respectively represent the blue and green pixel values after color conversion. By using the above formula, the entire process of color conversion can be completed.

[0082] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An application method of an endoscope device based on white light LED illumination characteristic spectral imaging, characterized in that, The method includes the following steps: Step 1, Image acquisition: The CMOS sensor acquires the original image of the detection scene; Step 2, Original image: The original image is transmitted to the FPGA by the connector; Step 3, Image processing: The FPGA performs preliminary image processing operations; Step 4, Color transformation: The FPGA performs color transformation of feature spectral imaging on the processed image; Step 5, Resultant image: The FPGA generates the resultant image; The specific content of step 4 includes: Step 4.1, The gray values obtained by different color pixels of the CMOS sensor are represented as shown in the following formula: Among them, GV c is the gray value obtained by the CMOS pixel, λ represents the wavelength, λ l , λ h are the maximum and minimum wavelengths of the light source, Q e is the relative quantum efficiency of the current pixel for light of a specific wavelength, φ s is the relative luminous flux of light of a specific wavelength in the light source, V λ is the flux power conversion coefficient of light of a specific wavelength, P is the luminous power of the light source, and I is the proportional relationship between the light power received by the sensor and the pixel gray value; Step 4.2, Convert the formula in step 4.1: Among them, GV g and GV b represent the gray scale values of the green and blue pixel points recovered from the CMOS sensor after being processed in step 4.1, and represent the green and blue pixel values recovered by the above formula; After obtaining the restored green and blue gray values, perform color transformation using the following formula: where and represent the gray values after the color transformation of blue and green, and represent the quantum conversion efficiencies of the sensor for light at 540 nm and 415 nm, respectively, and represent the relative luminous fluxes of the light source at 540 nm and 415 nm, V 540 and V 415 represent the flux power conversion ratios of light at 540 nm and 415 nm; thus, the color transformation of blue and green light is completed; Step 4.4, The conversion results of the above blue-green light are summarized into the following two formulas: Using the above formulas, the color conversion of blue-green light can be completed; Step 4.5, In order to ensure the color balance of the image, the following processing is performed on the red channel: In the above formula, represents the R-channel pixel value after conversion, and respectively represent the blue and green pixel values after color conversion; Step 4.2 is used to eliminate the components in the gray values that do not belong to green and blue, that is, to obtain the ratio of the responses of the green and blue wavelengths to the full-band light range sensor and the light source distribution, so as to restore the original gray values within the green and blue bands.

2. The method according to claim 1, characterized in that, The specific content of step 3 is: Perform noise reduction, structure enhancement, dynamic range correction and enhancement, and color correction and enhancement processing on the original image acquired by the CMOS sensor.

3. The method according to claim 1, characterized in that, Step 4.3 can convert the conversion ratio of the light power gray value of green light from 505nm to 566nm into the conversion ratio of 540nm light; similarly, blue light from 350nm to 480nm is also converted into the conversion ratio of 415nm light.

4. The method according to claim 1, characterized in that, In practical applications, the following discrete form is also applicable to step 4.4: where Δλ is the difference in discrete light wavelengths.

5. An endoscope device for the method according to claim 1, comprising an endoscope handle, a connecting line, a host computer, and a display. Images collected by the endoscope handle are transmitted to the host computer through the connecting line and then transmitted to the display after being processed by the host computer. It is characterized in that, The endoscope handle described above includes a white light LED for illuminating the detection scene and a CMOS sensor for imaging the detection scene.

6. The endoscopic device according to claim 5, characterized in that, The connecting wire described above is internally provided with a light source controller for periodically controlling the illumination brightness of the white light LED.

7. The endoscopic device according to claim 5, characterized in that, The host described above includes a video input, an FPGA, and a video output; The video input is used to receive the image acquired from the CMOS sensor transmitted by the connecting wire; the FPGA is used to provide instructions for the light source controller to guide it to complete the white light LED control, and convert the acquired image according to the white light LED spectrum and sensor characteristics; the video output is used to transmit the image processed by the FPGA to the display.

8. The endoscope apparatus according to claim 5, characterized in that, The surface of the pixels of the CMOS sensor described above is integrated with a micro filter, so that each pixel of the sensor can image light of different wavelengths respectively.

Images