Endoscopic image processing method and device based on hemoglobin concentration

By collecting image signals of different center wavelengths, calculating hemoglobin concentration and calculating gain coefficients, the contrast of bleeding points in endoscopic image processing is improved, and the problems of high cost and difficulty in equipment in the existing technology are solved, and low-cost and efficient The bleeding point highlights the effect.

CN118806210BActive Publication Date: 2024-11-22HANGZHOU LINGMOU MEDICAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411295574.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-18
Publication Date
2024-11-22
Estimated Expiration
2044-09-18

AI Technical Summary

Technical Problem

In the existing endoscopic image processing technology, methods based on tissue spectral characteristics are costly, and deep learning methods require a large amount of sample data and a powerful data processor, which leads to the cost and difficulty of highlighting bleeding points.

Method used

By collecting image signals of illuminating light of three different center wavelengths to the target area, the hemoglobin concentration is calculated using spectral inversion method, and the gain coefficients of the hemoglobin G channel and R channel are calculated based on the hemoglobin concentration, so as to improve the difference between the R channel and the G channel. Achieve high contrast display of bleeding points.

Benefits of technology

The cost of highlighting the color contrast of bleeding points is reduced, and the color contrast of bleeding points is improved through hemoglobin gain, which can effectively distinguish bleeding points and blood accumulation areas, simplify the number of spectral light-emitting devices, and realize real-time high-contrast image display.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118806210B_ABST
    Figure CN118806210B_ABST
Patent Text Reader

Abstract

The present application discloses an endoscopic image processing method and device based on hemoglobin concentration, and relates to the field of image processing technology. The method comprises collecting a first image signal, a second image signal and a third image signal obtained by irradiating a target area with illumination light of three different central wavelengths; using a spectral inversion method to calculate the spectral information of the target area according to the three image signals; filtering the spectral information based on the target wavelength to obtain a target image signal; calculating the hemoglobin concentration according to the target image signal and the third image signal; calculating the gain coefficient of the hemoglobin G channel and the gain coefficient of the R channel according to the hemoglobin concentration; using the second image signal as the B channel of the output image, multiplying the gain coefficient of the G channel with the target image signal as the G channel of the output image, and multiplying the gain coefficient of the R channel with the third image signal as the R channel of the output image. The present application can reduce the image processing cost of highlighting bleeding points.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an endoscopic image processing device, method and apparatus based on hemoglobin concentration. Background Art

[0002] In the medical field, for curative resection of superficial gastrointestinal lesions, surgery such as endoscopic submucosal dissection (ESD) is usually used. In this type of surgery, the doctor confirms the location of the relatively thick blood vessels while performing the incision operation to avoid severe bleeding caused by cutting the thick blood vessels with electric knives.

[0003] At present, in the field of endoscopic image processing, the technologies for highlighting bleeding points mainly include methods based on tissue spectral characteristics and deep learning methods. The method based on tissue spectral characteristics requires the integration of spectral light-emitting devices of multiple bands in the light source, which has the problem of high equipment cost. The deep learning method requires a large amount of sample data as input and needs to be equipped with a powerful data processor, so the equipment cost will increase significantly and it is difficult to implement. Summary of the invention

[0004] The purpose of the present application is to provide an endoscopic image processing device method and device based on hemoglobin concentration, which can reduce the image processing cost of highlighting bleeding points.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides an endoscopic image processing method based on hemoglobin concentration, comprising:

[0007] A first image signal, a second image signal and a third image signal are obtained by collecting illumination lights of three different central wavelengths to illuminate a target area; the first image signal, the second image signal and the third image signal are all image signals collected by an endoscope;

[0008] Using a spectral inversion method, the spectral information of the target area image is calculated according to the first image signal, the second image signal and the third image signal;

[0009] Filtering the spectral information based on the target wavelength to obtain a target image signal;

[0010] Calculating the hemoglobin concentration according to the target image signal and the third image signal;

[0011] Calculating a gain coefficient of a hemoglobin G channel and a gain coefficient of a hemoglobin R channel according to the hemoglobin concentration;

[0012] The second image signal is used as the B channel of the output image, the gain coefficient of the hemoglobin G channel is multiplied by the target image signal as the G channel of the output image, and the gain coefficient of the hemoglobin R channel is multiplied by the third image signal as the R channel of the output image.

[0013] Optionally, the three different central wavelengths are a first central wavelength, a second central wavelength and a third central wavelength, and the range of the first central wavelength is , the range of the second central wavelength is , the range of the third central wavelength is ,in, is the first central wavelength, is the second central wavelength, is the third central wavelength.

[0014] Optionally, when the first central wavelength is 430 nm, the second central wavelength is 540 nm, and the third central wavelength is 630 nm, the target wavelength is 600 nm.

[0015] Optionally, a calculation formula for calculating the hemoglobin concentration according to the target image signal and the third image signal is expressed as: ;

[0016] in, is the hemoglobin concentration, is the third image signal, is the target image signal.

[0017] Optionally, the calculation formula for obtaining the gain coefficient of the hemoglobin G channel according to the hemoglobin concentration is: ;

[0018] in, is the gain coefficient of the hemoglobin G channel, is the first constant, is the first conversion function, is the hemoglobin concentration.

[0019] Optionally, the calculation formula for obtaining the gain coefficient of the hemoglobin R channel according to the hemoglobin concentration is: ;

[0020] in, is the gain coefficient of the hemoglobin R channel, is the second constant, is the second conversion function.

[0021] Optionally, the first conversion function and the second conversion function are both exponential functions, linear functions or Gaussian functions.

[0022] Optionally, the light sources of the three illumination lights with different central wavelengths are LED light sources.

[0023] Optionally, after taking the second image signal as the B channel of the output image, multiplying the gain coefficient of the hemoglobin G channel with the target image signal as the G channel of the output image, and multiplying the gain coefficient of the hemoglobin R channel with the third image signal as the R channel of the output image, the endoscopic image processing method based on hemoglobin concentration further includes:

[0024] The output image is displayed in real time on a display.

[0025] In a second aspect, the present application provides an endoscopic image processing device based on hemoglobin concentration, comprising:

[0026] An image signal acquisition module, used for acquiring a first image signal, a second image signal and a third image signal obtained by irradiating a target area with illumination lights of three different central wavelengths;

[0027] a spectral information determination module, configured to calculate the spectral information of the target area image according to the first image signal, the second image signal and the third image signal by using a spectral inversion method;

[0028] A filtering module, used for filtering the spectral information based on a target wavelength to obtain a target image signal;

[0029] a hemoglobin concentration calculation module, configured to calculate the hemoglobin concentration according to the target image signal and the third image signal;

[0030] A gain coefficient calculation module, used for calculating the gain coefficient of the hemoglobin G channel and the gain coefficient of the hemoglobin R channel according to the hemoglobin concentration;

[0031] The output image determination module is used to use the second image signal as the B channel of the output image, multiply the gain coefficient of the hemoglobin G channel with the target image signal as the G channel of the output image, and multiply the gain coefficient of the hemoglobin R channel with the third image signal as the R channel of the output image.

[0032] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0033] The present application provides an endoscopic image processing device method and apparatus based on hemoglobin concentration, which calculates the gain coefficient of the hemoglobin green (G) channel and the gain coefficient of the hemoglobin red (R) channel according to the hemoglobin concentration; uses the second image signal as the blue (B) channel of the output image, multiplies the gain coefficient of the hemoglobin G channel with the target image signal as the G channel of the output image, and multiplies the gain coefficient of the hemoglobin R channel with the third image signal as the R channel of the output image, thereby achieving the difference between the R channel and the G channel by increasing the hemoglobin gain, thereby making the bleeding point a redder area, the hemoglobin concentration is high, and the color contrast at the bleeding point is higher than that at the blood accumulation area, thereby being able to distinguish the bleeding point from the blood accumulation area. The lighting light source of the present application is simple in design, and only three different central wavelengths need to be provided, thereby reducing the number of spectral light-emitting devices, thereby reducing the cost of highlighting the color contrast of the bleeding point. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A schematic flow chart of an endoscopic image processing method based on hemoglobin concentration provided in one embodiment of the present application;

[0036] Figure 2 A schematic diagram of a data processing flow of an endoscopic image processing method based on hemoglobin concentration provided in one embodiment of the present application;

[0037] Figure 3 A schematic diagram of functional modules of an endoscopic image processing device based on hemoglobin concentration provided in one embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0040] The present application provides an endoscopic image processing method based on hemoglobin concentration, such as Figure 1 As shown, the endoscopic image processing method based on hemoglobin concentration includes the following steps.

[0041] Step 101: Collect a first image signal, a second image signal and a third image signal obtained by irradiating a target area with illumination light of three different central wavelengths; the first image signal, the second image signal and the third image signal are all image signals collected through an endoscope.

[0042] Step 102: using a spectral inversion method to calculate spectral information of the target area image according to the first image signal, the second image signal and the third image signal.

[0043] Step 103: filtering the spectral information based on the target wavelength to obtain a target image signal.

[0044] Step 104: Calculate the hemoglobin concentration according to the target image signal and the third image signal.

[0045] Step 105: Calculate the gain coefficient of the hemoglobin G channel and the gain coefficient of the hemoglobin R channel according to the hemoglobin concentration.

[0046] Step 106: Use the second image signal as the B channel of the output image, multiply the gain coefficient of the hemoglobin G channel by the target image signal to obtain the G channel of the output image, and multiply the gain coefficient of the hemoglobin R channel by the third image signal to obtain the R channel of the output image.

[0047] The target area in this application is the area that needs to be observed in real time through the endoscope.

[0048] The present application calculates the gain coefficient of the hemoglobin G channel and the gain coefficient of the hemoglobin R channel according to the hemoglobin concentration; uses the second image signal as the B channel of the output image, multiplies the gain coefficient of the hemoglobin G channel with the target image signal as the G channel of the output image, and multiplies the gain coefficient of the hemoglobin R channel with the third image signal as the R channel of the output image, thereby achieving the difference between the R channel and the G channel by increasing the hemoglobin gain, thereby making the bleeding point a redder area, the hemoglobin concentration is higher, and the color contrast at the bleeding point is higher than that at the blood accumulation area, thereby being able to distinguish between the bleeding point and the blood accumulation area, and being able to achieve the effect of highlighting the color contrast of the bleeding area and the blood accumulation area in endoscopic surgery. The lighting light source of the present application is simple in design, and the number of spectral light-emitting devices is small, thereby reducing the cost of highlighting the color contrast of the bleeding point.

[0049] In addition, the present application displays the output image in real time through a display, wherein the display is a display of an endoscope. The present application has a simple image processing process and can display a high-contrast image of a bleeding point on the display in real time with low latency. By switching the imaging mode, the doctor can intuitively see a real-time image highlighting the bleeding point on the display.

[0050] The three different central wavelengths are the first central wavelength, the second central wavelength and the third central wavelength. The range of the first central wavelength is , the range of the second central wavelength is , the range of the third central wavelength is ,in, is the first central wavelength, is the second central wavelength, is the third central wavelength.

[0051] In an exemplary embodiment, when the first central wavelength is 430 nm, the second central wavelength is 540 nm, and the third central wavelength is 630 nm, the target wavelength is 600 nm. Figure 2 As shown, the first image signal B1 obtained by illumination with a central wavelength of 430nm, the second image signal G1 obtained by illumination with a central wavelength of 540nm, and the third image signal R1 obtained by illumination with a central wavelength of 630nm. In the spectrum inversion unit, the spectrum information of the target area image is calculated according to the spectrum inversion method from the first image signal B1, the second image signal G1, and the third image signal R1. In the 600nm calculation unit, the 600nm target image signal A1 is filtered according to the spectrum inversion method (for example, using a 3×3 digital filter matrix) based on the spectrum information calculated by the spectrum inversion unit. In the hemoglobin concentration calculation unit, the hemoglobin concentration information is calculated according to the hemoglobin concentration calculation formula (Formula 1) from the filtered 600nm target image signal A1 and the third image signal R1. In the hemoglobin R gain unit, the R channel gain coefficient is calculated according to Formula (2) from the hemoglobin concentration information, and the R channel gain coefficient is multiplied by the input third image signal R1 to obtain the output R channel signal. In the hemoglobin G gain section, the gain of the G channel is calculated according to the hemoglobin concentration information according to formula (3), and the G channel coefficient is multiplied by the input 600nm target image signal A1 to obtain the output G channel signal.

[0052] The present application obtains spectral information of about 600nm by the method of spectral inversion, and can highlight bleeding points and deep blood vessels by the difference between the inverted spectral information of 600nm and the red light spectral information of about 630nm. Amber light is about 600nm, and the present application can highlight bleeding points by the difference between amber light and red light at 600nm.

[0053] In an exemplary embodiment, the light source of the illumination light outputs a narrow-band blue light with a first central wavelength of 430 nm and a half-width of 20 nm, a broadband green light with a second central wavelength of 540 nm and a half-width of 96 nm, and a narrow-band red light with a third central wavelength of 625 nm and a half-width of 20 nm. The first central wavelength may also be selected as a narrow-band blue-violet light with a half-width of 415 nm and a half-width of 20 nm.

[0054] The light sources of the three illumination lights with different central wavelengths are LED light sources, but all or part of the plurality of light sources may be LEDs.

[0055] The calculation formula for calculating the hemoglobin concentration according to the target image signal and the third image signal is expressed as:

[0056] (1)

[0057] in, is the hemoglobin concentration, is the third image signal, is the target image signal.

[0058] The calculation formula for the gain coefficient of the hemoglobin R channel calculated according to the hemoglobin concentration is:

[0059] (2)

[0060] in, is the gain coefficient of the hemoglobin R channel, is the second constant, is the second transfer function, is the hemoglobin concentration.

[0061] The calculation formula for the gain coefficient of the hemoglobin G channel calculated according to the hemoglobin concentration is:

[0062] (3)

[0063] in, is the gain coefficient of the hemoglobin G channel, is the first constant, is the first conversion function, is the hemoglobin concentration.

[0064] The first conversion function and the second conversion function are both exponential functions, S-curve functions, linear functions or Gaussian functions. The first conversion function and the second conversion function may be the same or different functions. For example, the exponential function takes the IHB power of 10. The determination of the first conversion function and the second conversion function depends on the expected effect of the enhancement.

[0065] The gain effect of the present application is as follows: in the area with high hemoglobin concentration (such as the bleeding point), the hemoglobin gain part can increase the difference between the R and G channels, so that the bleeding point becomes a redder area; in the area with high hemoglobin concentration, but not as high as the bleeding point (such as the blood accumulation area), the hemoglobin R gain and G gain are both smaller than the values ​​at the bleeding point, that is, the color contrast at the bleeding point is higher than that in the blood accumulation area, so that the bleeding point and the blood accumulation area can be distinguished; in the area with low hemoglobin concentration, the value of the hemoglobin gain part is almost equal to 1, and because the second image signal G1 is assigned to the B channel, the image of the area with low hemoglobin concentration has almost no change, so that the blood accumulation area and the non-blood accumulation area can be distinguished, helping doctors to distinguish between the blood accumulation area and the non-blood accumulation area, thereby guiding more accurate treatment and shortening the hemostasis time and the operation time. Among them, the blood accumulation area refers to the area where the hemoglobin concentration is lower than the bleeding point after the blood is diluted.

[0066] The image processing method of the present application can be applied to a variety of medical scenarios, including but not limited to inspections and surgeries of the digestive tract, respiratory tract, etc., and has a wide range of uses.

[0067] Based on the same inventive concept, the embodiment of the present application also provides an endoscopic image processing device based on hemoglobin concentration for implementing the endoscopic image processing method based on hemoglobin concentration involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more endoscopic image processing devices based on hemoglobin concentration provided below can refer to the limitations of the endoscopic image processing method based on hemoglobin concentration above, and will not be repeated here.

[0068] In an exemplary embodiment, Figure 3 As shown, an endoscopic image processing device based on hemoglobin concentration is provided, which includes the following functional modules.

[0069] The image signal acquisition module 201 is used to acquire a first image signal, a second image signal and a third image signal obtained by irradiating a target area with illumination lights of three different central wavelengths.

[0070] The spectral information determination module 202 is used to calculate the spectral information of the target area image according to the first image signal, the second image signal and the third image signal by using a spectral inversion method.

[0071] The filtering module 203 is used to filter the spectral information based on the target wavelength to obtain a target image signal.

[0072] The hemoglobin concentration calculation module 204 is used to calculate the hemoglobin concentration according to the target image signal and the third image signal.

[0073] The gain coefficient calculation module 205 is used to calculate the gain coefficient of the hemoglobin G channel and the gain coefficient of the hemoglobin R channel according to the hemoglobin concentration.

[0074] The output image determination module 206 is used to use the second image signal as the B channel of the output image, multiply the gain coefficient of the hemoglobin G channel by the target image signal as the G channel of the output image, and multiply the gain coefficient of the hemoglobin R channel by the third image signal as the R channel of the output image.

[0075] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0076] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. An endoscopic image processing method based on hemoglobin concentration, characterized in that: The endoscopic image processing method based on hemoglobin concentration includes: A first image signal, a second image signal and a third image signal are obtained by collecting illumination lights of three different central wavelengths to illuminate a target area; the first image signal, the second image signal and the third image signal are all image signals collected by an endoscope; Using a spectral inversion method, the spectral information of the target area image is calculated according to the first image signal, the second image signal and the third image signal; Filtering the spectral information based on the target wavelength to obtain a target image signal; Calculating the hemoglobin concentration according to the target image signal and the third image signal; Calculating a gain coefficient of a hemoglobin G channel and a gain coefficient of a hemoglobin R channel according to the hemoglobin concentration; The second image signal is used as the B channel of the output image, the gain coefficient of the hemoglobin G channel is multiplied by the target image signal as the G channel of the output image, and the gain coefficient of the hemoglobin R channel is multiplied by the third image signal as the R channel of the output image.

2. The endoscopic image processing method based on hemoglobin concentration according to claim 1, characterized in that: The three different central wavelengths are the first central wavelength, the second central wavelength and the third central wavelength. The range of the first central wavelength is , the range of the second central wavelength is , the range of the third central wavelength is ,in, is the first central wavelength, is the second central wavelength, is the third central wavelength.

3. The endoscopic image processing method based on hemoglobin concentration according to claim 2, characterized in that: When the first central wavelength is 430 nm, the second central wavelength is 540 nm, and the third central wavelength is 630 nm, the target wavelength is 600 nm.

4. The endoscopic image processing method based on hemoglobin concentration according to claim 1, characterized in that: The calculation formula for calculating the hemoglobin concentration according to the target image signal and the third image signal is expressed as: ; in, is the hemoglobin concentration, is the third image signal, is the target image signal.

5. The endoscopic image processing method based on hemoglobin concentration according to claim 1, characterized in that: The calculation formula for the gain coefficient of the hemoglobin G channel calculated according to the hemoglobin concentration is: ; in, is the gain coefficient of the hemoglobin G channel, is the first constant, is the first conversion function, is the hemoglobin concentration.

6. The endoscopic image processing method based on hemoglobin concentration according to claim 5, characterized in that: The calculation formula for the gain coefficient of the hemoglobin R channel calculated according to the hemoglobin concentration is: ; in, is the gain coefficient of the hemoglobin R channel, is the second constant, is the second conversion function.

7. The endoscopic image processing method based on hemoglobin concentration according to claim 6, characterized in that: The first conversion function and the second conversion function are both exponential functions, linear functions or Gaussian functions.

8. The endoscopic image processing method based on hemoglobin concentration according to claim 1, characterized in that: The light sources of the three illumination lights with different central wavelengths are LED light sources.

9. The endoscopic image processing method based on hemoglobin concentration according to claim 1, characterized in that: After taking the second image signal as the B channel of the output image, multiplying the gain coefficient of the hemoglobin G channel by the target image signal as the G channel of the output image, and multiplying the gain coefficient of the hemoglobin R channel by the third image signal as the R channel of the output image, the endoscopic image processing method based on hemoglobin concentration further includes: The output image is displayed in real time on a display.

10. An endoscopic image processing device based on hemoglobin concentration, characterized in that: The endoscopic image processing device based on hemoglobin concentration includes: An image signal acquisition module, used for acquiring a first image signal, a second image signal and a third image signal obtained by irradiating a target area with illumination lights of three different central wavelengths; a spectral information determination module, configured to calculate the spectral information of the target area image according to the first image signal, the second image signal and the third image signal by using a spectral inversion method; A filtering module, used for filtering the spectral information based on a target wavelength to obtain a target image signal; a hemoglobin concentration calculation module, configured to calculate the hemoglobin concentration according to the target image signal and the third image signal; A gain coefficient calculation module, used for calculating the gain coefficient of the hemoglobin G channel and the gain coefficient of the hemoglobin R channel according to the hemoglobin concentration; The output image determination module is used to use the second image signal as the B channel of the output image, multiply the gain coefficient of the hemoglobin G channel with the target image signal as the G channel of the output image, and multiply the gain coefficient of the hemoglobin R channel with the third image signal as the R channel of the output image.