An adaptive enhancement method for endoscopic images

Abstract

The present invention discloses an adaptive enhancement method for endoscopic images, which belongs to the interdisciplinary technical field of combining computer vision and medicine. The present invention uses a guided filter to filter the endoscopic image to obtain a low-frequency component with good edge and gradient preservation, and constructs an illumination gain function based on the low-frequency components of different regions of the image to adaptively generate an illumination gain factor; and constructs a gradient gain function based on the Laplace operator convolution results of different regions of the endoscopic image to adaptively generate a gradient gain factor; finally, the illumination gain factor and the gradient gain factor are combined to generate the adaptive gain coefficient required by the USM method to complete the improvement of the USM method. Experiments have shown that when the present invention is applied to the enhancement of small intestinal endoscopy images, it can effectively enhance the detail information of the small intestinal villi while suppressing the dark area noise and preventing the edge overshoot phenomenon, effectively highlighting the edge detail information with tiny villi structure.

Description

Technical Field

[0001] The present invention relates to an endoscope image adaptive enhancement method, belonging to the cross-disciplinary technical field combining computer vision and medicine. Background Art

[0002] Wireless Capsule Endoscopy (WCE) can effectively observe the patient's gastrointestinal information. However, due to the limited lighting power of WCE and the complex gastrointestinal environment, the images taken by WCE are often unclear, especially in the small intestine covered with tiny villi. The surface of the small intestine is covered with tiny villi used to absorb nutrients into the blood vessels, resulting in blurred images of the small intestine taken by WCE, insufficient or even disappearance of villus information. Therefore, it is necessary to enhance the WCE small intestine images to highlight the edge details of the tiny villus structure, help doctors better observe the images, and provide support for real-time navigation planning based on WCE-acquired images, and later three-dimensional reconstruction of the small intestine.

[0003] Currently, there are many methods for image enhancement. Common classic methods fall into three categories: histogram equalization (HE), Retinex-based methods, and unsharping mask (USM). HE-based methods achieve image enhancement by remapping the grayscale of the input image based on its probability distribution. However, their enhancement results suffer from issues such as insufficient enhancement, excessive enhancement, and severe noise amplification. Retinex-based methods treat the image as a combination of illumination and reflectance components and enhance the image by adjusting the corresponding components. However, the effect of Retinex-based methods on highlighting image detail information is not very ideal. The USM method, on the other hand, has a significant advantage in emphasizing image detail information. Specifically, the traditional USM method uses a fixed gain factor to enhance high-frequency components. Subsequently, some scholars have improved the USM method and proposed an adaptive gain factor based on local image information. In addition, in the past decade, many scholars have proposed methods for endoscopic image enhancement based on USM, but these methods do not consider the balance between detail enhancement and noise suppression, resulting in edge overshoot and noise amplification problems in the enhancement results. Therefore, they are not directly suitable for the generation of endoscopic small intestine characteristic gain factors. Summary of the Invention

[0004] In order to solve the problem that the image enhancement effect of wireless capsule endoscope is not obvious, the present invention provides an endoscope image adaptive enhancement method, and the technical solution is as follows:

[0005] A first object of the present invention is to provide an endoscopic image adaptive enhancement method, comprising:

[0006] Step 1: Obtain an original endoscopic image, filter and smooth the original endoscopic image using a guided filter to obtain its low-frequency component, and then subtract the filtered low-frequency component from the original endoscopic image to obtain a high-frequency component;

[0007] Step 2: constructing an illumination gain function based on the low-frequency components of different regions of the original endoscopic image to adaptively generate an illumination gain factor w1, thereby enhancing the high-frequency components while suppressing the noise in the dark region;

[0008] Step 3: Based on the Laplacian operators of different regions of the endoscopic image, convolution is performed to obtain a gradient gain function to adaptively generate a gradient gain factor w2;

[0009] Step 4: Perform matrix multiplication on the illumination gain factor w1, the gradient gain factor w2 and the high-frequency component to obtain a gain high-frequency component, and then superimpose the gain high-frequency component on the original image to enhance the endoscopic image.

[0010] Optionally, the illumination gain factor w1 is obtained by an illumination gain function, and the illumination gain function is:

[0011]

[0012] Where (x, y) is the image pixel coordinate, is the image of the original endoscopic image after being filtered by a low-pass filter, and Mean is The mean of .

[0013] Optionally, the gradient gain factor w2 is calculated as follows:

[0014]

[0015] in, It is the absolute value of the result after the original image is convolved with the Laplacian operator template and normalized, representing the edge information of the image.

[0016] Optionally, the enhanced image obtained in step 4 is:

[0017]

[0018] k=α·w1·w2

[0019] Where ZI(x, y) is the enhanced image, I(x, y) is the original endoscopic image, and α is the control maximum value parameter.

[0020] Optionally, the guided filter includes an input image p, a guided image G, and an output image q, and the linear relationship between the guided image G and the output image q is:

[0021]

[0022] Among them, ω m It is a square window with the center of the window located at m; a m and b m is the linear function coefficient, and the input image p is the image to be filtered;

[0023] Then a is obtained by minimizing the linear cost function m and b m , thereby obtaining the output image q, the linear cost function is:

[0024]

[0025] Among them, ε is to prevent a m Too large regularization factor.

[0026] Optionally, step 1 first converts the endoscopic image from the RGB color space to the HSI color space, and then uses the guided filter to filter and smooth the I component of the HSI color space of the endoscopic image to obtain its low-frequency component.

[0027] Optionally, in step 4, the gained high-frequency component is superimposed on the I component of the original endoscopic image HSI color space.

[0028] Optionally, step 4 further includes: converting the enhanced endoscopic image from the HSI color space to the RGB color space.

[0029] The second object of the present invention is to provide an endoscopic image adaptive enhancement device, comprising a processor and a memory, wherein the memory stores instructions executed by the processor, and when the instructions are executed by the processor, the endoscopic image adaptive enhancement device implements any one of the above-mentioned endoscopic image adaptive enhancement methods.

[0030] The third object of the present invention is to provide a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they can implement any of the above-mentioned endoscopic image adaptive enhancement methods.

[0031] The beneficial effects of the present invention are:

[0032] The present invention filters the endoscopic image using a guided filter to obtain a low-frequency component with good edge and gradient preservation, and constructs an illumination gain function based on the low-frequency components of different regions of the endoscopic image to adaptively generate an illumination gain factor; and constructs a gradient gain function based on the Laplace operator convolution results of different regions of the endoscopic image to adaptively generate a gradient gain factor; finally, the above-mentioned illumination gain factor and gradient gain factor are combined to generate the adaptive gain coefficient required by the USM method to complete the improvement of the USM method. Experiments have shown that when the present invention is applied to small intestinal endoscopy image enhancement, it can effectively enhance the detail information of the small intestinal villi while suppressing dark area noise and preventing edge overshoot, effectively highlighting the edge detail information with tiny villus structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 This is a flowchart of the present invention's method for adaptively enhancing small intestine images through endoscopy based on unsharp masking.

[0035] Figure 2 1 and 2 are the results of processing the small intestine image in Example 2 of the present invention, where (a)-(d) are the original images, and (e)-(h) are the enhanced images. DETAILED DESCRIPTION

[0036] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0037] Example 1:

[0038] This embodiment provides an endoscopic image adaptive enhancement method, see Figure 1 , methods include:

[0039] Step 1: Obtain the original endoscopic image, filter and smooth the original endoscopic image using a guided filter to obtain its low-frequency component, and then subtract the filtered low-frequency component from the original endoscopic image to obtain the high-frequency component;

[0040] Step 2: Construct an illumination gain function based on the low-frequency components of different regions of the original endoscopic image to adaptively generate the illumination gain factor w1, which enhances the high-frequency components while suppressing the noise in the dark area;

[0041] Step 3: Based on the Laplacian operator of different regions of the endoscopic image, the gradient gain function is convolved to adaptively generate the gradient gain factor w2;

[0042] Step 4: Perform matrix multiplication on the illumination gain factor w1, the gradient gain factor w2 and the high-frequency component to obtain the gain high-frequency component, and then superimpose the gain high-frequency component on the original image to enhance the endoscopic image.

[0043] Example 2:

[0044] This embodiment provides an endoscopic image adaptive enhancement method for enhancing WCE small intestine images. Figure 1 , the method comprising:

[0045] Step 1: Obtain an image of the small intestine taken by a wireless capsule endoscope. First, convert the WCE small intestine image from the RGB color space to the HSI color space. Then, use a guided filter to filter and smooth the WCE small intestine image to obtain its low-frequency component. Then, subtract the filtered low-frequency component from the original image to obtain the high-frequency component.

[0046] Unsharp mask (USM) is a sharpening enhancement technology. The unsharp mask calculation is shown in formula (1):

[0047]

[0048] Among them, ZI(x, y) is the enhanced image; I(x, y) is the original image, k is the gain coefficient, is the image after being filtered by a low-pass filter.

[0049] It can be seen that in order to obtain more accurate high-frequency components, it is crucial to use a low-pass filter to obtain low-frequency components with good edge preservation. The guided filter includes an input image p, a guided image G, and an output image q. The guided image G and the output image q have a linear relationship as shown in formula (2):

[0050]

[0051] Among them, ω m It is a square window with the center of the window located at m; a m and b m are linear function coefficients, and the input image can be the image to be filtered.

[0052] Then a is obtained by minimizing the linear cost function m and b m , thus obtaining the output image q, the linear cost function is shown in formula (3).

[0053]

[0054] Among them, ε is to prevent a m Too large regularization factor.

[0055] Step 2: Based on the low-frequency components of different regions of the WCE image obtained in step 1, an illumination gain function is constructed to adaptively generate the illumination gain factor w1, which enhances the high-frequency components while suppressing the noise in the dark area;

[0056] The illumination gain factor is introduced into the gain coefficient in formula (1). The illumination gain factors of different regions of the WCE image are obtained through the illumination gain function. The illumination gain function is shown in formula (4):

[0057]

[0058] Among them, (x, y) is the image pixel coordinate, Mean is The mean of .

[0059] Since the illumination changes slowly and smoothly, it belongs to low-frequency information. Therefore, this embodiment uses the low-frequency information obtained by the guided filter to As the illumination information of the WCE image. When the brightness of the WCE image is less than the Mean, that is, in the dark area of ​​the WCE image, this embodiment uses a compressed sine function to obtain an illumination gain factor, which is then introduced into the gain coefficient to enhance the high-frequency information in the dark area while suppressing the noise in the dark area. When the brightness of the WCE image is greater than the Mean, this embodiment uses a sine function to obtain the illumination gain factor. The illumination gain factor value when the WCE image brightness is moderate is significantly greater than the illumination gain factor value in the bright area, and the illumination gain factor decreases as the brightness increases. The illumination gain factor obtained in this way can focus on enhancing the high-frequency information of the WCE image when the brightness is moderate, thereby enriching the detail information of the enhanced WCE image and effectively achieving the adaptability of the illumination gain factor.

[0060] Step 3: Based on the Laplacian operator of different regions of the WCE small intestine image, the gradient gain function is convolved to adaptively generate the gradient gain factor w2;

[0061] In this embodiment, the gradient gain factors of different regions are obtained according to the gradient gain function, and then the gradient gain factors are introduced into the gain coefficients in formula (1), thereby realizing adaptive enhancement of the WCE small intestine image. The gradient gain function w2 is shown in formula (5).

[0062]

[0063] Where (x, y) is the image pixel coordinate; It is the absolute value of the result after the original image is convolved with the Laplacian operator template and normalized, which represents the edge information of the image.

[0064] When the edge information value of the WCE small intestine image is less than 0.1, the gradient gain function grows rapidly, so that a larger gradient gain factor can be quickly obtained in the part with smaller edge information value of the WCE small intestine image. The gradient gain factor is then introduced into the gain coefficient to achieve the purpose of enriching the image detail information; when the edge information value of the WCE small intestine image is greater than 0.1, the slope of the function curve decreases slowly, so that the gradient gain factor decreases slowly, and the WCE small intestine image still requires a larger gradient gain factor at this time. When the edge information value is close to 1, it is a strong edge of the image, and the gradient gain factor is small at this time to prevent the occurrence of edge overshoot.

[0065] Step 4: Perform matrix multiplication on the illumination gain factor, gradient gain factor, and high-frequency component obtained above to obtain the gain high-frequency component, and then superimpose the gain high-frequency component on the original image to enhance the WCE small intestine image.

[0066] This embodiment combines the illumination gain factor and the gradient gain factor obtained from equations (4) and (5) to generate the gain coefficient in equation (1), thereby achieving adaptive enhancement of the WCE small intestine image. The adaptive gain coefficient is shown in equation (6).

[0067] k=α·w1·w2 (6)

[0068] Among them, α is the control maximum parameter, w1 is the illumination gain factor, and w2 is the gradient gain factor.

[0069] The obtained high-frequency components are matrix multiplied with the obtained high-frequency components to obtain the gain high-frequency components of different regions of the WCE small intestine image, and then superimposed on the I component of the HSI color space of the original input image of the WCE small intestine to obtain the enhancement component I, thereby achieving the purpose of enhancing the edge of the WCE small intestine image.

[0070] Based on the above specific implementation methods, the effects of the present invention are verified in combination with specific experiments below:

[0071] like Figure 2 The following is the original input image of the small intestine of WCE. Figure 2 (e)-(h) are the corresponding results after enhancement using the method of the present invention.

[0072] exist Figure 2 In (e)-(h), the hair in the black box is obviously more than that in the original image, and the present invention uses an adaptive gradient gain factor, so the obtained results highlight the hair without edge overshoot. Figure 2 The white boxes in (e)-(h) are the enhanced dark areas. Figure 2It can be found in the original images (a)-(d) that the dark areas have little information and are full of noise, but the method of the present invention suppresses the noise in the dark areas in the enhanced results by using the illumination gain factor. Figure 2 It can be found that the small intestinal villi of WCE enhanced by the method of the present invention are well highlighted, and doctors will be able to better observe the image and diagnose the disease.

[0073] This example was completed in MATLAB 2019a. The hardware environment was a PC with an 8-core Intel i5-8520U, 1.80GHz CPU and 8GB RAM. The experimental process was relatively stable.

[0074] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0075] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for adaptive enhancement of endoscopic images, characterized in that: The method comprises: Step 1: Obtain an original endoscopic image, filter and smooth the original endoscopic image using a guided filter to obtain its low-frequency component, and then subtract the filtered low-frequency component from the original endoscopic image to obtain a high-frequency component; Step 2: constructing an illumination gain function based on the low-frequency components of different regions of the original endoscopic image to adaptively generate an illumination gain factor w1, thereby enhancing the high-frequency components while suppressing the noise in the dark region; The illumination gain factor w1 is obtained by the illumination gain function, which is: Among them, (x, y) is the image pixel coordinate, is the image of the original endoscopic image after being filtered by a low-pass filter, and Mean is The mean of Step 3: Based on the Laplacian operators of different regions of the endoscopic image, convolution is performed to obtain a gradient gain function to adaptively generate a gradient gain factor w2; The calculation method of the gradient gain factor w2 is: in, It is the absolute value of the result after the original image is convolved with the Laplacian operator template and normalized, representing the edge information of the image; Step 4: Perform matrix multiplication on the illumination gain factor w1, the gradient gain factor w2 and the high-frequency component to obtain a gain high-frequency component, and then superimpose the gain high-frequency component on the original image to enhance the endoscopic image.

2. The method for adaptive enhancement of endoscopic images according to claim 1, characterized in that: The enhanced image obtained in step 4 is: k=α·w1·w2 Where ZI(x,y) is the enhanced image, I(x,y) is the original endoscopic image, and α is the control maximum parameter.

3. The method for adaptively enhancing endoscopic images according to claim 1, wherein: The guided filter includes an input image p, a guided image G, and an output image q. The linear relationship between the guided image G and the output image q is: Among them, ω m It is a square window with the center of the window located at m; a m and b m is the linear function coefficient, and the input image p is the image to be filtered; Then a is obtained by minimizing the linear cost function m and b m , thereby obtaining the output image q, the linear cost function is: Among them, ε is to prevent a m Too large regularization factor.

4. The method for adaptively enhancing endoscopic images according to claim 1, wherein: The step 1 first converts the endoscopic image from the RGB color space to the HSI color space, and then uses the guided filter to filter and smooth the I component of the HSI color space of the endoscopic image to obtain its low-frequency component.

5. The method for adaptively enhancing endoscopic images according to claim 4, characterized in that: In step 4, the gained high-frequency component is superimposed on the I component of the original endoscopic image HSI color space.

6. The method for adaptively enhancing endoscopic images according to claim 5, characterized in that: The step 4 further includes: converting the enhanced endoscopic image from the HSI color space to the RGB color space.

7. An endoscopic image adaptive enhancement device, comprising a processor and a memory, wherein the memory stores instructions executed by the processor. When the instructions are executed by the processor, the endoscopic image adaptive enhancement device implements the endoscopic image adaptive enhancement method according to any one of claims 1 to 6.

8. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method for adaptively enhancing endoscopic images according to any one of claims 1 to 6.

Images