A Low-Light Image Enhancement Method for Wireless Capsule Endoscopes

Abstract

The present invention discloses a low - illumination image enhancement method for wireless capsule endoscopes, belonging to the field of computer vision. The method includes: extracting the illumination component of the V - component of the image by using a multi - scale fast guided filter function; calculating the brightness enhancement function parameter T from the illumination component by using the OTSU method; fitting the brightness enhancement function based on the parameter T and the Weber - Fechner theory; adjusting the saturation by using a wavelet function; realizing the overall enhancement of the image based on merging channels and converting to the RGB color space; The experimental results show that the method of the present invention has improvements in the quantitative evaluation indexes of image enhancement, namely, the mean, standard deviation, information entropy and image quality evaluation, compared with the existing methods; In addition, in terms of feature extraction and feature matching of the enhanced image, the verification effects are respectively improved by 59.3% and 32.9% on average in terms of quantity, and the depth estimation effect is also improved.

Description

Technical Field

[0001] The present invention relates to a low - illumination image enhancement method for wireless capsule endoscopes, belonging to the field of computer vision. Background Art

[0002] Since the commercialization of capsule endoscopes in 2006, the problem of low - illumination capsule endoscope low - light image enhancement has always been an active research topic. The common practice for low - illumination image enhancement is to design appropriate filtering techniques and grayscale processing functions to improve the brightness of the image. Among the existing methods for solving low - illumination image problems based on image enhancement, methods based on Histogram Equalization (HE), Gamma Correction (GC), and Retinex theory are several widely recognized methods. The core idea of HE is to enhance the image by adjusting the histogram distribution of the low - light image. However, if there are peaks in the histogram of the image, the brightness will be over - enhanced, and then the imbalance between saturation and brightness will cause the loss of image details. Although the GC image enhancement method has a low computational cost and is good at processing bright images. However, it is very difficult to manually set an appropriate gamma value to achieve an ideal effect, and when enhancing images in an overall dark scene, it often cannot enhance the overall brightness of the image well. The Retinex theory decomposes the image into reflection and illumination components through a Gaussian smoothing function and utilizes the characteristics of dynamic range compression and color constancy, which is beneficial to detail enhancement. However, it will over - increase the noise in the dark area, resulting in the over - amplification of the image noise, and at the same time, there will also be a problem of imbalance between saturation and brightness. In recent years, although a series of achievements have been made in the image enhancement method specifically for WCE, the common problem is that these methods cannot achieve a balance between brightness and saturation when enhancing WCE low - light images. In particular, the over - bright enhancement leading to the loss of local image details has become a difficult problem. Summary of the Invention

[0003] In order to achieve a good brightness enhancement effect on low - illumination images of wireless capsule endoscopes and at the same time balance the natural perception of brightness and saturation, the present invention provides a low - illumination image enhancement method for wireless capsule endoscopes, including:

[0004] Step 1: Obtain the image to be enhanced, and convert the image to be enhanced from the RGB color space to the HSV space;

[0005] Step 2: Use the multi - scale fast guided filter function to extract the illumination component of the V component of the image;

[0006] Step 3: Use the OTSU method to obtain the OTSU threshold from the illumination component image obtained in Step 2 as the value of the brightness enhancement function parameter T;

[0007] Step 4: Based on the luminance enhancement function parameter T and the Weber-Fechner theory, fit the luminance enhancement function through the image grayscale information and parameters;

[0008] Step 5: Perform operations on the S component of the image to achieve saturation adjustment of the image. Use the Harr wavelet function as the basis function, perform saturation adjustment on the low-frequency component image, and perform detail stretching on the high-frequency image;

[0009] Step 6: Merge the enhanced V component image, S component image, and the original H component image, and then convert to the RGB color space to obtain the final enhanced image.

[0010] Optionally, the image enhancement function in Step 4 is:

[0011]

[0012] where, I out (x, y) is the grayscale value of the output image at the point (x, y), and I v (x, y) is the grayscale value of the V image at the point (x, y), and I MSG (x, y) is the grayscale value of the output image of the multi-scale fast guided filter at the point (x, y), δ is the scale factor, and w1 and w2 are weights.

[0013] Optionally, the multi-scale fast guided filter in Step 2 is set as:

[0014]

[0015] where, I MSG represents the output image of the multi-scale fast guided filter, is the weight coefficient of the illumination component extracted by the multi-scale fast guided filter function, N is the number of scales used, FGIF() represents the fast guided filter operation, and I v represents the input initial V image, p is a guidance image, which is composed of the input image V; r and eps are two parameters that control the smoothness of the filtering result.

[0016] Optionally, the color space conversion formula in Step 1 is:

[0017]

[0018]

[0019] V = max(R, G, B)

[0020] Among them, max(R, G, B) and min(R, G, B) are respectively the maximum and minimum values among R, G, and B.

[0021] Optionally, the saturation adjustment formula in step 5 is:

[0022] I′ LF (x, y) = I LF (x, y) - τ

[0023] Among them, I LF (x, y) is the gray value of the low-frequency information image at (x, y), and τ is the brightness difference parameter, which is determined by the average gray value mean of the V component before and after enhancement:

[0024] τ = η(mean(V out ) - mean(V))

[0025] Among them, mean(V out ) represents the average gray value of the enhanced V component, mean(V) represents the average gray value before enhancement, and η is the proportionality coefficient.

[0026] Optionally, the scaling factor δ = 0.7, and the weights w1 = w2 = 0.5.

[0027] Optionally, the parameters r and eps for controlling the smoothness of the filtering result are set to 9 and 0.04.

[0028] Optionally, the proportionality coefficient η = 0.8.

[0029] The second object of the present invention is to provide a low-light image enhancement system for a wireless capsule endoscope, including:

[0030] An image acquisition module for acquiring a low-light image to be enhanced;

[0031] An image conversion module for converting the low-light image to be enhanced from the RGB color space to the HSV space;

[0032] A brightness enhancement module for enhancing the V component of the image to be enhanced by using the brightness enhancement function according to any one of claims 1-8;

[0033] A saturation adjustment module that uses the Harr wavelet function as the basis function to perform saturation adjustment on the low-frequency component image and detail stretching on the high-frequency image;

[0034] A channel merging module for merging the enhanced V component, S component, and the original H component;

[0035] An output display module for converting the HSV space image after channel merging into an RGB color space image and outputting it.

[0036] The third object of the present invention is to provide a computer-readable storage medium storing computer-executable instructions, which when executed by a processor implement the low-light image enhancement method for wireless capsule endoscopes described in any one of the above.

[0037] The beneficial effects of the present invention are:

[0038] The present invention designs an image enhancement function based on the Weber-Fechner theory. First, the illumination component of the image is obtained by using multi-scale fast guided filtering. Then, the OTSU method is used to set the function parameters according to the gray information of the illumination component. Finally, an image enhancement function is fitted based on the Weber-Fechner theory through the image gray information and parameters, which can naturally amplify the image brightness.

[0039] The present invention proposes a method for balancing brightness and saturation. This method is based on wavelet transform, and a brightness difference coefficient is set according to the average gray change of the V component of the image before and after enhancement, realizing the saturation adjustment and detail enhancement process based on brightness enhancement, thereby obtaining the balance of details in the bright area and dark area of the WCE image.

[0040] In an embodiment of the present invention, first, based on the HSV color space, multi-scale fast guided filtering is used to estimate the illumination component. Then, a brightness enhancement function based on the Weber-Fechner law is designed to enhance the brightness of the V component image. Secondly, the OTSU method is used to determine the function parameters according to the image gray level. The saturation of the image is adjusted on the S component image by using wavelet transform combined with the brightness difference coefficient before and after enhancement to realize the enhancement of details. Finally, the overall enhancement of the image is realized based on merging channels and converting to the RGB color space.

[0041] Experimental results show that the method of the present invention has improvements in the quantization evaluation indexes of image enhancement, namely Mean, Standard Deviation (Std), Information Entropy (IE), and Natural Image Quality Evaluator (NIQE), compared with existing traditional methods. In addition, in terms of feature extraction and feature matching of the enhanced image, the verification effects are respectively improved by an average of 59.3% and 32.9% in terms of quantity, and the depth estimation effect is also improved. Description of the Drawings

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0043] Figure 1 It is a flow structure diagram of a low-light image enhancement method for a wireless capsule endoscope according to the present invention.

[0044] Figure 2 (a) and (b) are schematic diagrams before and after brightness enhancement.

[0045] Figure 3 (a) and (b) are schematic diagrams before and after saturation adjustment.

[0046] Figure 4 (a) and (b) are schematic diagrams before and after overall image enhancement.

[0047] Figure 5 It is the experimental result of the method of the present invention on the endoscopic image in terms of Mean, Std, IE, and NIQE metrics.

[0048] Figure 6 It is an example diagram of the experimental result of the embodiment of the present invention on the endoscopic image set, where (a) and (b) are bar charts of the final image feature point extraction and matching quantity of the embodiment of the present invention.

[0049] Figure 7 It is the experimental result diagram of the embodiment of the present invention on the endoscopic image set for comparison with different methods, where (a) Mean, (b) Std, (c) IE, (d) NIQE. Detailed implementation manners

[0050] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will further describe the embodiments of the present invention in detail in conjunction with the accompanying drawings.

[0051] Embodiment 1:

[0052] This embodiment provides a low-light image enhancement method for a wireless capsule endoscope, including:

[0053] Step 1: Study the principle of the image color space, convert the image color space, convert the RGB color space to the HSV space, extract the S and V components for image enhancement, and provide support for realizing low-light image enhancement;

[0054] Step 2: Extract the illumination component from the V component of the image. Since a single-scale image filter cannot obtain a suitable brightness estimate, multi-scale fast guided filtering can achieve better results in maintaining edges and estimating illumination components.

[0055] Step 3: Use the OTSU method to obtain the parameter T of the brightness enhancement function by utilizing the gray feature information of the illumination component image.

[0056] Step 4: Set the function parameters using Step 3, and then fit the image enhancement function based on the Weber-Fechner theory through the image gray information and the parameters, which can naturally amplify the image brightness.

[0057] Step 5: Operate on the S component to adjust the saturation of the image. Use the Harr wavelet function as the basic function to adjust the saturation on the low-frequency component image and stretch the details on the high-frequency image.

[0058] Step 6: Merge the processed V component image and S component image in channels, and then convert to the RGB color space to obtain the final enhanced image.

[0059] Embodiment 2:

[0060] This embodiment provides a low-light image enhancement method for wireless capsule endoscopes. Refer to Figure 1 , including the following steps:

[0061] Step 1: First, convert the color space of the image from RGB to HSV and perform channel separation of the color space. The purpose is to more accurately describe the perceived color relationship than RGB and the calculation is simple. In the HSV color space, these three color components are independent of each other, and enhancing any one of the components will not affect the other two components. Therefore, compared with the result obtained by directly enhancing the low-light image in the RGB color space, the image enhancement result after being processed in the HSV color space has the characteristics of small color distortion and more vivid colors. The conversion formula from HSV to RGB is shown in Equation (1);

[0062]

[0063]

[0064] V = max(R, G, B)

[0065] Step 2: Use multi-scale fast guided filtering for illumination component estimation. Specifically, let I G = FGIF(I v , p, r, eps), where FGIF represents the fast guided filter operation, Iv represents the initial V image of the input. p is a guiding image, which is composed of the input image V; r and eps are two parameters that control the smoothness of the filtering result. In this embodiment, the two parameters r and eps are default set to 9 and 0.04, and I G is the filtered output image.

[0066] To balance the global and local properties of the estimated illumination component, a multi-scale guiding function is adopted. The guiding functions of multiple scales are used to extract the illumination component of the scene, and weights are assigned to the functions. Finally, the illumination component of the image is obtained, as shown in Equation (2).

[0067]

[0068] Among them, I MSG represents the output image of the multi-scale fast guided filter, is the weight coefficient of the illumination component extracted by the multi-scale fast guided filter function, and N is the number of scales used.

[0069] Step 3: And according to the illumination component in Step 2, the parameter T is estimated using the OTSU method. The OTSU threshold is taken from the illumination component image as the value of the parameter T of the brightness enhancement function, as shown in Equation (3);

[0070] T = OTSU(I MSG ) (3)

[0071] Among them, I MSG represents the output of the multi-scale fast guided filter, T is the output parameter, and OTSU is the maximum inter-class variance method.

[0072] Step 4: Using the parameter in Step 3 and the gray information of the V image itself, a brightness enhancement function is constructed in combination with the Weber-Fechner law as shown in Equation (4);

[0073]

[0074] Among them, I out (x, y) is the gray value of the output image at the point (x, y), I v (x, y) is the gray value of the V image at a certain point, I MSG (x, y) is I MSG The gray value of the image at the point (x, y). T is the function parameter, which is determined by the image I MSG using the OTSU method. δ is the scale factor, and w1 and w2 are weights, which are set to δ = 0.7, w1 = w2 = 0.5 in this embodiment.

[0075] Step 5: Operate on the S component to adjust the saturation of the image. Use the Harr wavelet function as the basic function to adjust the saturation on the low-frequency component image and perform detail stretching on the high-frequency image.

[0076] Step 6: Merge the processed V component image and S component image in channels, and then convert to the RGB color space to obtain the final enhanced image.

[0077] Based on the above specific implementation manners, combined with Figure 2 the image set shown below for testing experiments to verify the effect of the present invention:

[0078] The present invention first analyzes the principle of the color space, converts the color space of the image, converts the RGB color space to the HSV space, extracts the S and V components for image enhancement, and provides support for realizing low-illumination image enhancement. Extract the illumination component of the V component of the image. The multi-scale fast guided filter can estimate the edge and illumination component while keeping them, and use the OTSU method to estimate the parameter T. A brightness enhancement function is constructed in combination with the Weber-Fechner law for image enhancement, as shown in Figure 2 (b). Then operate on the S component to adjust the saturation of the image. Use the Harr wavelet function as the basic function to adjust the saturation on the low-frequency component image and perform detail stretching on the high-frequency image. The example effect is as shown in Figure 3 (b). Finally, merge the processed V component image and S component image in channels, and then convert to the RGB color space to obtain the final enhanced image as shown in Figure 4 (b).

[0079] The method of the present invention is run and tested on a standard data set, and comparative experiments are carried out with different methods. In the experiment, the deep learning algorithm Zero-DCE and the manual algorithms Wang et al., EndoIMLE, Li et al. and the method of the present invention are used to process 300 WCE low-light images. The results are reproduced by using the publicly available source code and the recommended parameters. In the experiment, the numbers 1 to 100 are WCE stomach images, the numbers 101 to 200 are small intestine images, and the last 100 are large intestine images.

[0080] The present invention first conducts comparative experiment verification with different methods in terms of the number of feature point extraction and detection, as shown in Figure 6As shown, it can be clearly seen that after using the present invention for image enhancement, the number of SIFT feature extractions and detections has increased significantly. Then, 300 test images of each method were evaluated, and experiments were conducted on the evaluation metrics Mean, Std, IE, and NIQE. For all four metrics, for Mean, Std, and IE, larger result values indicate higher image quality, and for NIQE, smaller result values indicate higher image quality.

[0081] As Figure 7 shown, all methods are higher than the Original metric in the Mean metric. The method of the present invention is slightly lower than Li et al. and higher than other methods. The present invention reaches the highest in the Std metric, which indicates that the present invention is effective to a certain extent in enhancing image brightness and quality. In the IE metric, the methods of Zero-DCE, Wang et al., and Li et al. are lower than Original, and the method of the present invention reaches the highest IE value. It is worth noting that the two methods of Wang et al. and Li et al. have lower metrics than Original because although these two methods have higher values in the Mean metric, oversaturation causes loss of their details. For Zero-DCE, it can be observed that the overall tone of the enhanced image is grayish, and the low contrast results in a lower IE value. In the NIQE metric, the results produced by the method proposed in the present invention are lower than Zero-DCE and better than other comparative methods, which indicates that the method proposed in the present invention is very effective in enhancing image quality to a certain extent. All in all, although the method of the present invention is slightly lower than Li et al. in Mean and slightly higher than Zero-DCE in NIQE, it is higher than them in other metrics. Overall, the method of the present invention ranks first in the Std and IE evaluation metrics and second in the Mean and NIQE evaluation metrics. In quantitative evaluation, it is difficult to determine that a method has achieved the best performance with one evaluation metric, but overall, the method proposed in the present invention has achieved good performance for each metric, which indicates that the method of the present invention has achieved high-quality image enhancement.

[0082] This embodiment is completed by using CLion and OpenCV 3.4.1 under the Ubuntu 18.04 operating system installed in VMware Workstation 16 Pro. The hardware environment is a laptop with an AMD Ryzen 5 5600H with Radeon Graphics processor (3.30 GHz) and 16 GB of running memory, and the experimental process is relatively stable.

[0083] Some steps in the embodiments of the present invention can be implemented by software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk, etc.

[0084] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A low-light image enhancement method for wireless capsule endoscopes, characterized in that, The method includes: Step 1: Obtain the image to be enhanced, and convert the image to be enhanced from the RGB color space to the HSV space; Step 2: Use the multi-scale fast guided filter function to extract the illumination component of the V component of the image; The multi-scale fast guided filter is set as: Among them, represents the output image of the multi-scale fast guided filter, is the weight coefficient of the illumination component extracted by the multi-scale fast guided filter function. N is the number of scales used, and FGIF() represents the fast guided filter operation, represents the input initial V image, p is a guidance image, which is composed of the input image V; r and eps are two parameters that control the smoothness of the filtering result; Step 3: Use the OTSU method to obtain the OTSU threshold from the illumination component image obtained in Step 2 as the value of the parameter T of the brightness enhancement function; Step 4: Based on the parameter T of the brightness enhancement function and the Weber-Fechner theory, fit the brightness enhancement function through the image gray information and the parameter; Step 5: Use the Harr wavelet function as the basic function to divide the S component of the image into a low-frequency component and a high-frequency component, then perform saturation adjustment on the low-frequency component image and detail stretching on the high-frequency image; The saturation adjustment formula is: Among them, is the gray value of the low-frequency information image at , is the brightness difference parameter, which is determined by the average gray value mean before and after the enhancement of the V component: Among them, represents the average gray value of the enhanced V component, represents the average gray value before enhancement, is the proportionality coefficient, set ; Step 6: Merge the enhanced V component image, S component image and the original H component image by channels, and then convert to the RGB color space to obtain the final enhanced image.

2. The low-light image enhancement method for wireless capsule endoscopes according to claim 1, wherein The image enhancement function in Step 4 is: where, is the gray value of the output image at the point . is the gray value of the V image at the point . is the gray value of the output image of the multi-scale fast guided filter at the point . is the scale factor, and are weights.

3. The low-light image enhancement method for a wireless capsule endoscope according to claim 1, characterized in that The color space conversion formula in Step 1 is: Among them, and are the maximum and minimum values in R, G, and B, respectively.

4. The low-light image enhancement method for a wireless capsule endoscope according to claim 2, characterized in that The said scale factor , weight .

5. The low-light image enhancement method for wireless capsule endoscopes according to claim 1, characterized in that The parameter for controlling the smoothness of the filtering result r and eps are set to 9 and 0.

04.

6. A low-light image enhancement system for wireless capsule endoscopes, characterized in that, The system includes: An image acquisition module for acquiring a low-light image to be enhanced; An image conversion module for converting the low-light image to be enhanced from the RGB color space to the HSV space; A brightness enhancement module for enhancing the V component of the image to be enhanced by using the brightness enhancement function according to any one of claims 1-5; A saturation adjustment module that uses the Harr wavelet function as the basic function to perform saturation adjustment on the low-frequency component image and detail stretching on the high-frequency image; A channel merging module for merging the enhanced V component, S component and the original H component by channels; An output display module for converting the HSV space image after channel merging into an RGB color space image and outputting it.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the low-light image enhancement method for a wireless capsule endoscope according to any one of claims 1 to 5 is implemented.

Images