An adaptive enhancement method for capsule endoscopy gastrointestinal faint light image

Abstract

The application discloses a kind of self-adaptive enhancement method of capsule endoscope gastrointestinal faint light image, belong to computer graphics technical field.The application is filtered and smoothed to WCE image using guide filter, to approximately estimate the illumination component of WCE image, and then decompose to obtain its reflection component.Then according to the positive correlation between minimum just noticeable difference threshold of illumination component and gain parameter of S-shaped function, obtain adaptive S-shaped function, to carry out adaptive enhancement to illumination component, and then carry out fusion with reflection component.Finally, again in combination with weighted distribution adaptive correction algorithm, the enhanced WCE image is carried out contrast enhancement.The application can automatically inhibit the over-enhancement of WCE faint light image bright area, and adaptively obtain the WCE image with better contrast, and the feature point extraction and matching number of WCE image after enhancement using the application are superior to classical enhancement algorithm, and average respectively increase by 67.1% and 57.3%.

Description

TECHNICAL FIELD

[0001] The application relates to a self-adaptive enhancement method for a capsule endoscopy gastrointestinal low-light image and belongs to the technical field of computer graphics. BACKGROUND

[0002] Wireless capsule endoscopy (WCE) images are usually taken under limited lighting conditions, and the particularity of the environment such as the convolution and peristalsis of the gastrointestinal tract leads to unclear WCE images, low contrast and even serious loss of details, which affects the diagnosis of doctors or the recognition of lesions by computers, and even brings interference to the automatic robot navigation of the capsule and causes serious missed detection.

[0003] In recent years, the enhancement method for low-light images, i.e. low-light images, has been widely studied. The histogram equalization method focuses on stretching the dynamic range of the entire image, and cannot overcome the more detailed color details of the image, and cannot strictly adjust the enhancement effect according to the local insufficient illumination in the low-light image. The Gamma correction method realizes the enhancement of the image by using the simplicity and efficiency of the power function, but the method needs to set the best parameters for the image or the local region of the image to achieve the best enhancement effect, which will lead to insufficient enhancement or even loss of information of the local details of the low-light image. The Retinex theory method uses the logarithmic transformation of the image and the illumination component generated by the Gaussian transformation to improve the image quality, but the method exists local detail loss while improving the image brightness. When the existing method is used to enhance the WCE image, the balance between the local detail enhancement and the brightness enhancement of the WCE image does not reach the ideal purpose, which is not conducive to the subsequent feature extraction and matching of the WCE image. SUMMARY

[0004] In order to solve the balance problem between the local detail enhancement and the brightness enhancement of the WCE image at present, the application provides a self-adaptive enhancement method for a capsule endoscopy gastrointestinal low-light image, and the technical scheme is as follows:

[0005] The first object of the application is to provide a self-adaptive enhancement method for a capsule endoscopy gastrointestinal low-light image, which comprises the following steps:

[0006] Step 1: applying a guided filter to filter and smooth the V component of the HSV color space of the WCE image, so as to approximately estimate the illumination component L of the V component;

[0007] Step 2: decomposing the V component based on the Retinex model to obtain the reflection component of the V component;

[0008] Step 3: obtaining an adaptive S-type function according to the positive correlation between the minimum just noticeable difference threshold of the illumination component L and the S-type function gain parameter, and then adaptively enhancing the illumination component L by using the adaptive S-type function;

[0009] Step 4: fusing the adaptively enhanced illumination component in Step 3 and the reflection component in Step 2 based on a Retinex model;

[0010] Step 5: performing contrast enhancement on the WCE image fused in Step 4 by using a weighted distribution adaptive Gamma correction algorithm.

[0011] Optionally, the step 1 comprises:

[0012] Step 1.1: converting the WCE image from an RGB color space to an HSV color space;

[0013] Step 1.2: in a guided filter, a guide image is the V component of the HSV color space, an output image is the illumination component L, and the guide image and the output image have a linear relationship as follows:

[0014]

[0015] wherein (x, y) is a pixel index, ω k is a square window with a side length of r, and the center of the window is k, a k and b k are linear function coefficients, and are solved by the following formula:

[0016]

[0017] wherein E(a k ,b k ) is a cost function of the window ω k , and ε is a regular factor for preventing a k from being too large;

[0018] a k and b k are substituted into the linear relationship expression to obtain the illumination component L.

[0019] Optionally, the relationship expression of the V component and the reflection component is as follows:

[0020] V(x, y) = R(x, y) L(x, y)

[0021] wherein V(x, y) is the V component of the HSV color space, R(x, y) represents the reflection component, and L(x, y) represents the illumination component;

[0022] The luminance component L in step 1 is brought into the above formula L(x, y) to obtain the reflection component R of the V component.

[0023] Optionally, the step 3 comprises:

[0024] Step 3.1: The minimum just noticeable difference threshold value of the luminance component L is calculated according to the luminance component L, and the calculation method is:

[0025]

[0026] Step 3.2: The luminance component L is enhanced by the S-shaped function with different gain parameters;

[0027]

[0028] Wherein, β is the gain parameter, L(x, y) is the luminance component L obtained in step 1, L E (x, y) is the luminance component after enhancement by the S function;

[0029] Step 3.3: By comparing the minimum just noticeable difference threshold value in step 3.1 with the S-shaped function output with different gain parameters in step 3.2, a positive correlation formula is obtained as follows:

[0030]

[0031] Wherein, β(x, y) is the gain parameter of the luminance component L at (x, y), and JND(x, y) is the minimum just noticeable difference threshold value of the luminance component L at (x, y);

[0032] The JND value calculated by step 3.1 is brought into the above positive correlation formula, so as to obtain the gain parameter β of different regions of the luminance component L, and the gain parameter β is brought into the S-shaped function to obtain the enhanced luminance component L E (x, y).

[0033] Optionally, the step 4 comprises:

[0034] Step 4.1: The enhanced luminance component in step 3 and the reflection component in step 2 are fused based on the Retinex model to obtain an enhanced V component;

[0035] Step 4.2: The WCE image is converted from the HSV color space to the RGB color space.

[0036] Optionally, the step 5 comprises:

[0037] Step 5.1: An adaptive parameter γ is calculated:

[0038] γ = 1 - cdf (l)

[0039] wherein, cdf(l) is the cumulative distribution function, and l is the image intensity;

[0040] Step 5.2: Through the parameter γ obtained in step 5.1, the WCE image obtained in step 4 is subjected to contrast enhancement by the Gamma correction function:

[0041]

[0042] wherein, l max is the maximum intensity of the image, and T(l) is the enhanced result of the WCE image after executing the function.

[0043] A second object of the present application is to provide a capsule endoscope gastrointestinal micro-light image recognition or classification method, which adopts the above-mentioned adaptive enhancement method of capsule endoscope gastrointestinal micro-light image to perform image enhancement, then performs feature extraction and matching on the enhanced image, and finally performs recognition or classification based on the extracted features.

[0044] A third object of the present application is to provide a capsule endoscope gastrointestinal micro-light image recognition or classification system, which comprises:

[0045] An image acquisition module, configured to acquire an image to be recognized or classified;

[0046] An image preprocessing module, configured to perform preprocessing on the acquired image, wherein the preprocessing process comprises adaptive enhancement of the image by adopting the above-mentioned adaptive enhancement method of capsule endoscope gastrointestinal micro-light image;

[0047] A feature extraction module, configured to perform feature extraction on the image after preprocessing;

[0048] An output display module, configured to output a recognition or classification result according to the feature extraction result.

[0049] The present application has the following beneficial effects:

[0050] The guide filter utilizes a linear relationship between a guide image and an output image, and an edge remains good. The application uses a guide filter to filter and smooth a WCE image to approximately estimate an illumination component of a WCE low-light image, so that the edge of the WCE low-light image is more accurately enhanced. Secondly, the application obtains an adaptive S-shaped function according to a positive correlation between a JND threshold of the illumination component and a gain parameter of the S-shaped function, to adaptively enhance the illumination component, so as to solve the problem of detail loss caused by over-enhancement of a bright area of the WCE low-light image and make up for the defect of insufficient detail enhancement of a dark area. However, after the S-shaped function enhancement, the dynamic range of the WCE image brightness is reduced, and the application further combines a weighted distribution adaptive Gamma correction algorithm to enhance the contrast of the enhanced WCE image, so that the overall contrast of the final enhancement result is better. After the WCE image is enhanced by the application, the feature point extraction and matching number are both better than those of a classical enhancement algorithm, and are averagely increased by 67.1% and 57.3%, respectively. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0052] Figure 1 is a flow chart of the adaptive enhancement method of the WCE low-light image of the capsule endoscope according to the present application.

[0053] Figure 2 is a WCE input image according to the second embodiment of the present application.

[0054] Figure 3 is an HSV color space V component image according to the second embodiment of the present application.

[0055] Figure 4 is an illumination component image according to the second embodiment of the present application.

[0056] Figure 5 is a reflection component image according to the second embodiment of the present application.

[0057] Figure 6 is an adaptive enhancement result image of the illumination component according to the second embodiment of the present application.

[0058] Figure 7 is an adaptive enhancement result image of the V component according to the second embodiment of the present application.

[0059] Figure 8 is a WCE image enhancement result RGB color space image according to the second embodiment of the present application.

[0060] Figure 9 is the WCE image enhancement result output graph of the embodiment two of the present application.

[0061] Figure 10 is the WCE image enhancement result of the embodiment two of the present application and the comparison graph of feature extraction and matching of other enhancement methods. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical scheme and advantages of the present application more clear, the following will make further detailed description to the embodiments of the present application in combination with the drawings.

[0063] Embodiment one:

[0064] The embodiment provides a self-adaptive enhancement method for capsule endoscopy gastrointestinal faint light image, as shown in the following steps. Figure 1

[0065] Step 1: using a guided filter to filter and smooth the V component of the WCE image in the HSV color space, so as to approximately estimate the illumination component L of the V component;

[0066] Step 2: decomposing the V component based on the Retinex model to obtain the reflection component of the V component;

[0067] Step 3: obtaining an adaptive S-shaped function according to the positive correlation between the minimum perceptible difference threshold of the illumination component L and the gain parameter of the S-shaped function, and then using the adaptive S-shaped function to perform adaptive enhancement on the illumination component L;

[0068] Step 4: fusing the adaptive enhanced illumination component in step 3 and the reflection component in step 2 based on the Retinex model;

[0069] Step 5: performing contrast enhancement on the WCE image obtained in step 4 by using a weighted distribution adaptive Gamma correction algorithm.

[0070] Embodiment two:

[0071] The embodiment provides a self-adaptive enhancement method for capsule endoscopy gastrointestinal faint light image, as shown in the following steps. Figure 1

[0072] Step 1: using a guided filter to filter and smooth the V component of the WCE image in the HSV color space, so as to approximately estimate the illumination component L of the V component;

[0073] Step 1.1: converting the WCE image from the RGB color space to the HSV color space, as shown in the following figure. Figure 2 Figure 3 ​​​V component of the HSV color space of the WCE image;

[0074] Step 1.2: In the guided filter, the guided image is the V component of the HSV color space, and the output image is the illumination component L, and there is a linear relationship between the guided image and the output image as follows:

[0075]

[0076] where (x, y) is the pixel index, ω k is a square window with a side length of r, and the center of the window is k, a k and b k are linear function coefficients, which are obtained by the following formula:

[0077]

[0078] where E(a k ,b k ) is the cost function of the window ω k , and ε is a regularization factor to prevent a k from being too large. Substitute a k and b k into the linear relationship expression to obtain the illumination component L, as shown in Figure 4 .

[0079] Step 2: Based on the Retinex model, decompose the V component to obtain its reflection component;

[0080] The Retinex model [Land E H. The retinex [J]. American Scientist, 1964, 52(2): 247-264] expression is as follows.

[0081] V(x, y) = R(x, y) · L(x, y)

[0082] where (x, y) is the pixel index, V(x, y) is the V component of the HSV color space, R(x, y) represents the reflection component, and L(x, y) is the illumination component obtained in step 1. Substitute the illumination component L in step 1 into L(x, y) in the above formula to obtain the reflection component R(x, y) of the V(x, y) component, as shown in Figure 5 , which is the obtained reflection component.

[0083] Step 3: According to the positive correlation between the just-noticeable difference (JND) threshold of the illumination component in step 1 and the gain parameter of the S-type function, obtain an adaptive S-type function to adaptively enhance the illumination component in step 1;

[0084] Step 3.1: Calculate the just-noticeable difference (JND) threshold of the luminance component according to the illumination component in step 1 [Jayant N. Signal compression: Technology targets and research directions [J]. IEEE Journal on Selected Areas in Communications, 1992, 10(5): 796-818]

[0085]

[0086] Step 3.2: Enhance the luminance component by S-shaped function with different gain parameters

[0087]

[0088] where (x, y) is the pixel index, β is the gain parameter, L(x, y) is the luminance component obtained in step 1, L E (x, y) is the luminance component enhanced by S function;

[0089] Step 3.3: Derive the positive correlation as follows by comparing the just-noticeable difference threshold in step 3.1 with the S-shaped function output with different gain parameters in step 3.2:

[0090]

[0091] where β(x, y) is the gain parameter of the luminance component L at (x, y), and JND(x, y) is the just-noticeable difference threshold of the luminance component L at (x, y).

[0092] The JND value calculated by step 3.1 is brought into the above positive correlation formula, so as to obtain the gain parameter β of the luminance component L in different regions, and β is brought into the above S-shaped function, so as to obtain the enhanced luminance component L E (x, y); as Figure 6 shown, is the luminance component enhanced by adaptive S-shaped function.

[0093] Step 4: Fuse the enhanced luminance component in step 3 with the reflection component in step 2 based on the Retinex model;

[0094] Step 4.1: Fuse the enhanced luminance component in step 3 with the reflection component in step 2 based on the Retinex model, to obtain the enhanced V component, as Figure 7 shown, is the enhanced V component;

[0095] Step 4.2: Convert the WCE image from HSV color space to RGB color space, as shown in Figure 8 Figure 4.2, which is the RGB color space result of the above enhanced WCE image;

[0096] Step 5: Perform contrast enhancement on the WCE image obtained in Step 4 by using a weighted distribution adaptive Gamma correction algorithm.

[0097] Step 5.1: Calculate the adaptive parameter γ.

[0098] γ = 1 - cdf(l)

[0099] where cdf(l) is the cumulative distribution function and l is the image intensity.

[0100] Step 5.2: Perform contrast enhancement on the WCE image obtained in Step 4 by using the Gamma correction function with the parameter γ obtained in Step 5.1:

[0101]

[0102] where l max is the maximum intensity of the image, and T(l) is the enhanced result of the WCE image after executing this function. As shown in Figure 9 Figure 4.3, which is the final WCE image enhancement result.

[0103] In order to further demonstrate the beneficial effects of the present application, the following comparative experiments were conducted.

[0104] The present application firstly adopts AFGTCR [Long M, Lan Z, Xie X, et al. Image Enhancement Method Based on Adaptive Fraction Gamma Transformation and Color Restoration for Wireless Capsule Endoscopy [C]. 2018 IEEE Biomedical Circuits and Systems Conference (BioCAS). IEEE, 2018: 1-4], AGCWD [Huang S C, Cheng F C, Chiu YS. Efficient contrast enhancement using adaptive gamma correction with weighting distribution [J]. IEEE Transactions on Image Processing, 2012, 22(3): 1032-1041], FU [Fu Q, Jung C, Xu K. Retinex-based perceptual contrast enhancement in images using luminance adaptation [J]. IEEE Access, 2018, 6: 61277-61286], LR3M [Ren X, Yang W, Cheng W H, et al. LR3M: Robust low-light enhancement via low-rank regularized retinex model [J]. IEEE Transactions on Image Processing, 2020, 29: 5862-5876] and the method of the present application to process 60 WCE low-light images, and then combine the classic SIFT [Lowe DG. Distinctive image features from scale-invariant keypoints [J]. International Journal of Computer Vision, 2004, 60(2): 91-110] feature extraction and FLANN [Muja M, Lowe DG. Fast approximate nearest neighbors with automatic algorithm configuration [J].Feature matching, and finally, the Random Sample Consensus (RANSAC) [Fischler M A, Bolles R C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography [J]. Communications of the ACM, 1981, 24(6): 381-395] is used to eliminate some false matching pairs and errors, and to obtain the accurate number of feature point matching. Figure 10 The results of feature point extraction and matching of the above prior art methods and the method of the present application are shown in (a)-(f). It can be found that after the enhancement of the present application, the feature point extraction and matching in the dark and bright areas of the WCE image are significantly increased. Table 1 shows the average number of feature point extraction and matching after the enhancement of 60 WCE micro-light images using different methods.

[0105] Table 1 Comparison of feature extraction and matching using different methods

[0106]

[0107] As can be seen from the data in Table 1, the feature extraction and matching effect after the enhancement of the method of the present application is the best. Compared with the AFGTCR, AGCWD, FU, and LR3M methods, the average number of feature extraction and matching of the present application is increased by 67.1% and 57.3%, respectively.

[0108] Some of the steps in the embodiments of the present application can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk, etc.

[0109] The above description is only the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for adaptive enhancement of capsule endoscopy gastrointestinal micro-light images, characterized in that, The method comprises: Step 1: filtering and smoothing the V component of the WCE image in the HSV color space by using a guided filter to approximately estimate the illumination component L of the V component; Step 2: decomposing the V component based on the Retinex model to obtain the reflection component of the V component; Step 3: obtaining an adaptive S-shaped function according to the positive correlation between the just noticeable difference threshold of the illumination component L and the gain parameter of the S-shaped function, and then adaptively enhancing the illumination component L by using the adaptive S-shaped function; Step 4: fusing the adaptively enhanced illumination component in Step 3 and the reflection component in Step 2 based on the Retinex model; Step 5: enhancing the contrast of the WCE image obtained in Step 4 by using a weighted distribution adaptive Gamma correction algorithm; The step 3 comprises: Step 3.1: calculating the just noticeable difference threshold of the illumination component L, and the calculation method is as follows: Step 3.2: enhancing the illumination component L by using the S-shaped function with different gain parameters; where β is a gain parameter, L(x, y) is the luminance component L obtained in step 1, and L E (x, y) is the luminance component enhanced by the S function. Step 3.3: obtaining the positive correlation formula by comparing the just noticeable difference threshold in Step 3.1 and the output of the S-shaped function with different gain parameters in Step 3.2 as follows: Wherein, β(x,y) is the gain parameter of the illumination component L at (x,y), and JND(x,y) is the just noticeable difference threshold of the illumination component L at (x,y); The JND value is calculated by step 3.1, and is brought into the above positive correlation formula, so as to obtain the gain parameter β of the illumination component L in different regions, and the gain parameter β is brought into the S-type function, so as to obtain the enhanced illumination component L E (x,y).

2. The method of claim 1, wherein the method is performed by a computer system. The step 1 comprises: Step 1.1: converting the WCE image from the RGB color space to the HSV color space; Step 1.2: in the guided filter, the guide image is the V component of the HSV color space, the output image is the illumination component L, and the guide image and the output image have a linear relationship as follows: where (x, y) is the pixel index, ω k is a square window with side length r, and the center of the window is located at k, a k and b k are linear function coefficients, which are obtained by the following equation: where E(a k ,b k ) is the cost function of the window ω k , and ε is a regularization factor to prevent a k from being too large. Substitute a k and b k into the linear relationship expression to obtain the luminance component L.

3. The method of claim 2, wherein the method is performed by a computer system. The relationship formula of the V component and the reflection component is: V(x,y)=R(x,y)·L(x,y) Wherein, V(x,y) is the V component of the HSV color space, R(x,y) represents the reflection component, and L(x,y) represents the illumination component; The reflection component R of the V component is obtained by substituting the illumination component L in Step 1 into the above formula L(x,y).

4. The method of claim 1, wherein the method is performed by a computer system. The step 4 comprises: Step 4.1: fusing the enhanced illumination component in Step 3 and the reflection component in Step 2 based on the Retinex model to obtain the enhanced V component; Step 4.2: converting the WCE image from the HSV color space to the RGB color space.

5. The method of claim 1, wherein the method is performed by a computer system. The step 5 comprises: Step 5.1: calculating the adaptive parameter γ: γ=1-cdf(l) Wherein, cdf(l) is a cumulative distribution function, and l is an image intensity; Step 5.2: performing contrast enhancement on the WCE image obtained in Step 4 by substituting the parameter γ obtained in Step 5.1 into the Gamma correction function: where, l max is the maximum intensity of the image, and T(l) is the WCE image enhancement result after performing this function.

6. A method of identifying or classifying a capsule endoscopy gastrointestinal micro-light image, the method comprising: The capsule endoscopy gastrointestinal micro-light image recognition or classification method first performs image enhancement by using the capsule endoscopy gastrointestinal micro-light image adaptive enhancement method of any one of claims 1-5, then performs feature extraction and matching on the enhanced image, and finally performs recognition or classification based on the extracted features.

7. A system for recognizing or classifying microscopic images of the gastrointestinal tract using capsule endoscopy, characterized in that, The system comprises: An image acquisition module is configured to acquire an image to be recognized or classified. An image preprocessing module is configured to preprocess the acquired image, and the preprocessing process comprises an adaptive enhancement method of the capsule endoscopy gastrointestinal micro-light image according to any one of claims 1-5. A feature extraction module is configured to extract features from the preprocessed image. An output display module is configured to output a recognition or classification result according to the feature extraction result.

Images