Image Retinex enhancement method, system and computer-readable medium based on convolution gradient optimization

Abstract

This invention discloses a Retinex image enhancement method based on convolution gradient optimization. First, a reflective layer convolution model and a Gaussian field objective function are constructed. Using gradient descent techniques, Retinex decomposition optimization is performed. The resulting reflective layer convolution model is then corrected. Finally, the reflective layer is recovered from the reflective layer convolution model, preserving details in both dark and bright areas. This method transforms Retinex decomposition into an unconstrained optimization problem. Using only simple optimization techniques, high-quality low-light image enhancement can be achieved, reducing the difficulty of engineering the method.

Description

Technical Field

[0001] The present invention belongs to the technical field of image enhancement processing, and in particular relates to an image Retinex enhancement method based on convolution gradient optimization. Background Art

[0002] High-definition, high-visibility images are the eternal goals of imaging technology. However, due to the limitations of specific application scenarios and sensor properties, the captured images often have shortcomings such as low brightness, narrow dynamic range, and low contrast. Therefore, high-quality image grayscale enhancement methods are needed to improve image quality.

[0003] Retinex decomposition provides the basis for illumination estimation and reflectance recovery. A Gaussian filter is used to obtain the illumination layer, and SSR regards the reflectance as the desired enhancement result. MSR can be regarded as a weighted combination of multiple SSRs at different scales. SSR and MSR are widely used to enhance grayscale images. However, since insufficient attention has been paid to the imaging model under low-light conditions, low-light images cannot be effectively enhanced. The simultaneous reflection and brightness layer estimation method (SRIE) provides an algorithm architecture that simultaneously optimizes the brightness and reflection layers of the image, achieving effective low-light image enhancement (X.Fu, D.Zeng, Y.Huang, X.-P.Zhang, and X.Ding, “A weighted variational model for simultaneous reflectanceand illumination estimation,” in Proc.IEEE Conf.Comput.Vis.Pattern Recognit.(CVPR),Jun.2016,pp.2782-2790.). Based on SRIE, Guo et al. proposed a low-light image enhancement method based on illumination map estimation (LIME), introducing the maximum RGB prior to improve the performance of brightness layer estimation (X.-J. Guo, Y. Li, and H.-B. Ling, “LIME: Low-light image enhancement via illumination map estimation,” IEEE Trans. Image Process., vol. 26, no. 2, pp. 982-993, Feb. 2017). LR3M can suppress noise in low-light image enhancement results through low-rank prior (X.-T. Ren, W.-H. Yang, W.-H. Cheng, and J.-Y. Liu, “LR3M: robust low-light enhancement via low-rank regularized Retinex model,” IEEE Trans. Image Process., vol. 29, pp. 5862-5876, Apr. 2020.).

[0004] Retinex decomposition-based image enhancement algorithms, restoring reflectance by simultaneously estimating both reflectance and illumination, is a popular and effective method. However, solving for both unknowns simultaneously is an ill-posed problem. Converting Retinex decomposition into a multi-block optimization problem requires the use of alternating direction techniques (such as the alternating direction multiplier method) to solve the optimization problem. This results in increasingly complex enhancement algorithm models and structures, significantly increasing the difficulty of implementation. Many researchers have recognized this problem, and many SRIE-based enhancement methods have developed their own acceleration strategies. However, the complexity of multi-block convex optimization makes the practical application of low-light image enhancement extremely difficult. Summary of the Invention

[0005] In view of the above shortcomings, the present invention provides an image Retinex enhancement method based on convolution gradient optimization.

[0006] According to a first aspect of the present invention, an image Retinex enhancement method based on convolution gradient optimization is proposed, the method comprising the following steps:

[0007] Step 1: Logarithmic transformation of the input image;

[0008] Step 2: Perform a rough fuzzy filter on the logarithmic transformed image;

[0009] Step 3: Based on the logarithmic transformation image and the rough blurred image, the gradient descent method is used to perform Retinex decomposition optimization to estimate the optimal reflection layer convolution model;

[0010] Step 4: calibrate the optimal reflection layer model;

[0011] Step 5: Restore the reflection layer from the corrected reflection model to obtain the final enhanced image.

[0012] The formula for logarithmic transformation of the image in step 1 is as follows:

[0013]

[0014] Where L is the input image, is the image after logarithmic transformation.

[0015] In step 2, the rough fuzzy filtering includes the following steps:

[0016] 1) Yes Apply Gaussian blur;

[0017] 2) After Gaussian blur In the example, the values ​​of pixels whose horizontal and vertical coordinates are odd (or even) are set to 0, and the generated image is represented by express.

[0018] In step 3, the reflection layer convolution model Γ is in the following form:

[0019]

[0020] Where X and Y represent the coordinate matrix formed by adding 1 to the normalized horizontal coordinate and 1 to the normalized vertical coordinate of each pixel in the input image, which is consistent with the dimension of the input image. d is the dark channel convolution kernel, ω b is the bright channel convolution kernel, both are n ω dimensional square matrix, n ω Only odd numbers can be used. represents the convolution operation, It represents the Hadamard product, that is, the corresponding elements in the same position are multiplied.

[0021] In step 3, the Retinex decomposition optimization uses the following objective function:

[0022]

[0023] in express The value of the i-th pixel in , Γ i represents the value of the i-th pixel in Γ, ||·||2 represents the L2 norm, [ω d ,ω b ] indicates that ω d and ω b Spliced ​​n ω ×2n ω dimensional convolution kernel matrix, λ is the stability adjustment coefficient, and σ is the scale coefficient.

[0024] In step 3, in the Retinex decomposition optimization using the gradient descent method, the first-order derivative of the objective function is as follows:

[0025]

[0026] where Ω i represents n centered at the i-th pixel ω ×n ω neighborhood, so Represents Ω in the X matrix i The submatrix in the neighborhood. Similarly, Indicates Ω in Y matrix i The submatrix in the neighborhood, express Ω in the matrix i Submatrix within the neighborhood. Represents the n formed by splicing these sub-matrices ω×2n ω dimensional matrix.

[0027] In step 4, the reflection layer model is corrected as follows:

[0028]

[0029] where Γ c Represents the reflection layer model matrix after correction, 1 represents a matrix with the same dimension as the input image and all elements are 1.

[0030] In step five, the reflection layer is restored from the corrected reflection model to obtain the final enhanced image using the following formula:

[0031]

[0032] Where R represents the final enhanced image. π is the matrix r=(1-exp{Γ c})-The marking matrix of the elements less than 0 in L, if the element value in r is less than 0, then the corresponding element value in π is 1, otherwise it is 0.

[0033] Compared with the existing technology, the image Retinex detail enhancement method proposed in this invention has the following significant advantages:

[0034] The method of the present invention uses a Gaussian field-based objective function and rough fuzzy filtering to transform Retinex decomposition into an unconstrained optimization problem. Only simple gradient optimization techniques (such as the quasi-Newton method) are needed to recover the reflection layer from the input image, which greatly reduces the difficulty of algorithm implementation.

[0035] At the same time, convolution operations are used to construct a reflection layer model to improve the nonlinearity of the model, so that the reflection layer model can simultaneously retain the rich detail textures of both bright and dark areas in the image, effectively reducing the loss of details in dark areas and overexposure of details in bright areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Flowchart of an image Retinex enhancement method based on convolution gradient optimization according to an exemplary embodiment of the present invention.

[0037] Figure 2a 、 2b 2c are schematic diagrams comparing the enhancement results of three groups of images using the Retinex enhancement method of the present invention. DETAILED DESCRIPTION

[0038] In order to better understand the technical content of the present invention, specific embodiments are given and described below with reference to the accompanying drawings.

[0039] Combine Figure 1The image Retinex enhancement method based on convolution gradient optimization of the example shown includes: performing logarithmic transformation on the input image; performing rough blur filtering on the logarithmic transformed image; performing Retinex decomposition optimization processing based on the logarithmic transformed image and the rough blur image using the gradient descent method to estimate the optimal reflection layer convolution model; correcting the optimal reflection layer model; and finally, restoring the reflection layer from the corrected reflection model to obtain the final enhanced image.

[0040] The following will describe the implementation process of the aforementioned embodiment of the present invention in more detail with reference to the processes shown in the accompanying drawings.

[0041] Step 1: Logarithmic transformation of the input image

[0042] According to a specific embodiment of the present invention, the grayscale linear stretching of the visible light image is first performed to scale the grayscale value range to [0, 1]. Then, the natural logarithm corresponding to each pixel value is calculated to form a visible light logarithmic image. The formula is as follows:

[0043]

[0044] Where L is the input image, is the image after logarithmic transformation.

[0045] Step 2: Perform rough fuzzy filtering on the logarithmic transformed image

[0046] According to a specific embodiment of the present invention, the rough fuzzy filtering includes the following steps:

[0047] Step 201: Apply Gaussian blur;

[0048] Step 202: Gaussian blur In the example, the values ​​of pixels whose horizontal and vertical coordinates are odd (or even) are set to 0, and the generated image is represented by express.

[0049] Step 3: Based on the logarithmic transformation image and the rough blurred image, the gradient descent method is used to perform Retinex decomposition optimization to estimate the convolution model of the reflection layer.

[0050] In this embodiment, the specific method includes the following steps:

[0051] Step 301: Define the reflection layer convolution model Γ as follows:

[0052]

[0053] Where X and Y represent the coordinate matrix formed by adding 1 to the normalized horizontal coordinate and 1 to the normalized vertical coordinate of each pixel in the input image, which is consistent with the dimension of the input image. d is the dark channel convolution kernel, ω b is the bright channel convolution kernel, both are n ω dimensional square matrix, n ω Only odd numbers can be used. In this embodiment, it is preferred to set n ω =3. represents the convolution operation, Represents the Hadamard product operation, that is, corresponding elements in the same position are multiplied;

[0054] Step 302: Define the objective function used in Retinex decomposition optimization as follows:

[0055]

[0056] in express The value of the i-th pixel in Γ i represents the value of the i-th pixel in Γ, ||·||2 represents the L2 norm, [ω d ,ω b ] indicates that ω d and ω b Spliced ​​n ω ×2n ω dimensional convolution kernel matrix;

[0057] Where σ is the scale coefficient; λ is the stability adjustment coefficient;

[0058] Step 303: Determine the first-order derivative of the objective function defined in step 302 as follows:

[0059]

[0060] where Ω i represents n centered at the i-th pixel ω ×n ω neighborhood, so Represents Ω in the X matrix i The submatrix in the neighborhood. Similarly, Indicates Ω in Y matrix i The submatrix in the neighborhood, express Ω in the matrix i Submatrix within the neighborhood. Represents the n formed by splicing these sub-matrices ω ×2n ω dimensional matrix;

[0061] Step 304: Based on formula (3) and formula (4), the objective function is optimized using the quasi-Newton method to obtain the optimal dark channel convolution kernel. and bright channel convolution kernel

[0062] Step 4: Correct the reflection layer model.

[0063] According to a specific embodiment of the present invention, the optimal dark channel convolution kernel obtained in step 304 is and bright channel convolution kernel Substitute the following correction formula:

[0064]

[0065] where Γ c Represents the reflection layer model matrix after correction, 1 represents a matrix with the same dimension as the input image and all elements are 1.

[0066] Step 5: Recover the reflection layer from the corrected reflection model to obtain the final enhanced image

[0067] According to a specific embodiment of the present invention, the following formula is used to restore the reflective layer:

[0068]

[0069] Where R represents the final enhanced image. π is the matrix r=(1-exp{Γ c})-The marking matrix of the elements less than 0 in L, if the element value in r is less than 0, then the corresponding element value in π is 1, otherwise it is 0.

[0070] In an optional embodiment, it is preferred to set λ=0.02, and correspondingly set σ=0.4.

[0071] In another embodiment, the present invention may also be configured to be implemented in the form of an image retinex enhancement system based on convolution gradient optimization, which includes:

[0072] one or more processors;

[0073] A memory stores operable instructions, wherein when the instructions are executed by the one or more processors, the image Retinex enhancement method based on convolution gradient optimization of any of the aforementioned embodiments is implemented.

[0074] In another embodiment, the present invention can also be configured to be implemented in the form of a computer-readable medium storing a computer program, wherein the computer program includes instructions that can be executed by one or more computers, and when the instructions are executed by the one or more computers, the image Retinex enhancement method based on convolution gradient optimization of any of the aforementioned embodiments is implemented.

[0075] Combined with the detailed optimization and enhancement design of the above embodiments and the corresponding parameter settings, Figure 2a 、 2b Figures 2 and 2c demonstrate the grayscale enhancement effect of the present invention. Each figure includes three sets of images: the input image, the result of the reflection layer convolution model, and the final enhanced image. As can be seen from the figures, the detail enhancement of the present invention not only significantly improves the brightness of dark areas, but also maintains and enhances the details of bright areas.

[0076] While the present invention has been disclosed above with reference to preferred embodiments, this is not intended to limit the present invention. Persons skilled in the art will readily appreciate that various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. An image Retinex enhancement method based on convolution gradient optimization, characterized in that: The following steps are involved: Step 1: Logarithmic transformation of the input image; Step 2: Logarithmically transformed image Perform rough blur filtering; Step 3: Based on the logarithmic transformation image and the rough blurred image, the gradient descent method is used to perform Retinex decomposition optimization to estimate the optimal reflection layer convolution model. ; Step 4: Optimal reflection layer model Make corrections; as well as Step 5: Recover the reflection layer from the corrected reflection model to obtain the final enhanced image; Among them, the gradient descent method is used to perform Retinex decomposition optimization, and the objective function used is: ； in, express The value of the i-th pixel in , express The value of the i-th pixel in , express L 2-norm, Represents the dark channel convolution kernel and bright channel convolution Nuclear splicing dimensional convolution kernel matrix, is the stability adjustment coefficient, is the scale coefficient.

2. The image Retinex enhancement method based on convolution gradient optimization according to claim 1, characterized in that: The input image is logarithmically transformed, and the transformation formula used is as follows: ； in, is the input image, is the image after logarithmic transformation.

3. The image Retinex enhancement method based on convolution gradient optimization according to claim 1, characterized in that: The method of performing rough fuzzy filtering on the logarithmically transformed image comprises the following steps: 1) For the logarithmic transformed image Apply Gaussian blur; 2) After Gaussian blur In the example, the values ​​of pixels whose horizontal and vertical coordinates are both odd or even are set to 0, and the generated image is represented by .

4. The image Retinex enhancement method based on convolution gradient optimization according to claim 3, characterized in that: The reflection layer convolution model The expression is as follows: ； in, and Represents the coordinate matrix formed by adding 1 to the normalized horizontal coordinate and 1 to the normalized vertical coordinate of each pixel in the input image, and is consistent with the input image dimension; Dark channel convolution kernel and bright channel convolution Both dimensional square matrix, Only odd numbers can be taken; represents the convolution operation, Represents the Hadamard product operation, that is, corresponding elements in the same position are multiplied.

5. The image Retinex enhancement method based on convolution gradient optimization according to claim 4, characterized in that: In the process of Retinex decomposition optimization using the gradient descent method, the first-order derivative of the objective function is as follows: ； in Represents the center of the i-th pixel neighborhood, so Representation matrix middle submatrices within the neighborhood; Representation matrix middle The submatrix in the neighborhood, express Image corresponding matrix submatrices within the neighborhood; Represents the concatenation of submatrices dimensional matrix.

6. The image Retinex enhancement method based on convolution gradient optimization according to claim 5, characterized in that: The correction formula used for correcting the optimal reflection layer model is as follows: ; in, represents the reflection layer model matrix after correction, Represents a matrix with the same dimensions as the input image and all elements are 1.

7. The image Retinex enhancement method based on convolution gradient optimization according to claim 6, characterized in that: The reflection layer is restored from the corrected reflection model to obtain the final enhanced image, wherein the formula used in the restoration process is as follows: ； in, represents the final enhanced image, is a matrix A marker matrix for elements whose values ​​are less than 0. ; In the matrix If the element value is less than 0, then it is in the marking matrix The corresponding element value is 1, otherwise it is 0.

8. An image Retinex enhancement system based on convolution gradient optimization, characterized in that: include: one or more processors; A memory storing operable instructions, wherein the instructions, when executed by the one or more processors, implement the image Retinex enhancement method based on convolution gradient optimization according to any one of claims 1 to 7.

9. A computer-readable medium storing a computer program, characterized in that: The computer program includes instructions that can be executed by one or more computers, and when the instructions are executed by the one or more computers, the image Retinex enhancement method based on convolution gradient optimization according to any one of claims 1 to 7 is implemented.

Images