Unsupervised image enhancement algorithm based on prior guidance

Abstract

The invention discloses an unsupervised image enhancement algorithm based on prior guidance, which comprises a DFMNet network structure, the DFMNet network structure comprises a main branch and an LAB color space adjustment branch, the main branch is used for color recovery and contrast enhancement of an image, the LAB color space adjustment branch is used for adjusting the brightness of the image, and the main branch is used for adjusting the contrast of the image; and two adjacent parts in the main branch and the LAB color space adjustment branch carry out information transmission through residual connection and a self-defined learnable mapping layer to ensure that the image is enhanced at multiple levels, and the unsupervised image enhancement algorithm based on prior guidance aims to improve the quality of the image in a low-light environment and improve the image enhancement efficiency. Especially in the aspects of contrast, brightness and color consistency, the DFMNet realizes effective enhancement of a low-illumination image by introducing two key branches, namely a main branch and an LAB color space adjustment branch.

Description

Technical Field

[0001] The present invention relates to the technical field of image enhancement, and specifically to an unsupervised image enhancement algorithm based on prior guidance. Background Art

[0002] With the development of low-light image enhancement technology, traditional image enhancement methods can no longer meet the image quality requirements in complex low-light environments. The main challenges of low-light images lie in the loss of image details, the increase in noise, and color deviation. Especially in night shooting or low-light environments, problems such as insufficient exposure, uneven brightness, and strong noise are often encountered. To address these issues, many deep learning-based image enhancement methods have emerged.

[0003] Among them, Zero-DCE (Zero-Reference Deep Curve Estimation) is a relatively popular low-light image enhancement technology. Zero-DCE and its variants mainly rely on deep neural networks to enhance low-light images by learning a highly adaptable image enhancement function. The basic steps of Zero-DCE are as follows: 1. Input image preprocessing: Zero-DCE first receives a low-light image and converts it into a format that can be processed by the neural network (such as resizing, normalizing, etc.). 2. Enhancement curve learning: Zero-DCE learns an enhancement curve suitable for the image through the convolutional layers in the network. This curve is automatically generated by the network without manual setting or reference image. 3. Iterative enhancement process: In each iteration, Zero-DCE predicts the enhancement curve through a convolutional neural network (CNN) and combines the current image with this enhancement curve. The result of each iteration is simply superimposed with the previous result to enhance the image brightness and contrast. 4. Output the enhanced image: Finally, the image after multiple iterations is output as the enhanced low-light image.

[0004] Although Zero-DCE has good enhancement effects, it has the problem of over-enhancement, and it is difficult to adaptively adjust the enhancement parameters in different scenarios due to the lack of support from supervision signals.

[0005] Zero-DCE simply superimposes the results of each iteration during multiple iterations, resulting in over-enhancement of image brightness, contrast, etc., thus affecting the naturalness of the image. Moreover, Zero-DCE fails to effectively utilize the supervision loss function, resulting in difficulty in optimizing the enhancement effect when dealing with complex low-light images. In addition, methods such as Zero-DCE fail to finely control the enhancement degree in each iteration and cannot adaptively adjust according to the local features of the image, resulting in possible over-enhancement or under-enhancement in some areas. Summary of the Invention

[0006] The purpose of this section is to outline some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section, as well as in the abstract and title of the present application, to avoid obscuring the purpose of this section, the abstract, and the title. However, such simplifications or omissions shall not be used to limit the scope of the present invention.

[0007] Therefore, the object of the present invention is to provide an unsupervised image enhancement algorithm based on prior guidance, aiming to improve the quality of images in low-light environments, especially in terms of contrast, brightness, and color consistency. DFMNet achieves effective enhancement of low-light images by introducing two key branches: the main branch and the LAB color adjustment branch.

[0008] To solve the above technical problems, according to one aspect of the present invention, the following technical solutions are provided:

[0009] An unsupervised image enhancement algorithm based on prior guidance, comprising: a DFMNet network structure, the DFMNet network structure including an input layer, a main branch, and an LAB color space adjustment branch;

[0010] The input layer is used to input a three-channel image with a resolution of [None, None, 3];

[0011] The main branch is used for color restoration and contrast enhancement of the image, and the LAB color space adjustment branch adjusts the brightness of the image;

[0012] Moreover, information is transmitted between adjacent two parts within the main branch and the LAB color space adjustment branch through residual connections and a custom learnable mapping layer to ensure that the image is enhanced at multiple levels.

[0013] As a preferred solution of the unsupervised image enhancement algorithm based on prior guidance according to the present invention, wherein, the main branch includes a plurality of residual blocks, and each residual block includes the following steps:

[0014] Customizable trainable mapping activation function;

[0015] Use Conv2D(3×3) convolution with 8 channels;

[0016] The feature map is concatenated with the original feature map through channel connection;

[0017] The output feature maps of the residual blocks are stacked layer by layer through channel connection, and the final number of channels is 64;

[0018] Use Conv2D(3×3) convolution with 3 channels to map the feature map to a 3-channel output, and the activation function is tanh.

[0019] As a preferred solution of an unsupervised image enhancement algorithm based on prior guidance according to the present invention, the color adjustment branch in the LAB color space adjustment branch includes 8 sub-branches, and the steps of each sub-branch are as follows:

[0020] The input image is converted into the luminance channel L and the chrominance channels A and B through the LAB color space;

[0021] The custom trainable mapping layer is used for non-linear luminance adjustment of the luminance channel L;

[0022] The adjusted L channel is combined with the original A and B channels and then converted back to the RGB color space;

[0023] The feature map is mapped to a 3-channel output using a Conv2D(3×3) convolution with 3 channels, and the activation function is tanh.

[0024] As a preferred solution of an unsupervised image enhancement algorithm based on prior guidance according to the present invention, the trainable mapping

[0025]

[0026] where where k is a trainable parameter value like the network model parameters, and max_k and min_k are set to 5 and 3 respectively.

[0027] As a preferred solution of an unsupervised image enhancement algorithm based on prior guidance according to the present invention, the total loss function L of the DFMNet network structure total consists of three parts, namely the illumination loss L illumination , the perceptual loss L perceptual and the color consistency loss function L color ;

[0028] The specific formula is as follows: L total = 10 * L illumination + L perceptual + L color .

[0029] As a preferred solution of an unsupervised image enhancement algorithm based on prior guidance according to the present invention, the illumination loss I illumination helps the model avoid overexposure or uneven brightness by calculating the smoothness of the image brightness, and the formula is as follows:

[0030]

[0031] where,

[0032]

[0033] where I is the enhanced image I obtained through model calculation enhanced , N is the picture number, H is the height of the picture, W is the width of the picture, C is the channel number of the image, and n, i, j, and c are all integers, where 1 ≤ n ≤ N, 1 ≤ i ≤ H, 1 ≤ j ≤ W, and 1 ≤ c ≤ C.

[0034] As a preferred solution of an unsupervised image enhancement algorithm based on prior guidance according to the present invention, the perceptual loss formula is as follows:

[0035]

[0036] where I enhanced is the enhanced image obtained by calculating the input low-illumination image I(x, y) through the DFMNet model, and y pre_enhanced is obtained by pre-enhancing the input original low-illumination image I(x, y), represents the pre-trained model, such as networks like VGG19, Squeeze, etc. The present invention uses the VGG19 network;

[0037] The pre-enhancement processing formula is as follows:

[0038]

[0039] where I(x, y) is the pixel value of the input low-illumination image, μ is the pixel mean when adjusting the contrast, and Max represents taking the maximum value.

[0040] As a preferred solution of an unsupervised image enhancement algorithm based on prior guidance according to the present invention, the color consistency loss function L color has the following formula:

[0041]

[0042] where I enhanced represents the enhanced image obtained by calculating the input picture through the model during model training, and I original is the input original low-illumination image.

[0043] Compared with the prior art, the beneficial effects of the present invention are:

[0044] 1. The DFMNet algorithm proposed by the present invention effectively solves the problems of contrast restoration, color restoration, and brightness enhancement in low-light image enhancement by combining the main branch and the LAB color adjustment branch. The main branch uses residual learning and a custom learnable mapping layer, which not only improves the image feature extraction ability but also overcomes the problem of gradient disappearance and enhances the model training efficiency. The LAB color adjustment branch dynamically adjusts the brightness through the conversion from RGB to the LAB space, enhancing the adaptability and stability of the image under different lighting conditions. Combining these technologies, DFMNet significantly improves the visual effect of low-light images, making the image details more abundant, the contrast and color performance more natural, and it is applicable to various practical application scenarios such as night image processing.

[0045] 2. The advantages of this invention are mainly reflected in the following aspects: First, the multi-branch network structure adopted can flexibly process multiple important features of the image, which can not only ensure the natural restoration of color and brightness but also achieve efficient feature extraction and enhancement. Second, with the help of the adaptive learning mechanism, DFMNet can adjust the processing strategy according to different images to ensure optimized effects under various low-light conditions and avoid the performance degradation of traditional methods under certain specific lighting conditions. Finally, by comprehensively using the supervised loss function and prior knowledge, the model not only improves the image quality but also enhances its generalization ability and robustness, effectively coping with various low-light scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below in conjunction with the drawings and detailed embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings. Among them:

[0047] Figure 1 It is a schematic diagram of the DFMNet network structure of an unsupervised image enhancement algorithm based on prior guidance of the present invention;

[0048] Figure 2 It is the training process of the DFMNet network structure. Since DFMNet belongs to an unsupervised algorithm and does not require paired pictures (normal illumination images corresponding to low-light images), in order to make more use of prior knowledge, the low-light images are pre-enhanced so that the model can use the supervised loss function;

[0049] Figure 3 It is the application test process of the DFMNet network structure. When a low-light image is input and processed by DFMNet, an enhanced image can be obtained;

[0050] Figure 4The low - illumination images, normal - illumination images, MIRNet, LightenNet, TBEFN, RetinexNet, R2RNet, DSLR, CLAHE, LIME, Zero - DCE, RUAS, SCI, PSENet provided by the present invention and the test results of the present invention. Detailed implementation manners

[0051] To make the above - mentioned objects, features, and advantages of the present invention more obvious and understandable, the following detailed description of the specific implementation manners of the present invention is provided in conjunction with the accompanying drawings.

[0052] The present invention provides an unsupervised image enhancement algorithm based on prior guidance, aiming to improve the quality of images in low - light environments, especially in terms of contrast, brightness, and color consistency.

[0053] DFMNet realizes the effective enhancement of low - light images by introducing two key branches: the main branch and the LAB color adjustment branch. The following sections will elaborate on each component of this technical solution in detail and help with understanding through the accompanying drawings.

[0054] 1. System architecture and function implementation

[0055] The overall structure of DFMNet is as Figure 1 shown. The model consists of two major branches: the main branch and the LAB color adjustment branch. The main branch is responsible for image color restoration and contrast enhancement, while the LAB color adjustment branch is mainly responsible for image brightness adjustment. Each part of the model conducts information transmission through residual connections and a custom - designed learnable mapping layer to ensure image enhancement at multiple levels.

[0056] 2. Model structure

[0057] (1) Input layer

[0058] Input: A three - channel image with a resolution of [None, None, 3];

[0059] (2) Main branch

[0060] Use a Conv2D(3×3) convolution with 8 channels (stride of 1, activation function of ReLU, and boundary padding of "same").

[0061] Residual block structure

[0062] Each residual block contains the following steps:

[0063] Custom - trainable mapping activation function;

[0064] Use a Conv2D(3×3) convolution with 8 channels (stride of 1, activation function ReLU, padding "same");

[0065] The feature map is concatenated with the original feature map through channel concatenation;

[0066] There are a total of 8 residual blocks, each block outputting 8 feature channels;

[0067] The output feature maps of the residual blocks are stacked layer by layer through channel concatenation, and the final number of channels is 64;

[0068] Output stage

[0069] Use a Conv2D(3×3) convolution with 3 channels to map the feature map to a 3-channel output, with the activation function being tanh.

[0070] Auxiliary LAB color adjustment branch

[0071] (3) LAB color space adjustment branch. The color adjustment branch contains 8 sub-branches, and the steps of each sub-branch are the same, as follows:

[0072] The input image is converted to the luminance channel (L) and chrominance channels (A, B) through the LAB color space conversion (RGB2LAB);

[0073] Use a custom trainable mapping layer to perform non-linear luminance adjustment on the luminance channel (L);

[0074] The adjusted L channel is combined with the original A and B channels (LAB2RGB), and then converted back to the RGB color space;

[0075] Use a Conv2D(3×3) convolution with 3 channels to map the feature map to a 3-channel output, with the activation function being tanh.

[0076] Each LAB adjustment generates a result, and there are a total of 8 LAB adjustment branches;

[0077] (4) Combination and enhancement:

[0078] The output of each LAB adjustment is fused with the main branch result through step-by-step accumulation (Add operation).

[0079] Form a gradually enhanced image effect and output the final result.

[0080] (5) The trainable mapping described in steps 2 and 3

[0081]

[0082] Among them Among them, k, like the network model parameters, is a trainable parameter value.

[0083] max_k and min_k are set to 5 and 3 respectively.

[0084] (6) Loss function

[0085] The total loss function L total consists of three parts, as shown in Equation (2), namely the illumination loss L illumination , the perceptual loss L perceptual and the color consistency loss function L color .

[0086] L total = 10 * L illumination + L perceptual + L color (2)

[0087] The illumination loss L illumination : By calculating the smoothness of the image brightness, it helps the model avoid overexposure or uneven brightness phenomena, as shown in Equation (3).

[0088]

[0089]

[0090]

[0091] Among them, I is the enhanced image I obtained through model calculation, enhanced , N is the picture number, H is the picture height, W is the picture width, and C is the image channel number. n, i, j, and c are all integers, 1 ≤ n ≤ N, 1 ≤ i ≤ H, 1 ≤ j ≤ W, 1 ≤ c ≤ C.

[0092] The perceptual loss is as shown in Equation (6).

[0093]

[0094] Among them, I enhanced is the enhanced image obtained by calculating the input low-light image I(x, y) through the DFMNet model, and y pre_enhanced is obtained by pre-enhancing the input original low-light image I(x, y). represents the pre-trained model, such as networks like VGG19, SqueezeNet, etc. In this invention, the VGG19 network is adopted.

[0095] The pre-enhancement processing steps are as shown in Equation (7):

[0096]

[0097] Among them, I(x, y) is the pixel value of the input low-illumination image, μ is the pixel mean when adjusting the contrast, and Max represents taking the maximum value.

[0098] Color consistency loss function L color As shown in formula (8):

[0099]

[0100] Where I enhanced represents the enhanced image obtained by the input image passing through the model calculation during the model training process. I original is the original low-illumination image input.

[0101] 5. Number of model parameters

[0102] Figure 2 For the training process of the DFMNet network structure, the number of model parameters of DFMNet is relatively small, and the total number of parameters is 6897. This model includes multiple convolutional layers, residual blocks, and custom learnable mapping layers. The number of parameters in each layer has been carefully designed to balance the complexity and computational efficiency of the model. The floating-point operation count of the model is 1.6 GFLOPs.

[0103] 6. To further verify the technical effect of the present invention, as Figure 3 shown, for the application test of the DFMNet network structure, when inputting a low-illumination image and processing it through DFMNet, an enhanced image can be obtained, providing Figure 4 as well as the test results in Table 1 and Table 2.

[0104] Table 1 - Objective evaluation results of LOL v2 database

[0105]

[0106] Table 2 - Floating-point operation count, parameters, running time, and platform information

[0107]

[0108]

[0109] 7. Summary of technical solutions

[0110] From Figure 4From the test results in Table 1 and Table 2, it can be seen that by introducing the main branch and the LAB color adjustment branch, DFMNet can effectively enhance the quality of low-light images while ensuring the color consistency and brightness balance of the images. Through deep residual learning, learnable mapping layers, and adaptive brightness adjustment methods, the model realizes the deep feature learning and effective enhancement of low-light images. The combination of these technical solutions improves the effect of image enhancement and ensures the stability and efficiency of the model in diverse lighting environments compared with other solutions.

[0111] Although the present invention has been described above with reference to the embodiments, various improvements can be made thereto and components thereof can be replaced with equivalents without departing from the scope of the present invention. In particular, as long as there is no structural conflict, the various features in the disclosed embodiments of the present invention can be combined with each other in any way, and the exhaustive description of the situations of these combinations is omitted in this specification only for the sake of saving space and resources. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. An unsupervised image enhancement algorithm based on prior guidance, characterized in that: include: DFMNet network structure, the DFMNet network structure includes an input layer, a main branch and a LAB color space adjustment branch; The input layer is used to input a three-channel image with a resolution of [None,None,3]; The main branch is used to restore the color and enhance the contrast of the image, and the LAB color space adjustment branch is used to adjust the brightness of the image; Furthermore, information is transmitted between two adjacent parts in the main branch and the LAB color space adjustment branch through residual connections and a custom learnable mapping layer, ensuring that the image is enhanced at multiple levels.

2. The unsupervised image enhancement algorithm based on prior guidance according to claim 1, characterized in that: The main branch includes a plurality of residual blocks, each of which includes the following steps: Custom trainable mapping activation functions; Use Conv2D(3×3) convolution with 8 channels; The feature map is concatenated with the original feature map through channel connection; The output feature map of the residual block is stacked layer by layer through channel connections, and the final number of channels is 64; Use Conv2D (3 × 3) convolution with 3 channels to map the feature map to 3-channel output, and the activation function is tanh.

3. The unsupervised image enhancement algorithm based on prior guidance according to claim 2, characterized in that: LAB color space adjustment branch The color adjustment branch contains 8 sub-branches, and the steps of each sub-branch are as follows: The input image is converted into the luminance channel L and the chrominance channels A and B through the LAB color space; Use a custom trainable mapping layer to perform nonlinear brightness adjustment on the brightness channel L; The adjusted L channel is combined with the original A and B channels and then converted back to RGB color space; Use Conv2D (3 × 3) convolution with 3 channels to map the feature map to 3-channel output, and the activation function is tanh. Each LAB adjustment generates a result, and a total of 8 LAB adjustment branches are performed.

4. The unsupervised image enhancement algorithm based on prior guidance according to claim 3, characterized in that: The trainable mapping in Here, k is the same as the network model parameter, which is a trainable parameter value. Max_k and min_k are set to 5 and 3 respectively.

5. The unsupervised image enhancement algorithm based on prior guidance according to claim 1, characterized in that: The total loss function L of the DFMNet network structure total It consists of three parts: light loss L illumination , Perceptual loss L perceptual And the color consistency loss function L color ; The specific formula is as follows: total =10*L illumination +L perceptual +L color .

6. The unsupervised image enhancement algorithm based on prior guidance according to claim 5, characterized in that: Light loss L illumination By calculating the smoothness of the image brightness, the model can avoid overexposure or uneven brightness. The formula is as follows: in, Where I is the enhanced image I obtained by model calculation enhanced , N is the image number, H is the height of the image, W is the width of the image, C is the channel number of the image, n, i, j and c are all integers, 1≤n≤N, 1≤i≤H, 1≤j≤W, 1≤c≤C.

7. The unsupervised image enhancement algorithm based on prior guidance according to claim 5, characterized in that: The perceptual loss formula is as follows: Among them I enhanced is the enhanced image obtained by calculating the DFMNet model after inputting the low-light image I(x,y), pre_enhanced It is obtained by pre-enhancing the input original low-light image I(x,y). Represents a pre-trained model, such as VGG19, squeeze and other networks. The present invention uses the VGG19 network; The pre-enhancement processing formula is as follows: Among them, I(x, y) is the pixel value of the input low-light image, μ is the pixel mean when adjusting the contrast, and Max means taking the maximum value.

8. The unsupervised image enhancement algorithm based on prior guidance according to claim 5, characterized in that: Color consistency loss function L color The formula is as follows: Among them I enhanced It represents the enhanced image obtained by the model calculation during the model training process. original is the original low-light image input.

Images