Low-illumination Image Enhancement Method with Adjustable Brightness Based on Pure Noise Training

Abstract

The present invention relates to a low - illumination image enhancement method with adjustable brightness based on pure noise training. Its technical features are as follows: During the training stage, random Gaussian noise is sampled as the low - illumination input image; the low - illumination input image and the brightness change factor are respectively sent into the image processing module and the brightness processing module to obtain the feature map and the feature vector; through operations of convolution and non - linear transformation on the feature map and the feature vector, the fusion and reconstruction of features are realized, and curve parameters with 6 output channels are output; the low - illumination input image is enhanced into an output image with a specific brightness guided by the amplitude of the brightness change factor; mean and contrast transformations are performed to obtain a pseudo - reference image; the total loss is calculated, and the network is iteratively optimized. The present invention adopts a pure - noise training strategy, which helps the model bypass the commonly used color constancy and illumination smoothing losses, simplifies the difficult - to - design non - linear curve form into a linear curve form, and realizes the function of cost reduction and efficiency improvement in the field of low - illumination enhancement.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision images, relates to an image enhancement method, and in particular to a low-light image enhancement method with adjustable brightness based on pure noise training. Background Art

[0002] With the digital technology entering thousands of households, images have gradually become an important way for people to capture moments and convey ideas. However, in low-light scenarios and when the exposure time is insufficient, it is easy to take images with problems such as more noise, color distortion, and insufficient brightness. This not only makes the images unsatisfactory in terms of visual perception but may also reduce the reliability of downstream target recognition, detection, and other systems. Therefore, a post-processing method with the ability to repair illumination is needed for image enhancement. Among them, most of the existing low-light enhancement methods based on deep learning require paired low-light - normal illumination datasets for training. The acquisition of paired datasets is time-consuming and requires professional techniques. Slight operation errors will result in unusable image pairs. At the same time, the manually captured datasets can only cover limited scenarios in the real world, which will weaken the generalization ability of the method to handle complex scenarios. Various existing image enhancement methods also do not make detailed enhancement processing for different brightness levels of images in complex scenarios. Therefore, how to provide a more flexible low-light image brightness adjustment scheme based on less image acquisition cost is a challenging research topic in the field of computer vision.

[0003] Low-Light Image Enhancement refers to the technology of enhancing an image with low contrast, insufficient illumination, slight noise, and color deviation to a corresponding normal illumination and high-quality image. This technology can assist people in observing and evaluating images and also provide support for computers to understand and process images. It plays an important role in fields such as monitoring and security.

[0004] Traditional low-light image enhancement methods can be roughly divided into three categories: (1) Methods based on histogram equalization. After the equalization function acts on the image, the pixels of the whole image will be rearranged according to probability to achieve an approximately uniform distribution effect. (2) Methods based on non-linear transformations such as gamma correction, which helps to adjust the image contrast while retaining details. (3) Methods based on the Retinex theory, which decouples the image into a reflectance map and an illumination map and makes further enhancements according to their respective characteristics.

[0005] In the context of the great development of deep neural network models, many advanced methods have been proposed and used for low-light enhancement problems. Among them, according to the data used during training, they can be classified into the following learning strategies: (1) Supervised learning: Both the RetinexNet and KinD enhancement methods consist of two parts of networks, namely, a decomposition network and an enhancement network. The decomposition network is responsible for decoupling the input image into a reflectance map independent of light intensity and a smooth illumination map with structure perception ability, and the enhancement network is responsible for further brightening the illumination map to achieve low-light restoration. (2) Unsupervised learning: EnlightenGAN uses an attention-guided U-Net type generator and a discriminator to calibrate the enhanced image to make it close to the sensory effect of the real world. (3) Zero-shot learning: The RRDNet method does not require prior training. It only needs to continuously iterate for a single test image and minimize a specialized loss function composed of a reconstruction loss, a texture enhancement loss, and an illumination-guided noise prediction loss. The RetinexDIP method combines the Retinex decomposition idea and the deep network prior technology based on DeepImage Prior (DIP), that is, randomly sampled white noise is fed into the model, and two DIP networks respectively generate the reflectance and illumination maps of the original image. (4) No-reference learning: RUAS is a newly proposed no-reference learning method based on architecture search; Zero-DCE formulates brightness enhancement as a prediction task of an image-specific curve; SCI is a self-calibrating illumination learning architecture. Its training adopts a cascaded mode, and a self-calibrating module is added before each cascaded illuminance prediction block, which is beneficial to the convergence of the results at each stage. Only the first basic block in the network is considered in the test stage, which reduces the computational cost. Although the above methods gradually reduce the use of paired supervised data during the training process, they still require unpaired data related to the low-light task to assist in training, which makes the quality of image acquisition have a great impact on the training effect. In addition, the exposure control loss designed by methods such as Zero-DCE stipulates a fixed target exposure level, which makes users unable to fine-tune the brightness level of the output image according to their personal preferences. Zero-shot methods such as RRDNet have outstanding effects, but they take several minutes to test a single image and cannot balance the test effect and real-time performance. Summary of the Invention

[0006] An object of the present invention is to overcome the deficiencies of the prior art and provide a low-light image enhancement method with adjustable brightness based on pure noise training, which is reasonably designed and can effectively improve the training efficiency and test efficiency.

[0007] The technical problem of the present invention is solved by adopting the following technical solutions:

[0008] A low-light image enhancement method with adjustable brightness based on pure noise training includes the following steps:

[0009] Step 1: In the image input stage, randomly sample from the Gaussian distribution to form a noise image set, and normalize the image pixel values to [0, 1] to obtain the low-illumination input image I;

[0010] Step 2: Send the low-illumination input image I and the brightness change factor V into the image processing module and the brightness processing module of the network respectively for feature extraction, to obtain the feature map corresponding to the low-illumination input image I and the feature vector corresponding to the brightness change factor;

[0011] Step 3: Through operations of convolution and non-linear transformation on the feature map and the feature vector obtained in Step 2, realize the fusion and reconstruction of features, and output the curve parameters with 6 channels, where the first 3 channels and the last 3 channels respectively represent two different parameter terms k and b of the curve;

[0012] Step 4: Through the linear curve formula: O(x) = k(x)I(x) + b(x), enhance the low-illumination input image I into the output image O guided by the amplitude of the brightness change factor;

[0013] Step 5: Perform mean and contrast transformation on the low-illumination input image I obtained in Step 1 to obtain the pseudo-reference image I'';

[0014] Step 6: Use the low-illumination input image I obtained in Step 1, the output image O obtained in Step 4, and the pseudo-reference image I'' obtained in Step 5 to calculate the total loss, and iteratively optimize the network;

[0015] The specific implementation method of Step 3 includes:

[0016] Step 3.1: Reshape the feature map and the feature vector obtained in Step 2 into a feature map with a dimension of 4, to obtain an image feature map of 1×32×128×128 and a brightness feature map of 1×32×1×1;

[0017] Step 3.2: Perform an inner product on the two 4D feature maps in Step 3.1 to obtain an image with a size of 1×1×128×128. This image then passes through a convolutional layer with a kernel size of 3×3 and the subsequent Sigmoid activation function to obtain the curve parameters k and b;

[0018] The specific implementation method of Step 6 includes the following steps:

[0019] Step 6.1: If in the test stage, end Step 6; if in the training stage, enter Step 6.2;

[0020] Step 6.2: Through the formula: L rec (I'', O) = L1(I'', O) = ∑|I''(x) - O(x)| to calculate the reconstruction loss, and obtain the pixel-level mean absolute error between the output image O and the pseudo-reference image I'';

[0021] Step 6.3: Calculate the spatial consistency loss through the formula: where \(O'\) and \(I'\) respectively represent the images obtained after the output and input images undergo spatial and channel average pooling, \(K\) represents the number of remaining local regions after the image undergoes 4×4 spatial pooling, and \(\Omega(i)\) are the 4 nearest neighbors of the central region \(i\);

[0022] Step 6.4: Sum the losses obtained in Step 6.2 and Step 6.3 to obtain the total loss. The network aims to minimize the total loss and continuously iterates the parameters of each module to optimize the performance.

[0023] Furthermore, the specific implementation method of the said Step 1 includes:

[0024] Step 1.1: If in the test phase, directly enter Step 2; otherwise, enter Step 1.2;

[0025] Step 1.2: Sample an input noise image \(I\) with a size of 128×128×3 from a standard Gaussian distribution with a mean of 0 and a standard deviation of 1 noise ;

[0026] Step 1.3: Calculate the maximum pixel value \(I_{max}\) and the minimum pixel value \(I_{min}\) of the input noise image \(I\) respectively, and calculate the normalized low - illumination input image \(I\) through the formula noise of the input noise image \(I\) max and the minimum pixel value \(I\) min through the formula

[0027] Furthermore, the image processing module is composed of a convolutional layer and an activation function; the brightness processing module is composed of a linear layer, a convolutional layer, and an activation function.

[0028] Furthermore, the specific implementation method of the said Step 2 includes:

[0029] Step 2.1: Send the low - illumination input image \(I\) into the image processing module of the network, and perform non - linear transformation through 6 symmetrically cascaded convolutional layers of the image processing module and the ReLU activation function following each convolutional layer. The size of the convolutional kernel used in each convolutional layer is 3×3, and the convolution operation is set in the form of a stride of 1 and a padding mode of replicating one circle. Finally, output the feature map corresponding to the low - illumination input image \(I\);

[0030] Step 2.2: Send the brightness change factor \(V\) into the brightness processing module of the network. After passing through the linear fully - connected layer in the brightness processing module, expand the sequence with a size of 1 into a sequence with a size of 32, and output the feature vector corresponding to the brightness change factor.

[0031] Furthermore, the specific implementation method of the said Step 5 includes:

[0032] Step 5.1: If in the test phase, end Step 5; if in the training phase, proceed to Step 5.2;

[0033] Step 5.2: Apply contrast transformation to the low - illumination input image I obtained in Step 1 through the formula: I'(x)=(I(x)-0.5)*r + 0.5 or to obtain the transformed image I', where r represents the new value width of the image for adjusting the dynamic range, d represents the intensity of the contrast transformation for adjusting the kurtosis of the Gaussian distribution, ε1 and ε2 represent two extremely small positive numbers, and clip represents the clipping operation to control the value range within [0,1];

[0034] Step 5.3: Add the value of the brightness change factor to the image I' obtained in Step 5.2 to achieve mean transformation and obtain the transformed pseudo - reference image I".

[0035] Furthermore, the value range of the brightness change factor V is: - 0.5 to 1.0.

[0036] The advantages and positive effects of the present invention are:

[0037] 1. The present invention adopts a pure - noise training strategy, reducing the acquisition and screening costs of real - world data sets. At the same time, using random Gaussian noise training can help the model bypass common color constancy and illumination smoothing losses, simplify the difficult - to - design non - linear curve form into a linear curve form, and achieve the function of reducing costs and increasing efficiency in the field of low - illumination enhancement.

[0038] 2. The present invention introduces the brightness change factor as another input item of the network. By integrating the brightness processing module with the image processing module of the backbone, the brightness of the output image can be continuously and real - time adjusted during the test phase. With the help of the pure - noise strategy, the present invention gradually updates the "many - to - one" brightness adjustment method commonly used in existing methods to "one - to - one" and "one - to - many", providing a more flexible brightness adjustment scheme.

[0039] 3. The present invention is not only superior to existing methods in quantitative index evaluation and qualitative visual assessment, but also has high training efficiency and test efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is the overall network architecture of the present invention;

[0041] Figure 2 is the framework diagram of the image processing module and the brightness processing module of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0042] The following further details the embodiments of the present invention in conjunction with the accompanying drawings:

[0043] A low - illumination image enhancement method with adjustable brightness based on pure noise training, as Figure 1 and Figure 2 shown, includes the following steps:

[0044] Step 1: In the image input stage, randomly sample the Gaussian distribution to form a noise image set, and normalize the image pixel values to [0, 1] to obtain the low - illumination input image I. The specific implementation method of this step is as follows:

[0045] Step 1.1: If in the test stage, directly enter Step 2; if in the training stage, perform the following operations.

[0046] Step 1.2: Sample an input noise image I with a size of 128×128×3 from a standard Gaussian distribution with a mean of 0 and a standard deviation of 1 noise .

[0047] Step 1.3: Calculate the maximum pixel value I noise and the minimum pixel value I max of the input noise image I respectively, and obtain the normalized low - illumination input image I through the formula min .

[0048] Step 2: Send the low - illumination input image I and the brightness change factor V into the network, and perform feature extraction through the image processing module and the brightness processing module in the network respectively, and output the feature map corresponding to the low - illumination input image I and the feature vector corresponding to the brightness change factor. The specific implementation method of this step is as follows:

[0049] Step 2.1: Send the low - illumination input image I into the image processing module of the network. This image processing module is composed of a convolutional layer and an activation function. The convolutional layer is 6 symmetrically cascaded convolutional layers, and each convolutional layer is followed by a ReLU activation function to achieve non - linear transformation. The size of the convolutional kernel used in each convolutional layer is 3×3, and the convolution operation is set in the form of a step size of 1 and a padding mode of replicating a circle, and finally output the feature map corresponding to the input image.

[0050] Step 2.2: Send the brightness change factor V into the brightness processing module of the network. This brightness processing module is composed of a linear layer, a convolutional layer and an activation function. After passing through the linear fully - connected layer, the sequence with a size of 1 is expanded into a sequence with a size of 32, and output the feature vector corresponding to the brightness change factor.

[0051] ​Step 3: The feature map and feature vector obtained in Step 2 are subjected to operations of convolution and non-linear transformation to achieve feature fusion and reconstruction, and curve parameters with 6 output channels are output, where the first 3 channels and the last 3 channels respectively represent two different parameter terms k and b of the curve. The specific implementation method of this step is as follows:

[0052] Step 3.1: The feature map and feature vector obtained in Step 2 are reshaped into a feature image with a dimension of 4, that is, an image feature map of 1×32×128×128 and a brightness feature map of 1×32×1×1.

[0053] Step 3.2: The inner product of the two 4D feature maps in Step 3.1 is taken to obtain an image with a size of 1×1×128×128. This image then passes through a convolutional layer with a kernel size of 3×3 and the subsequent Sigmoid activation function to obtain the curve parameters k and b.

[0054] Step 4: Through the linear curve formula O(x) = k(x)I(x) + b(x), the low-illumination input image I is enhanced into an output image O with a specific brightness guided by the amplitude of the brightness change factor.

[0055] Step 5: Mean and contrast transformations are performed on the low-illumination input image obtained in Step 1. The transformed image is used as a pseudo-reference image to participate in model training and optimization. The specific implementation method of this step is as follows:

[0056] Step 5.1: If in the test phase, this step is ignored, that is, Step 4 is regarded as the final step; if in the training phase, enter this step.

[0057] Step 5.2: The low-illumination input image I obtained in Step 1 first undergoes contrast transformation through the formula I'(x) = (I(x) - 0.5)*r + 0.5 or to obtain the transformed image I', where r represents the new value width of the image for adjusting the dynamic range, and d represents the severity of the contrast transformation for adjusting the kurtosis of the Gaussian distribution. The larger d is, the flatter the transformed image is and the smaller the hue contrast is. ε1 and ε2 represent two extremely small positive numbers to prevent division by zero or the logarithm being zero inside, and clip represents a clipping operation to control the value range within [0,1].

[0058] Step 5.3: The image I' obtained in Step 5.2 then undergoes mean transformation by adding the value of the brightness change factor, that is, I”(x) = I'(x) + V, to obtain the final pseudo-reference image I” after transformation, where the brightness reference factor can take any value within the range of [-0.5, 1.0].

[0059] Step 6: Calculate the total loss using the low - illumination input image I obtained in Step 1, the output image O obtained in Step 4, and the pseudo - reference image I” obtained in Step 5, and iteratively optimize the network.

[0060] Step 6.1: If in the test phase, ignore this step, that is, regard Step 4 as the final step; if in the training phase, enter this step.

[0061] Step 6.2: Calculate the reconstruction loss through the formula L rec (I”, O)=L1(I”, O)=∑|I”(x)-O(x)|, that is, calculate the pixel - level mean absolute error between the output image and the pseudo - reference image.

[0062] Step 6.3: Calculate the spatial consistency loss through the formula where O' and I' respectively represent the images obtained after spatial and channel average pooling of the output and input images, K represents the number of remaining local regions after 4×4 spatial pooling of the image, and Ω(i) are the 4 nearest neighbors of the central region i.

[0063] Step 6.4: Sum the losses in Step 6.2 and Step 6.3 to obtain the total loss. The network aims to minimize the total loss and continuously iterates the parameters of each module to optimize the performance.

[0064] Next, test according to the method of the present invention to illustrate the effect of the present invention.

[0065] Test environment: python3.8.15; PyTorch framework; Ubuntu18.04 system; NVIDIA GTX 1080Ti GPU.

[0066] Test sequences: The selected datasets are the low - light datasets LOL, LSRW, NPE, MEF, and LIME for low - illumination image enhancement. Among them, the LOL, LSRW_huawei, and LSRW_nikon datasets contain 15 pairs, 30 pairs, and 20 pairs of low - illumination - normal illumination image pairs respectively, and the NPE, MEF, and LIME datasets contain 8, 17, and 10 low - illumination images respectively.

[0067] Test metrics: The present invention uses PSNR and SSIM metrics to evaluate the reference test set, and uses NIQE and BRISQUE metrics to evaluate the non - reference test set. Calculate the metric data for different popular methods and then compare the results. In addition, compare the visual effects of the images under different methods to prove that the present invention can obtain better results in the field of low - illumination enhancement.

[0068] The test results are as follows:

[0069] Table 1. Performance Comparison between the Present Invention and Other Methods with a Reference Dataset

[0070]

[0071]

[0072] Table 2. Performance Comparison between the Present Invention and Other Methods without a Reference Dataset

[0073]

[0074] Table 3. Efficiency Comparison between the Present Invention and Other Methods

[0075]

[0076] From the above comparison data, it can be seen that the present invention achieves two optimal and two sub-optimal results on the reference dataset, one optimal and two sub-optimal results on the non-reference dataset, and is optimal in terms of training efficiency, showing a significant improvement compared with the existing methods.

[0077] It should be emphasized that the embodiments described in the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation manners. Any other implementation manners obtained by those skilled in the art based on the technical solutions of the present invention also fall within the scope of protection of the present invention.

Claims

1. A low-light image enhancement method with adjustable brightness based on pure noise training, characterized in that: It includes the following steps: Step 1: In the image input stage, randomly sample from a Gaussian distribution to form a set of noise images, and normalize the image pixel values to [0, 1] to obtain a low-illumination input image I; Step 2: Send the low-illumination input image I and the brightness change factor V into the image processing module and the brightness processing module of the network respectively for feature extraction to obtain a feature map corresponding to the low-illumination input image I and a feature vector corresponding to the brightness change factor; Step 3: Through operations of convolution and non-linear transformation on the feature map and the feature vector obtained in Step 2, realize the fusion and reconstruction of features, and output curve parameters with 6 channels, where the first 3 channels and the last 3 channels represent 2 different parameter terms k and b of the curve respectively; Step 4: Through the linear curve formula: O(x) = k(x)I(x) + b(x), enhance the low-illumination input image I into an output image O guided by the amplitude of the brightness change factor; Step 5: Perform mean and contrast transformation on the low-illumination input image I obtained in Step 1 to obtain a pseudo-reference image I''; Step 6: Use the low-illumination input image I obtained in Step 1, the output image O obtained in Step 4, and the pseudo-reference image I'' obtained in Step 5 to calculate the total loss, and iteratively optimize the network; The specific implementation method of Step 3 includes: Step 3.1: Reshape the feature map and the feature vector obtained in Step 2 into a feature map with a dimension of 4 to obtain an image feature map of 1×32×128×128 and a brightness feature map of 1×32×1×1; Step 3.2: Perform an inner product on the two 4D feature maps in Step 3.1 to obtain an image with a size of 1×1×128×128. This image then passes through a convolutional layer with a kernel size of 3×3 and a subsequent Sigmoid activation function to obtain the curve parameters k and b; The specific implementation method of Step 6 includes the following steps: Step 6.1: If in the test stage, end Step 6; if in the training stage, enter Step 6.2; Step 6.

2. Calculate the reconstruction loss through the formula: L rec (I”, O) = L1(I”, O) = ∑|I”(x) - O(x)| to obtain the pixel-level mean absolute error between the output image O and the pseudo-reference image I”. Step 6.3: Through the formula: calculate the spatial consistency loss, where O' and I' respectively represent the images obtained after the output and input images undergo spatial and channel average pooling, K represents the number of remaining local regions after the image undergoes 4×4 spatial pooling, and Ω(i) are the 4 nearest neighbors of the central region i; Step 6.4: Sum the losses obtained in Step 6.2 and Step 6.3 to obtain the total loss. The network aims to minimize the total loss and continuously iterates the parameters of each module to optimize the performance.

2. The method for enhancing low-light images with adjustable brightness based on pure noise training according to claim 1, wherein: The specific implementation method of Step 1 includes: Step 1.1: If in the test stage, directly enter Step 2, otherwise enter Step 1.2; Step 1.2: Sample an input noise image I of size 128×128×3 from a standard Gaussian distribution with a mean of 0 and a standard deviation of 1 noise ; Step 1.

3. Calculate the maximum pixel value \(I_{max}\) and the minimum pixel value \(I_{min}\) of the input noisy image \(I\) respectively, and obtain the normalized low-light input image \(I\) through the formula: noise \(I_{max}\) max and \(I_{min}\) min . Through the formula ​ 3. The low-light image enhancement method with adjustable brightness based on pure noise training according to claim 1, wherein: The image processing module consists of a convolutional layer and an activation function; the brightness processing module consists of a linear layer, a convolutional layer, and an activation function.

4. The low-light image enhancement method with adjustable brightness based on pure noise training according to claim 3, characterized in that: The specific implementation method of Step 2 includes: Step 2.1: Send the low-illumination input image I into the image processing module of the network, and perform non-linear transformation through 6 symmetrically cascaded convolutional layers of the image processing module and the ReLU activation function following each convolutional layer. The size of the convolutional kernel used in each convolutional layer is 3×3, and the convolution operation is set to a stride of 1 and a padding mode of replicating a circle. Finally, output the feature map corresponding to the low-illumination input image I; Step 2.2: Send the brightness change factor V into the brightness processing module of the network. Through the linear fully-connected layer in the brightness processing module, expand the sequence with a size of 1 into a sequence with a size of 32, and output the feature vector corresponding to the brightness change factor.

5. The low-light image enhancement method with adjustable brightness based on pure noise training according to claim 1, characterized in that: The specific implementation method of step 5 includes: Step 5.1: If it is in the test stage, end step 5; if it is in the training stage, enter step 5.2; Step 5.2: Apply the formula \(I'(x)=(I(x) - 0.5)\times r+0.5\) to the low - illumination input image \(I\) obtained in Step 1, or perform contrast transformation to obtain the transformed image \(I'\), where \(r\) represents the new value width of the image, used to adjust the dynamic range, \(d\) represents the intensity of the contrast transformation, used to adjust the kurtosis of the Gaussian distribution, \(\varepsilon_1\) and \(\varepsilon_2\) represent two extremely small positive numbers, and clip represents a clipping operation to control the value range within \([0,1]\); Step 5.3: Add the image I' obtained in step 5.2 to the value of the brightness change factor to achieve mean transformation and obtain the transformed pseudo-reference image I".

6. The low-light image enhancement method with adjustable brightness based on pure noise training according to any one of claims 1 to 5, characterized in that: The value range of the brightness change factor V is: -0.5 to 1.0.

Images