A method for enhancing dark light images

Abstract

The present invention provides a dark light image enhancement method, which completes the dark light image enhancement task in RAW format by constructing a two-stage image enhancement network. First, in the denoising stage, a CNN-based denoising module is designed to process the input image. This module is combined with the designed wavelet feature extraction module to capture noise characteristics of different frequencies to enhance the denoising effect. Subsequently, the denoised image is subjected to color restoration and brightness correction. In the second stage, a Transformer global correction module containing wavelet convolution is designed to better utilize the low-frequency information required for color restoration. In order to achieve dynamic brightness adjustment of different noise areas, the present invention adds a long-short-range interaction module that introduces an SNR map in the hidden layer of the network in the second stage. This module consists of a detail processing module and a Transformer global correction module. Finally, the processed RAW format image is converted into RGB format output.

Description

Technical Field

[0001] The present invention belongs to the technical field of image enhancement, and in particular relates to a dark light image enhancement method. Background Art

[0002] When photographing in poor lighting conditions, such as at night, in dim light, or in the shadows of buildings, the resulting images often exhibit poor quality due to the camera's limited dynamic range and reduced contrast. These low-light conditions often exhibit overall dimness, blurred details, and color distortion, negatively impacting downstream tasks such as object detection and semantic segmentation. Image enhancement techniques are used to enhance image brightness and contrast, restore true colors, and eliminate potential noise, significantly improving the visual experience and playing a crucial preprocessing role for high-level visual tasks.

[0003] Compared to RGB images, RAW images have higher color depth, recording more distinguishable low-intensity signals. Furthermore, the noise distribution in RAW images is easier to handle before the image signal processing pipeline. For applications requiring high-quality images and precise control, RAW images are the preferred choice. Therefore, enhancing RAW low-light images can facilitate deeper data mining and analysis in the industrial sector.

[0004] Previous image enhancement methods have relied on distribution mapping techniques, such as curve transformation and histogram equalization, to improve the pixel distribution of an image. These methods, due to their inability to develop effective models tailored to specific lighting conditions and their neglect of the intrinsic connections between image pixels, are limited in their ability to accurately capture and utilize semantic information. With the rise of convolutional neural networks (CNNs), deep learning-based image enhancement schemes have demonstrated advantages over traditional model-driven approaches, both in processing speed and image quality. CNNs excel at learning local features and focusing on detailed features in image enhancement tasks, but they are limited in capturing long-range correlations and non-local self-similarity, which are crucial for image restoration. Currently, most RAW image enhancement methods utilize CNNs, which map RAW images to the RGB domain by converting color channels to enhance low-light RAW images. Transformers were first proposed in natural language processing for machine translation. With the emergence of various variants, Transformers have also been applied to various machine vision tasks, such as image classification, image restoration, and image synthesis. Through its self-attention mechanism, the Transformer is able to capture long-range dependencies between image components, which is crucial for image enhancement. When processing an image, it can better understand and process the image's global information, thereby maintaining a more coherent visual effect when enhancing the image. The Transformer can effectively integrate multi-scale and multi-level features, which is a key advantage for image enhancement. During the image enhancement process, the Transformer is able to fuse features at different levels to produce higher-quality output images.

[0005] Wavelet transform theory inherits the ideas of Fourier transform theory and provides a more flexible and localized method to analyze the characteristics of signals. It is used to analyze data in both the time domain and the frequency domain for signals and images. The wavelet transform begins with a function called the mother wavelet ψ(t), which is a waveform of finite length whose energy is concentrated in a finite interval of time. The mother wavelet must meet certain conditions to ensure that the wavelet transform is reversible. The wavelet transform function is shown below:

[0006]

[0007] The discrete wavelet transform (DWT) is a type of wavelet transform. By discretizing the scale and displacement of the wavelet function, the DWT makes the transform computationally efficient and suitable for practical applications. The DWT performs multiresolution analysis by decomposing a signal into approximate coefficients and detail coefficients at different scales. The approximate coefficients capture the signal's low-frequency components, while the detail coefficients capture the signal's high-frequency details. In image enhancement, the wavelet transform can be used to remove or reduce noise. This is because noise often appears in high-frequency detail layers. Thresholding can selectively reduce the coefficients in these layers, achieving noise reduction. The wavelet transform allows the algorithm to adaptively adjust enhancement parameters based on image content, meaning that different enhancement strategies can be used for different image types or different image parts. The wavelet transform provides information about the localization of the signal in time and frequency, allowing it to enhance specific image regions without affecting other regions. This is very useful for enhancing specific areas of interest.

[0008] To better combine the CNN's advantages in processing detailed features with the Transformer's advantages in processing global information, this paper proposes a multi-stage RAW image enhancement network that combines CNN and Transformer. Based on the specifics of the task, this paper designs a CNN-based denoising module in the denoising stage and incorporates wavelet information to enhance frequency features. In the color and white balance restoration stage, a Transformer-based correction module is designed, and the signal-to-noise ratio (SNR) map is used to dynamically adjust color and white balance. Summary of the Invention

[0009] In view of this, the present invention aims to overcome the above-mentioned deficiencies in the prior art and proposes a dark light image enhancement method.

[0010] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0011] A first aspect of the present invention provides a low-light image enhancement method, comprising:

[0012] A two-stage U-Net network is constructed. The first-stage U-Net network includes an encoder 1 and a decoder 1, and the second-stage U-Net network includes an encoder 2, a long-short-range interaction module, and a decoder 2. The encoder 1 includes a 4-layer network, which is composed of the designed denoising module and wavelet feature enhancement module stacked 4 times and connected by 3 layers of downsampling modules. The decoder 1 includes a 4-layer network, which is composed of the designed denoising module and wavelet feature enhancement module stacked 4 times and connected by 3 layers of upsampling modules. The encoder 2 includes a 4-layer network with the same structure as the encoder 1. The long-short-range interaction module is located in the hidden layer of the second stage and is used to introduce an SNR map to guide image color and white balance restoration. It is composed of a designed 1-layer Transformer-based correction module (long-range) and a 3-layer CNN detail processing (short-range) module in parallel. The decoder 2 includes a 4-layer network, which is composed of the designed Transformer-based correction module stacked 4 times and connected by 3 layers of upsampling modules.

[0013] Acquire the image to be enhanced and perform amplification processing, and obtain an SNR map of the amplified image to be enhanced;

[0014] The amplified image to be enhanced is input to encoder 1 for denoising, wavelet feature enhancement, downsampling, and output image features.

[0015] The image features obtained after passing through encoder 1 are passed to decoder 1, and decoder 1 is used to perform denoising, wavelet feature enhancement, and upsampling processing;

[0016] The image features after each layer of processing of decoder 1 are fused with the image to be enhanced after the amplification processing and then output to encoder 2. Encoder 2 performs denoising, wavelet feature enhancement, downsampling processing, and outputs image features;

[0017] The image features obtained after encoder 2 are weighted and fused with the signal-to-noise ratio (SNR) map of the unenhanced image. The adjusted features are then input into the hidden layer. These features are then further extracted by the long- and short-range interaction module, outputting optimized image features.

[0018] It is used to restore the image color and white balance using decoder 2, convert the image color channels after upsampling, and output the enhanced image.

[0019] Furthermore, the denoising module is implemented as follows:

[0020] I1=f CCC (f Dc (I in ));

[0021]

[0022] Among them, I1 is the input feature, I out is the output feature, f DC It is a parallel 7×7 and 3×3 convolution operation, f CGC It is a nonlinear operation of convolution and GELU function, represents element-wise addition, ⊙ represents element-wise multiplication, and σ(·) represents the activation function symmetric about the origin to obtain the attention map.

[0023] Furthermore, the wavelet feature enhancement module is implemented as follows:

[0024]

[0025] F in is the input feature, F out is the output feature, DWT and IDWT represent wavelet transform and inverse wavelet transform respectively, f CD is the feature enhancement process, f CGC is the fully connected layer.

[0026] Furthermore, obtaining the SNR map of the amplified image to be enhanced includes:

[0027] For the input image Input in the first stage, calculate its grayscale value I y , and then calculate its SNR map S, the process is as follows:

[0028]

[0029] where f BLUR For Gaussian blur calculation, f ANS To calculate the absolute value, I y The denoising process is performed by a 5×5 Gaussian blur convolution block. N is the estimated noise distribution.

[0030] A second aspect of the present invention provides a low-light image enhancement device, comprising:

[0031] The first processing unit is used to construct a two-stage U-Net network. The first-stage U-Net network includes encoder 1 and decoder 1, and the second-stage U-Net network includes encoder 2, a Transformer correction module, a detail processing module, a long-short-range interaction module, and decoder 2. The encoder 1 includes a 4-layer network, which is composed of the designed denoising module and wavelet feature enhancement module stacked four times and connected by 3 layers of downsampling modules. The decoder 1 includes a 4-layer network, which is composed of the designed denoising module and wavelet feature enhancement module stacked four times and connected by 3 layers of upsampling modules. The encoder 2 includes a 4-layer network with the same structure as the encoder 1. The long-short-range interaction module is located in the hidden layer of the second stage and is used to introduce an SNR map to guide image color and white balance restoration. It is composed of a designed 1-layer Transformer-based correction module (long-range) and a 3-layer CNN detail processing (short-range) module in parallel. The decoder 2 includes a 4-layer network, which is composed of the designed Transformer-based correction module stacked four times and connected by 3 layers of upsampling modules.

[0032] a second processing unit, configured to acquire the image to be enhanced, perform amplification processing on it, and acquire a signal-to-noise ratio (SNR) map of the amplified image to be enhanced;

[0033] The third processing unit is used to input the image to be enhanced after the amplification processing to the encoder 1 for denoising, wavelet feature enhancement, downsampling processing, and output image features.

[0034] The fourth processing unit is used to pass the image features obtained after the encoder 1 to the decoder 1, and use the decoder 1 to perform denoising, wavelet feature enhancement, and upsampling processing;

[0035] A fifth processing unit is configured to fuse the image features processed by each layer of decoder 1 with the image to be enhanced after the amplification process, and output the results to encoder 2. Encoder 2 performs denoising, wavelet feature enhancement, and downsampling processing, and outputs the image features.

[0036] The sixth processing unit performs a weighted fusion of the image features obtained after encoder 2 processing with the signal-to-noise ratio (SNR) map of the unenhanced image. The adjusted features are then input into the hidden layer. These features are then further extracted through the long- and short-range interaction module, outputting optimized image features.

[0037] The seventh processing unit is configured to restore the color and white balance of the image using the decoder 2, perform color channel conversion on the image after upsampling, and output an enhanced image.

[0038] A third aspect of the present invention provides an electronic device, comprising a processor and a memory communicatively connected to the processor and configured to store instructions executable by the processor, wherein the processor is configured to execute the above-mentioned low-light image enhancement method.

[0039] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, which implements the above-mentioned low-light image enhancement method when executed by a processor.

[0040] Compared with the prior art, the low-light image enhancement method described in the present invention has the following advantages:

[0041] The multi-stage RAW low-light image enhancement network proposed in this paper combines the CNN and Transformer architectures. The proposed CNN module with a parameter-free attention mechanism and the Transformer-based global correction module leverage the CNN's advantages in local detail processing and the Transformer's global feature processing to specifically address the issues of noise removal and color and white balance restoration in RAW low-light images.

[0042] Furthermore, the present invention introduces a wavelet feature extraction block that adaptively processes the different frequency details in the RAW image, ensuring that image features at all levels are appropriately enhanced. Furthermore, to dynamically optimize the brightness of different regions, the present invention designs a long- and short-range interaction module guided by the signal-to-noise ratio (SNR) map, further improving image quality and visual quality.

[0043] In summary, the present invention provides a comprehensive and efficient solution to significantly improve the quality of RAW format low-light images. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which constitute part of the present invention, are provided to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are provided to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0045] Figure 1 This is the overall network structure diagram designed by the present invention for RAW format dark light image enhancement;

[0046] Figure 2 This is a schematic diagram of the CNN denoising module based on the parameter-free attention mechanism designed by the present invention;

[0047] Figure 3 Schematic diagram of the wavelet feature enhancement module designed by the present invention that integrates coordinate attention;

[0048] Figure 4Schematic diagram of the Transformer global correction module including wavelet convolution designed by the present invention;

[0049] Figure 5 This is a schematic diagram of the CNN detail processing module designed by the present invention;

[0050] Figure 6 This is an algorithm flow chart of a dark light image enhancement method designed by the present invention. DETAILED DESCRIPTION

[0051] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0052] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0053] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0054] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0055] Example 1:

[0056] like Figure 6 As shown, a dark light image enhancement method includes:

[0057] Construct a two-stage dark light image enhancement network. Figure 1 shown.

[0058] The first stage is the denoising stage, which adopts the U-net architecture. Its encoder and decoder are composed of a CNN-based denoising module and a wavelet transform feature enhancement module stacked alternately four times and connected through upsampling and downsampling layers. The denoising module adopts a multi-scale parallel CNN, which can effectively capture the various noise characteristics of RAW images. In order to further improve the performance of local feature recognition, the present invention adds a parameter-free attention mechanism to the denoising module to optimize the increased computational cost brought by the Transformer. In view of the diversity and complexity of RAW image noise, the present invention adopts a wavelet transform feature enhancement module to dynamically extract high-frequency details and low-frequency structures in the image, so that the model can effectively distinguish between noise and signal.

[0059] The second stage involves white balancing and color restoration. The encoder portion of the U-net used is the same as the first stage, designed to further denoise any remaining noise. The decoder consists of four stacked Transformer rectification modules fused with wavelet convolutions. This not only captures a wider range of global context but also uses wavelet convolutions to enhance focus on low-frequency features rich in color information, improving white balancing and color restoration. Furthermore, a long-short-range interaction module that incorporates a signal-to-noise ratio (SNR) map is added to the hidden layers of the U-net to adaptively adjust the brightness enhancement effect in areas with varying noise levels.

[0060] The input image is enhanced by building a two-stage U-Net network. The specific implementation steps are as follows:

[0061] The first step is to input the initial RAW format dark light image, amplify the input image, and pass it to the encoder part of the first stage.

[0062] After the RAW format dark light image is passed to encoder 1, it first enters the CNN denoising module for denoising. The input features are processed in parallel using 7×7 and 3×3 depthwise separable convolutions to extract different scale information of the image, such as Figure 2 As shown. The features are then processed nonlinearly by an MLP layer consisting of a 1×1 convolution and a GELU function. Then, a parameter-free attention mechanism is introduced, which dynamically generates an attention map of image features by using a convolution kernel combined with an activation function. The present invention uses an origin-symmetric nonlinear activation function to generate an attention map, and identifies key areas based on the output value of the feature map after convolution, thereby effectively reducing the additional parameter burden brought by the traditional attention mechanism. Specifically, the input feature I in After processing by the denoising module, I out The process is expressed as:

[0063] I1=f CGC (f DC (I in));

[0064]

[0065] Among them, I1 is the input feature, I out is the output feature, f DC It is a parallel 7×7 and 3×3 convolution operation, f CGC It is a nonlinear operation of convolution and GELU function, represents element-wise addition, ⊙ represents element-wise multiplication, and σ(·) represents the activation function symmetric about the origin to obtain the attention map.

[0066] Next, we will elaborate on the role of the parameter-free attention mechanism in the denoising module. During model training, the accumulated gradient generated by the denoising module can be expressed as:

[0067]

[0068] Where L is the loss during training, I in is the input image feature, D is all convolution modules in the denoising module, F D It is a convolution operation on image features. The gradient generated for each training. By introducing the parameter-free attention map and superimposing it with the input residual connection, the gradient accumulation can be adjusted to:

[0069]

[0070] Where σ°(·) and σ(·) are the center-symmetric attention maps generated by the activation function. The center-symmetric activation function satisfies the following formula:

[0071] |σ(x)|=|σ(|x|)|=|σ(-x)|

[0072] When extracting feature maps through convolution operations, the absolute value of the gradient represents the size of the feature quantity. Therefore, in areas with rich feature information, σ can obtain larger feature values ​​through larger I1 outputs, and thus the gradient value is also larger. During the training process, the gradient of information-rich areas is amplified, so that the model gives more attention weight to information-rich areas. In addition, by introducing residual connections to avoid information loss problems, the final output of the denoising module is (I1⊕I in )⊙σ(I1). The intensity of each pixel in the generated self-attention map is proportional to the feature recognition ability of the denoising process, helping the denoising module to achieve dynamic denoising based on the noise intensity level.

[0073] The image features after denoising are passed to the wavelet feature enhancement module to enhance the features of the image features at different frequencies. Figure 4As shown in the figure, the wavelet feature enhancement module uses discrete wavelet transform (DWT) to decompose the different frequency components of the signal, thereby enhancing the extraction of frequency features of different intensities. First, for a given image input feature Decomposition is performed using discrete wavelet transform. The decomposition process is as follows:

[0074] F LL ,{F LH ,F HL ,F HH = DWT(F in )

[0075] The decomposed image features are divided into four sub-bands: F LL (low-frequency component, containing color information), (High frequency components, reflecting global structure and detail information) This paper uses the coordinate attention mechanism to perform feature weighting on the four frequency sub-bands obtained after decomposition to generate enhanced wavelet features. This adapts to the different noise distribution and intensity in the image. The coordinate attention mechanism can efficiently identify and enhance the local details of the features after wavelet transformation, while suppressing the noise of the high-frequency details. The present invention applies the inverse wavelet transform to the enhanced wavelet features of each frequency band and reconstructs them into spatial domain weighted features. The conversion from frequency domain information to spatial domain structural information is realized.

[0076] In addition, in order to compensate for the information loss caused by wavelet transform, the input F in As the residual feature and F E Finally, the output is obtained through the MLP layer composed of convolutional layer and GELU function. The action process of the wavelet feature enhancement module is expressed as the following formula:

[0077]

[0078] F in is the input feature, F out is the output feature. DWT and IDWT represent wavelet transform and inverse wavelet transform respectively. CD is the feature enhancement process of coordinate attention, f CGC is a fully connected layer.

[0079] The image is then downsampled to compress the image features, halving the image size and doubling the number of channels. After undergoing four consecutive denoising operations, wavelet feature enhancement, and three downsampling operations, the image features are output by encoder 1.

[0080] In the second step, the image features obtained after encoder 1 are passed to decoder 1. First, the image features are input into the denoising module. After denoising, wavelet feature enhancement is performed. The processing process is the same as in the first step. Then, upsampling is performed to restore the spatial dimensions of the image feature map, doubling the image size and halving the number of channels. This denoising, wavelet feature enhancement, and downsampling process are repeated four times, and finally the image features are output to encoder 2.

[0081] In the third step, the image features output from decoder 1 are fused with the amplified RAW low-light image and then passed to encoder 2. To further process the remaining noise, encoder 2 performs a similar process to encoder 1, performing four consecutive denoising cycles, wavelet feature enhancement, and three downsampling cycles. The processed image features are then output to encoder 2.

[0082] The fourth step is to calculate the grayscale value I of the input image Input in the first stage y , and then calculate the eigenvalue S of its SNR graph. The process is shown in the following formula:

[0083]

[0084] Among them, f BLUR For Gaussian blur calculation, f ABS To calculate the absolute value, I y The denoising process is performed by a 5×5 Gaussian blur convolution block. N is the estimated noise distribution. Then, the output feature f2 after encoder 2 is input into the long-short-range interaction module composed of the global correction module and the detail processing module of 3×3 convolution of the Transformer.

[0085] The following is a detailed description of the working process of the Transformer's global correction module.

[0086] like Figure 4 As shown in the figure, in the multi-head self-attention part of the Transformer's generated attention map, the present invention adopts the transposed attention suitable for image restoration tasks. First, 1x1 convolution is used to aggregate the cross-channel information of each pixel. In order to prevent the model from being over-parameterized and to expand the receptive field of convolution, wavelet convolution is used instead of traditional deep convolution to help better obtain the low-frequency components required for color restoration. The features after convolution processing are reorganized into And get the attention map through the operation of Q and V The calculation formula is:

[0087]

[0088] where γ is for K T ·Q numerical stabilization introduced by the scaling factor.

[0089] Then the transposed attention map A T Perform matrix point multiplication A with V T V and point-wise convolution to get For its feedforward network part, the present invention designs a dual-scale gated feedforward module that also incorporates wavelet convolution to extract multi-range color information, and integrates two different multi-scale convolution paths during the transmission process by utilizing the expansion mechanism.

[0090] The feedforward network propagation process is expressed as follows:

[0091] O a ,O b =f Split (f PC (O1))

[0092] O out =f PU ((f Mish (f WT3 (O b )))⊙(f WT5 (O a )))

[0093] where f PC It is a 1×1 convolution and pixel rearrangement process, f Split Split the input O1 into two equal parts O according to the number of channels a ,O b , f WT3 is a 3×3 wavelet convolution of the feature, f WT5 is a 5×5 wavelet convolution of the feature, f Mish is the Mish activation function, f PU It is pixel inverse rearrangement and 1×1 convolution.

[0094] The entire propagation process of the TWC module can be expressed as:

[0095]

[0096] where f LN is the layer normalization, f MDTA is the module MDTA self-attention calculation, f DSFN It is the calculation process of the dual-scale feedforward network.

[0097] f2 is input into the global correction module to extract its long-distance features and obtain f L ; f2 input stacked three times of the detail processing module extractor short-distance features, the detail processing module obtains fs .like Figure 5 As shown in Figure 3, the detail processing module consists of two simple layers of 3×3 convolution and GELU functions.

[0098] Then, the SNR map S is adjusted to the same size as the two eigenvalues ​​and normalized to obtain S' as the weight to fuse f L and f s The weighted feature F is obtained. The specific process is expressed as follows:

[0099] F=f S ×S'+f L ×(1-S');

[0100] Finally, the designed Transformer correction module is stacked with four layers in the decoder 2 portion of the second-stage U-shaped network to white balance and color restore feature F. Simultaneously, upsampling gradually increases the spatial resolution of the image. After restoration, the RAW image is converted to RGB for output via color channel transformation.

[0101] Example 2:

[0102] A low-light image enhancement device, comprising:

[0103] The first processing unit is used to construct a two-stage U-Net network. The U-Net network of the first stage includes an encoder 1 and a decoder 1. The U-Net network of the second stage includes an encoder 2, a long-short-range interaction module, and a decoder 2. The encoder 1 includes a 4-layer network, which is composed of a denoising module and a wavelet feature enhancement module stacked 4 times and connected by 3 layers of downsampling modules. The decoder 1 includes a 4-layer network, which is composed of a denoising module and a wavelet feature enhancement module stacked 4 times and connected by 3 layers of upsampling modules. The encoder 2 includes a 4-layer network, and its structure is the same as that of the encoder 1. The long-short-range interaction module is located in the hidden layer part of the second stage and is used to introduce an SNR map to guide the restoration of image color and white balance. It is composed of a 1-layer Transformer-based correction module and a 3-layer CNN detail processing module in parallel. The decoder 2 includes a 4-layer network, which is composed of a Transformer-based correction module stacked 4 times and connected by 3 layers of upsampling modules.

[0104] The second processing unit is used to obtain the image to be enhanced, perform amplification processing on it, and obtain an SNR map of the amplified image to be enhanced;

[0105] The third processing unit is used to input the image to be enhanced after the amplification processing to the encoder 1 for denoising, wavelet feature enhancement, downsampling processing, and output image features.

[0106] The fourth processing unit is used to pass the image features obtained after the encoder 1 to the decoder 1, and use the decoder 1 to perform denoising, wavelet feature enhancement, and upsampling processing;

[0107] A fifth processing unit is configured to fuse the image features processed by each layer of decoder 1 with the image to be enhanced after the amplification process, and output the results to encoder 2. Encoder 2 performs denoising, wavelet feature enhancement, and downsampling processing, and outputs the image features.

[0108] The sixth processing unit is used to perform weighted fusion of the image features obtained after processing by the encoder 2 and the signal-to-noise ratio map of the unenhanced image. The adjusted features are then input into the hidden layer. These features are further extracted through the long-short range interaction module and the optimized image features are output;

[0109] The seventh processing unit is configured to restore the color and white balance of the image using the decoder 2, perform color channel conversion on the image after upsampling, and output an enhanced image.

[0110] Example 3:

[0111] An electronic device includes a processor and a memory communicatively connected to the processor and used to store instructions executable by the processor, wherein the processor is used to execute the above-mentioned low-light image enhancement method.

[0112] Example 4:

[0113] A computer-readable storage medium stores a computer program, which implements the above-mentioned low-light image enhancement method when executed by a processor.

[0114] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A low-light image enhancement method, characterized by: include: A two-stage U-Net network is constructed. The U-Net network of the first stage includes an encoder 1 and a decoder 1. The U-Net network of the second stage includes an encoder 2, a long-short-range interaction module, and a decoder 2. The encoder 1 includes a 4-layer network, which is composed of a denoising module and a wavelet feature enhancement module stacked 4 times and connected by 3 layers of downsampling modules. The decoder 1 includes a 4-layer network, which is composed of a denoising module and a wavelet feature enhancement module stacked 4 times and connected by 3 layers of upsampling modules. The encoder 2 includes a 4-layer network, and its structure is the same as that of the encoder 1. The long-short-range interaction module is located in the hidden layer part of the second stage and is used to introduce an SNR map to guide the restoration of image color and white balance. It is composed of a 1-layer Transformer-based correction module and a 3-layer CNN detail processing module in parallel. The decoder 2 includes a 4-layer network, which is composed of a Transformer-based correction module stacked 4 times and connected by 3 layers of upsampling modules. Acquire the image to be enhanced and perform amplification processing, and obtain an SNR map of the amplified image to be enhanced; The amplified image to be enhanced is input to encoder 1 for denoising, wavelet feature enhancement, downsampling, and output image features. The image features obtained after passing through encoder 1 are passed to decoder 1, and decoder 1 is used to perform denoising, wavelet feature enhancement, and upsampling processing; The image features after each layer of processing of decoder 1 are fused with the image to be enhanced after the amplification processing and then output to encoder 2. Encoder 2 performs denoising, wavelet feature enhancement, downsampling processing, and outputs image features; The image features obtained after encoder 2 are weightedly fused with the signal-to-noise ratio map of the unenhanced image. The adjusted features are then input into the hidden layer. These features are further extracted through the long-short range interaction module to output the optimized image features. Decoder 2 is used to restore the image color and white balance. After upsampling, the image is converted into color channels and an enhanced image is output.

2. The low-light image enhancement method according to claim 1, wherein: The denoising module implementation process is as follows: I1=f CGC (f DC (I in )); I out =(I1⊕I in )⊙σ(I1); Among them, I1 is the input feature, I out is the output feature, f DC It is a parallel 7×7 and 3×3 convolution operation, f CGC is a fully connected layer, ⊕ represents element accumulation, ⊙ represents element multiplication, σ(·) represents the activation function symmetric about the origin to obtain the attention map, I in are input image features.

3. The low-light image enhancement method according to claim 1, wherein: The wavelet feature enhancement module implementation process is as follows: F out =f CGC (IDWT(f CD (DWT(F in ))⊕F in ); F in is the input feature, F out is the output feature, DWT and IDWT represent wavelet transform and inverse wavelet transform respectively, f CD is the feature enhancement process, f CGC is a fully connected layer, and ⊕ represents element accumulation.

4. The method for enhancing low-light images according to claim 1, wherein: The obtaining of the SNR graph of the amplified image to be enhanced comprises: For the input image Input in the first stage, calculate its grayscale value I y , and then calculate its SNR map S, the process is as follows: where f BLUR For Gaussian blur calculation, f ABS To calculate the absolute value, I y The denoising process is performed by a 5×5 Gaussian blur convolution block. N is the estimated noise distribution.

5. A low-light image enhancement device, characterized by: include: The first processing unit is used to construct a two-stage U-Net network. The U-Net network of the first stage includes an encoder 1 and a decoder 1. The U-Net network of the second stage includes an encoder 2, a long-short-range interaction module, and a decoder 2. The encoder 1 includes a 4-layer network, which is composed of a denoising module and a wavelet feature enhancement module stacked 4 times and connected by 3 layers of downsampling modules. The decoder 1 includes a 4-layer network, which is composed of a denoising module and a wavelet feature enhancement module stacked 4 times and connected by 3 layers of upsampling modules. The encoder 2 includes a 4-layer network, and its structure is the same as that of the encoder 1. The long-short-range interaction module is located in the hidden layer part of the second stage and is used to introduce an SNR map to guide the restoration of image color and white balance. It is composed of a 1-layer Transformer-based correction module and a 3-layer CNN detail processing module in parallel. The decoder 2 includes a 4-layer network, which is composed of a Transformer-based correction module stacked 4 times and connected by 3 layers of upsampling modules. The second processing unit is used to obtain the image to be enhanced, perform amplification processing on it, and obtain an SNR map of the amplified image to be enhanced; The third processing unit is used to input the image to be enhanced after the amplification processing to the encoder 1 for denoising, wavelet feature enhancement, downsampling processing, and output image features. The fourth processing unit is used to pass the image features obtained after the encoder 1 to the decoder 1, and use the decoder 1 to perform denoising, wavelet feature enhancement, and upsampling processing; A fifth processing unit is configured to fuse the image features processed by each layer of decoder 1 with the image to be enhanced after the amplification process, and output the results to encoder 2. Encoder 2 performs denoising, wavelet feature enhancement, and downsampling processing, and outputs the image features. The sixth processing unit is used to perform weighted fusion of the image features obtained after processing by encoder 2 and the signal-to-noise ratio map of the unenhanced image. The adjusted features are then input into the hidden layer. These features are further extracted through the long-short-range interaction module to output optimized image features; The seventh processing unit is configured to restore the color and white balance of the image using the decoder 2, perform color channel conversion on the image after upsampling, and output an enhanced image.

6. An electronic device comprising a processor and a memory in communication with the processor and configured to store instructions executable by the processor, wherein: The processor is used to execute the low-light image enhancement method described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for enhancing a low-light image according to any one of claims 1 to 4 is implemented.

Images