Two-stage low-illumination image enhancement method based on wavelet transform

Abstract

The invention discloses a two-stage low-illumination image enhancement method based on wavelet transform. According to the method, a two-stage U-shaped low-illumination image enhancement network is utilized, and deep extraction of image features and integration of multi-scale information are carried out on a low-illumination image; inputting the image features into a wavelet domain feature fusion and enhancement module, capturing image features of different scales on a wavelet domain by using discrete wavelets, and enhancing important features by using dynamic gating space attention and lightweight fusion curve attention; a fusion perception function and a lightweight edge loss function are constructed, the fusion perception function considers the pixel-level error and the overall perception quality of the image, and the model is guided to generate a more natural and highly realistic image by combining the pixel loss and the characteristics of the perception network; the lightweight edge loss function focuses on a high-frequency texture structure, and the reconstruction capability of the model on details is further optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of image enhancement, and in particular to a two-stage low-illumination image enhancement method based on wavelet transform. Background Art

[0002] Image quality enhancement in low-light environments is a research hotspot in current computer vision tasks. Image enhancement technology not only has important preprocessing value for advanced visual tasks such as image classification and object detection, but is also regarded as a means of image restoration.

[0003] However, in the task of low-light image enhancement, there are inevitably problems including noise, recovery details, and artifacts. Existing research is committed to solving these problems to enhance visual effects. Traditional low-light image enhancement methods are mainly based on image processing techniques, such as histogram equalization and Retinex algorithm. Although these methods have achieved certain results in enhancement effects, with the rise of deep neural networks, their limitations have gradually emerged. Since deep neural network models have strong nonlinear representation capabilities and can autonomously and accurately learn the potential features in the input data, they have gradually become the mainstream method for low-light image enhancement. For example, the low-light deep autoencoder network named LLNet improves the contrast of low-light images while taking into account the denoising effect. The low-light image enhancement network Retinex-Net based on the Retinex algorithm consists of illumination estimation and reflection layer estimation modules, which can estimate illumination and reflection at the same time. In order to meet the needs of network training, a new dataset LOL with synthetic noise was established. The images of this dataset were obtained by changing the exposure time. In recent years, the U-Net network based on the fully convolutional neural network has shown excellent performance in the field of image restoration. Although U-Net and its variants have potential in the field of low-light image enhancement, further optimization and adaptive adjustment are still needed to cope with different challenges.

[0004] In addition to U-Net-based methods, wavelet transform has been shown to be an effective method for low-light image enhancement. For example, there are schemes that enhance images by combining curvelet transform and iterative projection techniques. Although this method performs well in detail preservation, it sometimes introduces artifacts or ringing effects due to over-enhancement. There are also schemes that use wavelet transform to reduce the spatial dimension of the input during the diffusion process. However, this method that relies on the diffusion model may have limited effectiveness when processing extreme dark light images because noise may dominate in these scenes. There are schemes that use the wavelet transform attention mechanism to detect and suppress noise in low-frequency layers in low-light images. Although this method focuses on low-frequency noise, it may ignore high-frequency details and easily lead to detail loss.

[0005] In addition, given the respective advantages of U-Net and wavelet transform, some studies have also explored methods to combine the two, such as combining U-Net and Haar wavelet pooling technology, which performs well in enhancing image structure and texture, but the use of Haar wavelet pooling may introduce artifacts, especially when processing texture and edge details, which may affect the final enhancement effect. This type of research shows that although wavelet transform has potential in image enhancement, its implementation method needs to be further optimized to balance the relationship between detail enhancement and noise suppression to ensure that the image is natural and realistic.

[0006] In summary, although the existing low-light image enhancement technology has made significant progress, it still has certain limitations in noise suppression and detail preservation, and there are problems of noise and detail loss in the restored images. Summary of the invention

[0007] The present invention provides a two-stage low-illumination image enhancement method based on wavelet transform to solve the problem that the existing low-illumination image enhancement technology still has certain limitations in noise suppression and detail retention, and there is noise and detail loss in the restored image.

[0008] The present invention provides a two-stage low-illumination image enhancement method based on wavelet transform, comprising:

[0009] Step 1: Acquire a low-light image, and use a two-stage U-shaped low-light image enhancement network to perform deep extraction of image features and integration of multi-scale information on the low-light image; the two-stage U-shaped low-light image enhancement network includes a primary restoration stage and a fine enhancement stage, the primary restoration stage includes: a first gradient denoising module, a first shallow feature module, an encoder, a decoder, and a supervised attention module; the fine enhancement stage includes a second gradient denoising module, a second shallow feature module, and a fusion module;

[0010] Step 2: Input the image features into the wavelet domain feature fusion and enhancement module, use discrete wavelets to capture image features of different scales in the wavelet domain, and use dynamic gated spatial attention and lightweight fusion curve attention to enhance important features;

[0011] Step three, construct a fused perception function and a lightweight edge loss function. The fused perception function takes into account the pixel-level error and the overall perceptual quality of the image. By combining the characteristics of the pixel loss and the perception network, it guides the model to generate more natural and realistic images; the lightweight edge loss function focuses on high-frequency texture structures, further optimizing the model's ability to reconstruct details.

[0012] Furthermore, in step 1, the primary restoration stage performs feature extraction, denoising and preliminary enhancement tasks on the input low-light image through the first gradient denoising module and the first shallow feature module; uses the combined encoder and decoder structure to perform feature extraction and preliminary enhancement processing on the image; and compares the feature image processed by the encoder and decoder with the feature X after the extraction and denoising processing. in Jointly input the supervised attention module for feature separation; a part of it is used to output the preliminary enhanced feature image X out , and the other part is used to generate auxiliary features Figure X f These two parts of features are input into the fusion module of the fine enhancement stage to provide feature information for detail recovery; the fine enhancement stage combines the auxiliary features recovered in the primary recovery stage Figure X f With the initial enhanced image X out Fusion; finally, the output of the fine enhancement stage is added pixel by pixel to the input image to generate the final enhanced image.

[0013] Furthermore, in step 1, the first gradient denoising module of the primary restoration stage processes the input image through two residual branches, and then adjusts the original image through the scale parameter to obtain a preliminary denoising and deblurring result;

[0014] The first shallow feature module is used to capture the spatial detail information in the low-light image after the preliminary deblurring and denoising by the first gradient denoising module. The first shallow feature module consists of a convolutional layer, a channel attention module, and a dilated convolutional module. The channel attention module first undergoes a feature processing process, which includes two convolutional layers and a ReLU activation function. Then, the channel attention layer is introduced, which is used to identify and focus on key information and enhance the importance of features. In addition, the module also integrates an enhanced wavelet domain feature fusion module to further optimize the feature representation. Finally, the processed feature map is element-wise summed with the original input feature map to obtain a feature information map as the input of the dilated convolutional layer. Subsequently, the dilated convolutional layer is introduced to expand the receptive field, thereby more effectively capturing spatial information.

[0015] The codec architecture integrates an enhanced wavelet domain feature fusion module, hybrid instance normalization, and residual structure. First, the enhanced wavelet domain feature fusion module performs denoising and optimization processing on the features. Then, hybrid instance normalization can effectively extract multi-scale feature information. Then, the processed features are fused and enhanced with the original input features through the residual structure. Before downsampling, the enhanced features are divided into two processing branches: one branch is fed into the decoder through a skip connection as the fused feature f res ; The other branch is downsampled and used as the input of the decoder;

[0016] The processing flow of the decoder is similar to that of the encoder. It performs upsampling before wavelet domain feature fusion and performs residual fusion with the features returned by the decoder. The encoder and decoder work together to obtain decoding features that integrate multi-scale information. These features are then input into the supervised attention module for further processing.

[0017] The supervised attention module combines the decoded output features of the input with the feature image X after preliminary denoising. in Combine: a part of the decoded output features is passed into the feature image X after convolution operation in And add it to get the spatial attention enhanced feature representation X f The other part is convolved with the spatial attention enhanced feature representation X f Add the features after preliminary processing Figure X out ; X f and X out It is then passed to the subsequent fine enhancement stage for further processing.

[0018] Furthermore, in step 1, the fine enhancement stage retains the feature X of the image’s detailed texture information. f Features processed by the second gradient denoising module and the second shallow feature module Figure X out ′ are input into the fusion module together. The fusion module concatenates the two in the feature dimension and integrates these features through a series of convolutional layers to convert them into high-quality feature maps suitable for image reconstruction. Then, the fused feature map is further converted through another convolutional layer to match the size of the original input image. Finally, the residual of the converted feature map is added to the original image to obtain the final enhanced image.

[0019] Furthermore, in step 2, the wavelet domain feature fusion and enhancement module performs the input feature F in Perform dimensional transformation and divide the input channel into separation and fusion features F containing complete low-frequency information of the image identity And the feature map F input into the discrete wavelet transform inw ; F identity Used as the final fusion recovery part, F inw It is the part to be processed by discrete wavelet transform; after discrete wavelet transform decomposition, F inw Decompose into high-frequency components and low-frequency components; for the high-frequency components, a series of deep encoding is performed using deep separable convolution and parameterized rectified linear units; the processed high-frequency components are learned through dual attention units to obtain deep feature information F inwd , input it into the inverse wavelet transform to obtain the weighted feature Finwt , and the separation fusion feature F containing the complete low-frequency information of the image in the identity domain identity Combine and splice, and use the original low-frequency information retained in the separation and fusion features to supplement the missing low-frequency information, and supplement the missing low-frequency information to obtain the splicing feature F r ; Finally, after convolution and PReLU activation function layer, the residual feature F is obtained out , and perform residual connection with the original input to get the final output.

[0020] Furthermore, the dynamic gated spatial attention first extracts spatial features from the input feature map through maximum pooling and average pooling operations, and then combines the two pooling results of maximum pooling and average pooling. Splice along the channel dimension, calculate the attention weight, and generate the initial spatial attention map F through a 2D 5×5 convolution layer and a Sigmoid activation function sa ; The gated weight is calculated through the gated convolution layer. This layer uses a 1×1 convolution kernel to adjust the feature response of each position. Its output is also processed by the Sigmoid function to generate the gated weight W G ; Set the gate weight W G With the spatial attention map F sa Multiply them together to get the attention weight F after gate modulation G ; Multiply the gated modulated attention weights by the input image F to obtain the feature map F to which the gated spatial attention is applied GP ; Residual connection applies gated spatial attention feature map F GP And the original input feature map F, through the learnable fusion ratio α to achieve adaptive weighted fusion of features, to obtain the processed feature map F P .

[0021] Furthermore, the lightweight fusion curve attention will fuse the feature information X Fe The curve parameters are estimated pixel by pixel through curve parameter calculation to obtain the curve parameters L i (i=1,2,3); The gradient amplification module will fuse the feature information X Fe The input is input into two ReLU activation functions for processing: one part remains unchanged, while the other part is reset to zero, and the difference between the two is calculated; the result of this operation is that some eigenvalues ​​are retained, while the remaining irrelevant eigenvalues ​​are reset to zero, thus forming a dynamically adjusted step curve adjusted feature information F a ; The feature information F processed by the gradient amplification module a and curve parameter L i Combined with the curve parameters, and introducing nonlinear transformation (1-F a); Finally, the adjusted features and feature information processed by the gradient amplification module F a Add and update the features to get the processed feature map F L .

[0022] Furthermore, in step three, the fusion perception function includes the Charbonnier loss function and the VGG perception loss function. The VGG perception loss function uses the pre-trained VGG network to measure the differences between feature layers, thereby capturing the texture and structural information in the image; the weighted sum of the Charbonnier loss function and the VGG perception loss function is used as the total loss function; the lightweight edge loss function uses the standard 3×3 Laplace convolution kernel.

[0023] The present invention has the following beneficial effects: A two-stage low-illumination image enhancement method based on wavelet transform of the present invention adopts a U-shaped network structure, including two stages of preliminary restoration and fine enhancement, and gradually realizes the encoding, decoding and enhancement of features. In order to effectively suppress noise and enhance the high-frequency detail information of the image, the enhanced wavelet domain feature fusion module organically combines discrete wavelet transform, inverse wavelet transform and dual attention mechanism. Furthermore, dynamic gated spatial attention and lightweight fusion curve attention are constructed, and the two work together in the enhanced wavelet domain feature fusion module to process image features in a more refined and flexible way. In addition, the present invention also designs two loss functions: fusion perception loss and lightweight edge loss, which generate more natural and realistic image enhancement effects by comprehensively considering pixel-level errors and overall perception quality. Experimental results show that the proposed method has excellent objective evaluation indicators, not only effectively suppresses noise, but also fully restores image details. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 This is the TSUNet network structure diagram;

[0026] Figure 2 This is the structure diagram of the gradient denoising module;

[0027] Figure 3 This is the structure diagram of the shallow feature module;

[0028] Figure 4 is the codec structure diagram;

[0029] Figure 5 This is the structure diagram of the supervised attention module;

[0030] Figure 6 To enhance the structure of wavelet domain feature fusion module;

[0031] Figure 7 This is a diagram of the dynamic gated spatial attention structure;

[0032] Figure 8 This is a diagram of the lightweight fusion curve attention structure;

[0033] Fig. 9 Provides a visual comparison of the enhancement results of different methods on stadium images of the League of Legends v1 dataset;

[0034] Fig.10 Visual comparison of different methods on locker enhancement results of LOLv1 dataset;

[0035] Fig.11 This is the subjective visual comparison of different methods on the LOLv2-synthetic dataset;

[0036] Fig.12 This is the subjective visual comparison of different methods on the LIME dataset;

[0037] Fig.13 This is the subjective visual comparison of different methods on the MEF dataset. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The technical solutions provided by the embodiments of the present invention are described in detail below in conjunction with the drawings.

[0039] The embodiment of the present invention provides a two-stage low-illumination image enhancement method based on wavelet transform, comprising:

[0040] Step 1: Acquire a low-light image, and use a two-stage U-shaped low-light image enhancement network to perform deep extraction of image features and integration of multi-scale information on the low-light image; the two-stage U-shaped low-light image enhancement network includes a primary restoration stage and a fine enhancement stage, the primary restoration stage includes: a first gradient denoising module, a first shallow feature module, an encoder, a decoder, and a supervised attention module; the fine enhancement stage includes a second gradient denoising module, a second shallow feature module, and a fusion module.

[0041] Step 2: Input the image features into the wavelet domain feature fusion and enhancement module, use discrete wavelets to capture image features of different scales in the wavelet domain, and use dynamic gated spatial attention and lightweight fusion curve attention to enhance important features.

[0042] Among them, the principles of the first gradient denoising module and the second gradient denoising module are the same, and the principles of the first shallow feature module and the second shallow feature module are the same.

[0043] Step three, construct a fused perception function and a lightweight edge loss function. The fused perception function takes into account the pixel-level error and the overall perceptual quality of the image. By combining the characteristics of the pixel loss and the perception network, it guides the model to generate more natural and realistic images; the lightweight edge loss function focuses on high-frequency texture structures, further optimizing the model's ability to reconstruct details.

[0044] In view of the problems of image detail loss, blur and noise that may occur in the existing technology when enhancing low-light images, this paper proposes a two-stage U-shaped low-light image enhancement network, namely TSUNet (Two-Stage WaveletRecovery U-Net). The overall network structure is as follows Figure 1 shown.

[0045] The TSUNet network consists of two stages: the primary restoration stage and the fine enhancement stage. The former is to extract features, denoise and perform preliminary enhancement tasks on the input low-light image through the Gradient Denoising Module (GDM) and the Shallow Features Module (SF). This stage uses the combined codec structure to extract features and perform preliminary enhancement on the image. The feature image processed by the codec is compared with the feature image processed by the extraction and denoising. in The supervised attention module (SA) is input together for feature separation. Part of it is used to output the preliminary enhanced feature image X out , and the other part is used to generate auxiliary features Figure X f , these two parts of features are input into the second stage fusion module (Mergeblock) to provide rich feature information for detail recovery. The latter focuses on image detail enhancement and further reconstructs and refines the features generated in the first stage. Figure X f With the initial enhanced image X outFusion. Finally, the output of the fine enhancement stage is added pixel by pixel to the input image to generate the final enhanced image. By redesigning an improved U-shaped network structure, a two-stage approach is adopted to achieve denoising, deblurring and detail enhancement tasks, gradually improving image quality.

[0046] Primary restoration stage: This stage aims to achieve basic low-light enhancement and provide richer image features for the subsequent fine enhancement stage. The main components include: gradient denoising module (GDM), shallow feature module (SF), encoder (Encoder), decoder (Decoder) and a supervised attention module (SA). The structure of the gradient denoising module (GDM) is as follows Figure 2 As shown in Figure 2, the module processes the input image through two residual branches, and then adjusts the original image through the scale parameter to obtain the preliminary denoising and deblurring results. Specifically, the input image is first input into the first residual branch (ResBlock, RB) to calculate the features of the input image i0, and then the input image i0 is subtracted from the calculated features i r , get the fuzzy noise texture information i j , the information is passed through the second residual branch (ResBlock, RB) again and multiplied by the self-learning parameters to obtain the processed fuzzy noise texture information i c The purpose of this process is to use the residual information to reduce the noise texture information i j The error in the image is reduced, thereby gradually improving the image quality. Among them, the self-learning scale parameter is similar to the learning rate in gradient descent, which is used to control the step size of the update. Finally, the input image i0 is subtracted from the processed noise texture information i c , perform preliminary denoising on the original image.

[0047] At the same time, the present invention designs a new shallow feature module (SF), whose structure is as follows: Figure 3 As shown in the figure. The purpose of this module is to capture the spatial detail information in the low-light image after the preliminary deblurring and denoising by the GDM module. It consists of a convolutional layer (Conv3×3), a channel attention block (CABlock) and a dilated convolution module (Dilated Conv).

[0048] Specifically, the shallow feature module is defined as a serialized module. Each module starts with a standard convolutional layer (Conv3×3), and then these features are fed into the channel attention module (CABlock). This CABlock design allows the local features of the image to be effectively captured without significantly increasing the number of parameters. At the same time, it provides high-quality input features for deeper encoder-decoder networks.

[0049] Among them, the channel attention module (CABlock) first undergoes a sophisticated feature processing process. The process includes two layers of convolutional layers (Conv) and a layer of ReLU activation function to ensure accurate feature extraction. Then, the channel attention layer (CALayer) is introduced, which is used to identify and focus on key information and enhance the importance of features. In addition, the module also integrates an enhanced wavelet-domain feature fusion module (EnhancedWavelet-Domain Feature Fusion Module, EWFM) to further optimize the feature representation. Finally, the processed feature map is element-wise summed with the original input feature map to obtain the feature information map as the input of the dilated convolution layer (Dilated Conv). The dilated convolution layer (DilatedConv) is then introduced to expand the receptive field, thereby more effectively capturing spatial information.

[0050] The structure of the codec is as follows Figure 4 As shown in the figure. The codec architecture integrates modules such as enhanced wavelet feature fusion module (EWFM), hybrid instance normalization (HIN) and residual structure (RB). First, EWFM denoises and optimizes the features. Then, HIN can effectively extract multi-scale feature information. Then, through the residual connection RB, the processed features are fused and enhanced with the original input features. Before downsampling (Down), the enhanced features are divided into two processing branches: one branch is sent to the decoder through a skip connection as the fused feature f res ; The other branch is downsampled and used as the input of the decoder.

[0051] The processing flow of the decoder is similar to that of the encoder. It performs an upsampling operation before performing wavelet domain feature fusion, and performs residual fusion with the features returned by the decoder. The encoder and decoder work together to obtain decoding features that integrate multi-scale information. These features are then input into the supervised attention module (SA) for further processing. Its structure is as follows Figure 5 This module combines the input decoded output features with the feature image X after preliminary denoising inCombine: A part of the decoded output features is passed into the feature image X after a convolution operation (Conv) in And add it to get the spatial attention enhanced feature representation X f The other part is convolutionally enhanced with the spatial attention feature representation X f Add the features after preliminary processing Figure X out The two outputs (X f and X out ) will be passed to the subsequent fine enhancement stage for further processing.

[0052] Fine enhancement stage: As a high-quality reconstruction stage of the image, the fine enhancement stage fuses the feature map output by the primary restoration stage with the detailed texture information feature information of the image, and finally reconstructs an enhanced high-quality output image. This stage is mainly composed of the gradient denoising module (GDM), the shallow feature module (SF), and the fusion module (Mergeblock). The main function of the fine enhancement stage is to preserve the feature X of the detailed texture information of the image. f Characteristics after GDM and SF treatment Figure X out ′ are input into the fusion module together, which concatenates the two in the feature dimension and integrates these features through a series of convolutional layers to convert them into high-quality feature maps suitable for image reconstruction. Then, another convolutional layer (Conv) further transforms the fused feature map to match the size of the original input image. Finally, the residual of the transformed feature map is added to the original image to obtain the final enhanced image.

[0053] Enhanced Wavelet-Domain Feature Fusion Module: The Enhanced Wavelet-Domain Feature Fusion Module (EWFM) is designed to effectively improve the quality of low-light images, especially in terms of noise processing and detail retention. This module combines discrete wavelet transform (DWT) and inverse wavelet transform (IWT), and uses a dual attention mechanism and feature separation and fusion technology. It not only fully taps the potential of the image in the wavelet domain, but also processes the high-frequency information of the image more carefully through the dual attention mechanism, thereby significantly enhancing the image's denoising, edge sharpening and texture restoration capabilities. The structure of this module is as follows: Figure 6 shown.

[0054] First, we input the feature F inPerform dimensional transformation (Resize) and divide the input channel into separation and fusion features F containing complete low-frequency information of the image identity And the feature map F input into DWT inw The former is used as the final fusion recovery part, while the latter is the part to be processed by discrete wavelet transform (DWT). After discrete wavelet transform (DWT) decomposition, F inw Decompose into high-frequency components (mainly including high-frequency information such as noise, edges, textures, etc.) and low-frequency components. For high-frequency components, a series of deep encoding is performed using depthwise separable convolution (DSC) and parametric rectified linear unit (PReLU). The processed high-frequency components are learned through dual attention units to obtain deep feature information F inwd , input it into the inverse wavelet transform (IWT) to obtain the weighted feature F inwt , and the separation and fusion features (F identity) Combine and splice, and use the original low-frequency information retained in the separation and fusion features to supplement the missing low-frequency information, and supplement the missing low-frequency information to obtain the splicing feature F r Finally, after convolution (Conv1×1) and PReLU activation function layer, the residual feature F is obtained. out , and perform residual connection with the original input (Input) to obtain the final output (Output).

[0055] Secondly, in order to more effectively process high-frequency information in the image, the present invention designs two attention mechanisms: Dynamic Gated Spatial Attention (DGSA) and Lightweight Fusion Curve Attention (LFCA). Together, they constitute a dual attention unit. The dual attention unit improves the model's enhancement capabilities in two different dimensions, thereby increasing the dynamic range of the image and making the details in the dark areas more obvious while avoiding overexposure in the bright areas. In addition, the unit can also fine-tune the local area of ​​the image to restore more details and texture information, significantly improving image quality.

[0056] The design of the dynamic gated spatial attention mechanism (DGSA) focuses on the key spatial regions of the image by dynamically adjusting the learned weights. The mechanism uses a gating mechanism to control the flow of information to ensure that only the most relevant spatial information is enhanced. The advantage of this mechanism is that it can dynamically adjust the focus of attention according to different image contents, thereby more effectively processing the spatial details and texture information in the image.

[0057] The design of the lightweight fusion curve attention mechanism (LFCA) focuses on the channel dimension and optimizes the color performance and contrast of the image by adjusting the weights between different channels. This mechanism reduces the computational burden by designing a lightweight network, and uses a curve fusion strategy to fine-tune the channel weights, which not only improves the efficiency of the model, but also ensures the restoration of color and contrast, making the image details richer and clearer. The dynamic gated spatial attention mechanism (DGSA) and the lightweight fusion curve attention mechanism (LFCA) work together to improve the image restoration effect of the wavelet domain feature fusion module. The two attention mechanisms will be introduced in detail below.

[0058] Dynamic Gated Spatial Attention: In existing low-light image enhancement research, the traditional attention mechanism mainly enhances the expressiveness of features by focusing on the salient areas of the image. However, when processing images with complex lighting and textures, this method often fails to fully capture all key details. To this end, the present invention designs an attention module containing a gated convolutional layer, namely dynamic gated spatial attention (DGSA), which aims to enhance the expressiveness of attention by introducing an additional gating mechanism and combining residual fusion connections with learnable fusion coefficients. This can further refine and adjust the attention allocation and improve the model's ability to restore low-light image details. Its structure is as follows Figure 7 shown.

[0059] The DGSA structure consists of two main parts: basic spatial attention calculation and gate modulation mechanism. First, the spatial features are extracted from the input feature map through the maximum pooling (MaxPool) and average pooling (AvgPool) operations. These two pooling operations can capture the extreme features and average response of the image respectively, which helps the model understand the global content and local details of the image. Then, the two pooling results of MaxPool and AvgPool are combined. Splice along the channel dimension and generate the initial spatial attention map F through attention weight calculation (AWC), that is, through a 2D 5×5 convolution layer (Conv5×5) and Sigmoid activation function sa , the process is shown in formula (1):

[0060]

[0061] Among them, σ represents the Sigmoid activation function, represents the feature map obtained after the average pooling operation, It represents the feature map obtained after the maximum pooling operation, Cat represents the feature concatenation operation, and Conv5 represents a 5×5 convolution.

[0062] In order to further improve the adaptability and accuracy of attention, a gated convolution layer is designed for gate weight calculation (GWC). This layer uses a 1×1 convolution kernel (Conv 1×1 ) to adjust the feature response of each position, and its output is also processed by the Sigmoid function to generate the gating weight W G The process is shown in formula (2):

[0063]

[0064] Among them, Conv1 represents a 1×1 convolution.

[0065] The gate weight W G With the spatial attention map F sa Multiply them together to get the attention weight F after gate modulation G The process is shown in formula (3):

[0066] F G =W G ×F SA (3)

[0067] Gated modulation enables the model to adjust the feature strength of the image more accurately by finely controlling the importance of each position. Multiplying the gated modulated attention weights with the input image F yields the feature map F to which gated spatial attention is applied. GP , the process is shown in formula (4):

[0068] F GP =F G ×F (4)

[0069] In addition, in order to effectively restore image details and enhance its visual effect, a residual connection mechanism with self-learning parameters is designed to provide higher adaptability and flexibility, thereby optimizing the dynamic weight adjustment capability during feature fusion. The residual connection applies the feature map F of the gated spatial attention. GP And the original input feature map F, through the learnable fusion ratio α to achieve adaptive weighted fusion of features, to obtain the processed feature map F P This design not only helps to preserve the original information and avoid information loss caused by over-modulation, but also can dynamically adjust the degree of feature fusion according to different image contents. Especially when processing low-light images with different lighting conditions, it can effectively restore image details and enhance its visual effect. The dynamic gated spatial attention is shown in formula (5).

[0070] F P=F GP +α×F (5)

[0071] Lightweight Fusion Curve Attention: The Dynamic Gated Spatial Attention (DGSA) mechanism optimizes the model's ability to capture global structures, mitigates detail loss due to insufficient local illumination, and effectively reduces the impact of noise background on image quality. Considering that if only DGSA is used, the model may not be able to effectively understand local illumination; at the same time, in order to better extract information and suppress noise, a Lightweight Fusion Curve Attention (LFCA) is designed in combination with curve estimation, which is more suitable for low-light image enhancement.

[0072] The lightweight fusion curve attention structure is as follows Figure 8 Its core function is to enhance the quality of low-light images. Through Curve Parameter Calculation (CPC), Gradient Amplifier (GA) and Multi Scale Feature Fusion (MSF), it effectively extracts and fuses multi-level features of the image, thereby restoring the details and colors of the image, reducing noise and enhancing contrast.

[0073] Specifically, in order to improve the image enhancement performance of the network, multi-scale feature fusion (MSF) is used to extract image features of different sizes (F S×S , (S = 1, 2, 3)), and the captured image features of different sizes are added and merged to obtain the fusion feature information X Fe . They are respectively passed to the curve parameter calculation module (CWC) and the gradient amplification module (GA). The process is shown in formula (6).

[0074]

[0075] Where S represents the convolution kernel of size S×S in multi-scale feature fusion; F s Indicates that features are extracted using a convolution kernel of size S×S; x represents the input feature image.

[0076] Fusion feature information X Fe The curve parameters L are obtained by estimating the curve parameters pixel by pixel through curve parameter calculation (CPC). i

[0077] (i=1,2,3). In addition, in order to reduce the computational overhead, the present invention introduces separable convolution (DSC) in the curve parameter calculation module, decomposing the standard convolution into two steps: depthwise convolution and pointwise convolution. This decomposition greatly reduces the number of model parameters and computational complexity, but still retains sufficient feature extraction capabilities, which is beneficial to subsequent processing such as denoising and detail recovery. The expression is:

[0078] L n =σ(Conv1(θ(DSC(θ(DSC(X Fe )))))) (7)

[0079] Where DSC represents depth-wise separable convolution; θ represents the ReLU activation function.

[0080] This design enhances the efficiency of the lightweight fusion curve attention mechanism in low-light image enhancement tasks and improves the processing speed, making it more suitable for real-time or resource-constrained application scenarios.

[0081] In the gradient amplification module (GA), the fusion feature information X Fe The input is input into two ReLU activation functions for processing: one part remains unchanged, while the other part is reset to zero, and the difference between the two is calculated. The result of this operation is that some eigenvalues ​​are retained, while the remaining irrelevant eigenvalues ​​are reset to zero, thus forming a dynamically adjusted step curve adjusted feature information F a The process is shown in formula (8):

[0082] F a =θ(X Fe )-θ(X Fe -1) (8)

[0083] This processing forms a nonlinear "window" that highlights important features in the interval [0,1], which helps the model focus more on these emphasized feature areas. When this curve is multiplied by the attention weight, the attention weight can be automatically assigned, so that the model pays more attention to important areas in the image and ignores irrelevant areas. Compared with manually designed attention weight shapes, this mechanism is more adaptive and flexible, and can better adapt to image features and task requirements in different scenarios.

[0084] Finally, the feature information F processed by the gradient amplification module a and curve parameter L i Combined with the curve parameters, and introducing nonlinear transformation (1-F a). This nonlinear transformation helps to suppress growth when the eigenvalue is close to 1 and promote growth when the eigenvalue is close to 0, thus realizing dynamic adjustment of the features. Finally, the adjusted features and the feature information processed by the gradient amplification module F a Add and update the features to get the processed feature map F L The lightweight fusion curve attention is shown in formula (9):

[0085]

[0086] Where i represents the curve parameter of the i-th pixel set.

[0087] This process enables the model to dynamically adjust its feature expression according to the content of each pixel, responding to different image features in a nonlinear way. This feature adjustment mechanism improves the model's adaptability and response accuracy to various structures in the image, especially when dealing with complex or changing visual scenes.

[0088] Fusion perceptual loss function: The traditional L1 loss function only focuses on the Euclidean distance between individual pixels, which cannot fully evaluate the similarity between image structure information and high-level semantics, resulting in the lack of realism in the generated image. To solve this problem, the present invention proposes a fusion perceptual loss function that combines the advantages of the Charbonnier loss function and the VGG perceptual loss function. This loss function comprehensively considers global features and local details, and is supplemented by an improved edge loss, achieving a balance between image detail information and overall visual perception.

[0089] Charbonnier loss is a smooth loss function. Compared with L1 loss, it has better differentiability at x=0, which helps to optimize the stability of the process. This loss is approximately equal to L2 loss when x is close to 0 and approximately equal to L1 loss when x is larger by adjusting the parameter ∈.

[0090] Charbonnier loss is used to calculate the single-pixel similarity between the generated image X and the labeled image Y, and the expression is:

[0091]

[0092] Among them, ∈ is a constant used to ensure numerical stability. This loss function usually dominates the total loss function.

[0093] In order to make the generated image more consistent with human vision, the present invention uses the pre-trained VGG model to extract high-level semantic features of low-light images and enhanced images to calculate the similarity between the two. This perceptual loss is closer to the processing mechanism of the human visual system and can effectively capture various information of the image from low-level details to high-level structures, which helps to restore the details of the image. The VGG perceptual loss function formula is as follows:

[0094]

[0095] Where j represents the jth layer of the network, φ(.) represents the VGG pre-trained network, and C j H j W j Indicates the size of the j-th layer feature map, represents the generated image, and y represents the labeled image.

[0096] In this study, the proposed fusion perceptual loss function aims to improve the visual effect and quantitative performance of the image enhancement model by combining VGG perceptual loss and Charbonnier loss. VGG perceptual loss uses the pre-trained VGG network to measure the differences between feature layers to capture the texture and structural information in the image. The Char-bonnier loss is a smooth L1 loss that helps reduce pixel-level errors, making the enhanced image more natural in brightness and contrast. By combining these two loss functions, the fusion perceptual loss function not only enhances the local detail expression of the image, but also optimizes the overall visual perception quality, making the model more robust and adaptable when processing low-light images. The weighted sum of the above two loss functions is used as the total loss function, and the expression is:

[0097] L total =α VGG L VGG +α Char L Char (12)

[0098] Among them, α VGG and α Char is the balance coefficient. In this experiment, they are set to α VGG =1.3 and α Char = 1.7 to help the model better preserve the structural information of the image when generating images while reducing noise and unnatural artifacts.

[0099] Lightweight edge loss: In the task of low-light image enhancement, detail recovery is particularly important, especially in the accurate reconstruction of high-frequency texture structures. In order to more effectively guide the detail recovery of low-light images, this study designed a new edge loss function specifically for the needs of this application scenario. In conventional schemes, Gaussian kernels are usually used to extract the edge information of the image, and then the loss is calculated by comparing the edge differences between the original image and the enhanced image. This process involves complex downsampling and upsampling operations, which not only increases the computational cost, but may also affect the accuracy of edge detection due to the cross-mixing of information between different color channels.

[0100] In order to overcome the limitations of conventional methods and better adapt to low-light image enhancement tasks, this paper proposes an improved edge loss function, namely lightweight edge loss. The design goal of this loss function is to reduce computational complexity and improve the accuracy of edge detection. The core of this loss function is to directly use edge information instead of indirectly obtaining it through Gaussian blur. The specific formula is as follows:

[0101]

[0102] Among them, because the main body still uses Charbonnier loss, X and Y represent the generated image and the labeled image respectively, ε is the adjustment parameter, Δ LO Denotes Laplace kernel processing.

[0103] The lightweight edge loss proposed in the present invention adopts a standard 3×3 Laplace convolution kernel, avoiding the traditional Gaussian convolution preprocessing and sampling operations, thereby reducing calculation errors and improving operation efficiency. At the same time, the loss function adopts a multi-channel processing mechanism to ensure that each color channel performs edge detection independently, effectively retaining edge detail information and improving the accuracy of edge detection. In addition, the loss function reduces the computational complexity, improves the algorithm operation efficiency, and reduces the demand for computing resources, which is more applicable in resource-constrained environments.

[0104] Experimental results and analysis:

[0105] Experimental details: This paper uses NVIDIA GeForce RTX 4090 GPU and PyTorch deep learning framework to train 300 rounds on the LOLv1 benchmark dataset. During training, low-light images are randomly cropped into 256*256 image blocks as the input of the model, and the batch size is set to 2. The experiment uses the Adam optimizer, and the initial learning rate is set to 1e-4, which is then decayed to 1e-6 using the cosine annealing algorithm. The test phase uses a 256×256 test set image for evaluation.

[0106] Dataset: The present invention is tested on the real datasets LOLv1, LSRW, LOLv2-real, LOLv2-synthetic and the reference-free datasets DICM, LIME, and MEF to verify the effectiveness of the method proposed in the present invention. Among them, LOLv1 contains 500 pairs of images, of which 485 pairs of images are used as training images and the remaining 15 pairs of images are used as test images. The LSRW dataset contains two sub-datasets: 2480 pairs of images taken with a Huawei mobile phone, of which 2450 pairs are used for training and 30 pairs are used for testing; 3170 pairs of images taken with a Nikon camera, of which 3140 pairs are used for training and 30 pairs are used for testing. The LOLv2 dataset includes two subsets, namely LOLv2-real and LOLv2-synthetic. The LOLv2-real subset is captured by adjusting ISO and exposure time in a real scene. The present invention uses 689 pairs of images for training and 100 pairs of images for testing. In the LOLv2-synthetic subset, low-light images are synthesized from RAW images by analyzing the illumination distribution of low-light images. This subset contains 1,000 pairs of low-light / normal images, 900 pairs of images are selected for training, and 100 pairs of images are used for testing. The unsupervised datasets DICM, LIME, and MEF contain 69, 10, and 44 low-light images collected by commercial digital cameras, respectively, all of which are used for testing.

[0107] Method comparison:

[0108] Objective evaluation: The proposed method is compared with current advanced methods, including unsupervised learning methods: EnlightenGAN and SCI; zero-shot learning method: ZeroDCE++; supervised learning methods: IATLLFormer, CF-Uformer, R2RNet, MIRNet, Instruct-IR, PPFormer and Entropy-SDE; wavelet transform-based methods: CFWD and HWMNet.

[0109] In the experiment, Peak Signal to Noise Ratio (PSNR), Structural Similarity Index (SSIM) and Learned Perceptual Image Patch Similarity (LPIPS) were used as objective evaluation indicators.

[0110] Table 1 shows the comparative experimental results of the proposed method and the existing advanced methods on the LOLv1 and LSRW datasets, where the best and suboptimal results are indicated by bold and underline respectively. The experimental results show that the proposed method is superior to all the compared methods in terms of PSNR, SSIM and LPIPS.

[0111] Table 1 Comparison of results on the LOLv1 dataset and the LSRW dataset

[0112]

[0113] At the same time, in order to demonstrate the stability of the model of the present invention, comparative experiments were conducted on the LOLv2-real and LOLv2-synthetic datasets, and the experimental results are shown in Table 2. As can be seen from Table 2, the method proposed in the present invention also achieved the best results in the three indicators of PSNR, SSIM, and LPIPS.

[0114] Table 2 Comparison of results on the LOLv2-real dataset and the LOLv2-synthetic dataset

[0115]

[0116]

[0117] In order to deeply understand the superior performance of this method in low-light image enhancement, we conducted a detailed analysis of the experimental results. Thanks to the enhanced wavelet domain feature fusion module designed by the present invention and the preliminary restoration stage in the two-stage network, the present method has achieved leading index scores in the four datasets of LOLv1, LSRW, LOLv2-real, and LOL-synthetic in terms of the two distortion evaluation indicators PSNR and SSIM. This shows that the image enhanced by the method proposed in the present invention has more high-frequency details and structures, providing higher quality image restoration results.

[0118] In addition, LPIPS is an indicator to measure the perceived quality of an image, and the lower its value, the closer the image is visually to the original image. Through comparative experiments, the method of the present invention is significantly better than the existing technology in the LPIPS indicator, thanks to the fusion perception loss function designed in the model of the present invention and the fine restoration stage in the two-stage network.

[0119] Furthermore, the experimental results of the LOLv2 dataset in Table 2 further verify the robustness of the method of the present invention. Whether on the LOLv2-real dataset of the real scene or the LOLv2-synthetic dataset of the synthetic scene, the proposed model has achieved the best in the three evaluation indicators of PSNR, SSIM, and LPIPS, which shows that the model proposed in the present invention can adapt to different environments and conditions and maintain stable performance in both real and synthetic scenes.

[0120] Finally, in order to better evaluate the effectiveness and generalization ability of the model proposed in this paper, comparative experiments were conducted using three reference-free datasets: DICM, LIME, and MEF. Since these three datasets have no reference images, the reference-free evaluation index (Natural Image Quality Evaluator, NIQE) was used to objectively evaluate the restoration effect of the model. The experimental results are shown in Table 3. The experimental results show that the proposed method achieves the best results in the LIME and MEF datasets, and is also superior to other comparison methods in terms of the average performance of the three datasets.

[0121] Table 3 Comparison of NIQE scores on DICM, LIME, and MEF datasets

[0122]

[0123]

[0124] The above experimental results show that the method proposed in this paper can not only effectively enhance image quality, but also has significant advantages in maintaining image structure and perceptual quality, and has good robustness in different scenes and conditions. These advantages are achieved thanks to the innovation of the method in algorithm design and model optimization, which enables it to maintain stable performance in different data sets and scenes.

[0125] MIRNet over-performs in brightness enhancement, resulting in loss of details and local blur. IAT performs well in color restoration, with clearer seat details, but there is also a lot of noise and point blur. In contrast, HWMNet and PPFormer are relatively superior in seat detail and color restoration, but a checkerboard effect appears when processing deep details, the edges of details also appear blurred, and the color saturation is insufficient. The method of the present invention performs outstandingly in optimizing detail preservation, more accurately restoring the details and contours of the seat, while effectively retaining the color information of the original image and greatly suppressing the blur phenomenon. These results show that the method of the present invention has potential in maintaining image quality, especially in detail protection and color authenticity.

[0126] Fig.10The enhancement results of different methods on locker images are shown. Fig.10 (a) is a low-light image. Fig.10 (b) is ZeroDCE, Fig.10 (c) is EnlightenGAN, Fig.10 (d) is SCI, Fig.10 (e) is MIRNET, Fig.10 (f) is HWMNet, Fig.10 (g) is IAT, Fig.10 (h) is PPFormer, Fig.10 (i) is the method of the present invention, Fig.10 (j) is the reference image.

[0127] It can be found that the effects of ZeroDCE, EnlightenGAN and SCI methods in improving brightness are limited, which is specifically manifested in the disappearance of the edge of the paper box and the complete invisibility of the box pattern; at the same time, Enlighten-GAN has a serious color cast problem. Although the brightness improvement of the MIRNet and IAT methods is normal, there are too many edge noise points, the details are not clear, and the pattern of the box becomes blurred due to the noise. Compared with other methods, HWMNet and PPFormer have better restoration effects, but there are still problems of unclear patterns and boxes and color cast. The method proposed in the present invention not only successfully restores the edge details of the box on the left, but also effectively suppresses noise and blur, while clearly restoring the pattern of the box on the right, and more accurately restoring the color of the box. These results further demonstrate the advantages of the method of the present invention in detail protection and color authenticity.

[0128] In order to comprehensively test the robustness of the method, we can use images from different datasets for testing. We randomly selected 3 images from the LOLv2-synthetic dataset and 1 image each from the LIME and MEF datasets. The enhancement effects are shown in Figure 2. Figures 11 to 13 shown. Fig.11 (a) is a low-light image. Fig.11 (b) is EnlightenGAN, Fig.11 (c) is R2R, Fig.11 (d) is MIRNet, Fig.11 (e) is SCI, Fig.11 (f) is HWMNet, Fig.11 (g) is CF-UFormer, Fig.11 (h) is the method of the present invention, Fig.11 (i) is the reference image.

[0129] Fig.12 (a) is a low-light image. Fig.12(b) is EnlightenGAN, Fig.12 (c) is R2R, Fig.12 (d) is MIRNet, Fig.12 (e) is IAT, Fig.12 (f) is HWMNet, Fig.12 (g) is ZeroDCE, Fig.12 (h) is SCI, Fig.12 (i) is PPFormer, Fig.12 (j) is the method of the present invention.

[0130] Fig.13 (a) is a low-light image. Fig.13 (b) is EnlightenGAN, Fig.13 (c) is R2R, Fig.13 (d) is MIRNet, Fig.13 (e) is IAT, Fig.13 (f) is HWMNet, Fig.13 (g) is ZeroDCE, Fig.13 (h) is SCI, Fig.13 (i) is PPFormer, Fig.13 (j) is the method of the present invention.

[0131] By comparison, we can find that other methods have certain problems in brightness enhancement, detail restoration and color restoration, while the method of the present invention performs better. While ensuring the image restoration brightness and color, it effectively suppresses noise amplification and successfully restores detail information, generating more realistic and natural results. This further proves that the TSUNet network proposed in the present invention has good low-light image restoration capabilities.

[0132] Ablation experiment: In order to verify the effectiveness of the sub-modules, ablation experiments were performed on the enhanced wavelet domain feature fusion module, the fusion perception loss function and the lightweight edge loss function, and the two-stage low-light image enhancement network. By testing these key sub-modules one by one, their effectiveness for the proposed low-light image enhancement network can be verified.

[0133] Fig. 9 The enhancement results of different methods on stadium images are shown. Fig. 9 (a) is a low-light image. Fig. 9 (b) is ZeroDCE, Fig. 9 (c) is EnlightenGAN, Fig. 9 (d) is SCI, Fig. 9 (e) is MIRNET, Fig. 9 (f) is HWMNet, Fig. 9 (g) is IAT, Fig. 9 (h) is PPFormer, Fig. 9 (i) is the method of the present invention, Fig. 9 (j) is the reference image.

[0134] By comparing and analyzing the reference images, we can find that although the ZeroDCE++ and SCI methods improve the image brightness, they perform poorly in recovering key details such as seats, with serious detail loss, accompanied by obvious noise and color deviation. These problems may have a negative impact on subsequent advanced computer vision tasks. Although EnlightenGAN has improved the overall brightness, the color is yellowish and has a large color deviation. It is also accompanied by detail loss and noise problems, which will seriously affect the performance of subsequent visual tasks.

[0135] Enhanced wavelet feature fusion module: Experiments have shown that the enhanced wavelet feature fusion module (EWFM) plays a positive role in noise control and detail preservation in the restoration of low-light images. In order to demonstrate the effects of EWFM, dual attention unit (DGSA+LFCA), dynamic gated spatial attention (DGSA) and lightweight fusion curve attention (LFCA) on image restoration, corresponding ablation experiments were designed. Through these targeted ablation experiments, the specific role and contribution of each component in low-light image restoration can be more clearly demonstrated, providing an experimental basis for the optimization of the overall network architecture.

[0136] The data in Table 4 clearly show the impact of each component on the model performance. First, removing the enhanced wavelet feature fusion module (EWFM) will cause a significant decrease in PSNR and SSIM indicators, which shows that EWFM plays a key role in the low-light image enhancement process, especially in noise control and detail preservation. The multi-scale analysis capability of wavelet transform enables EWFM to effectively process image information of different frequencies, thereby balancing noise suppression and detail enhancement during the enhancement process.

[0137] Table 4 Effect of enhanced wavelet domain feature fusion module on model performance

[0138]

[0139] Secondly, the results in Table 4 also reveal the impact of the dual attention unit (DGSA+LFCA) on model performance. The combination of dynamic gated spatial attention (DGSA) and lightweight fusion curve attention (LFCA) provides the model with a more sophisticated feature selection and fusion mechanism. From the results of the ablation experiment, removing either the DGSA or LFCA module will have a negative impact on the performance of the model, which confirms the complementarity of these two attention mechanisms in improving image quality. DGSA combines the important spatial region focusing ability of spatial attention and enhances the model's capture of key information by dynamically adjusting the importance weights of feature maps. LFCA uses the curve attention mechanism to optimize the feature fusion process and further improve the detail expression of the image.

[0140] Fusion perceptual loss function and lightweight edge loss function: In order to verify the impact of the two loss functions designed by the present invention on the model performance, namely: fusion perceptual loss function (including VGG perceptual loss VGGLoss and Charbonnier loss CharLoss) and lightweight edge loss function EdgeLoss, ablation experiments were performed on them, and the results are shown in Table 5. The experimental results show that the fusion perceptual loss function plays a significant positive role in improving the performance of the model. Specifically, the VGG perceptual loss effectively captures the high-level semantic information of the image by extracting features using the pre-trained VGG network, which helps the model maintain structural consistency and natural texture during the enhancement process. On the other hand, Charbonnier loss, as a smooth L1 loss, reduces the sensitivity to outliers, thereby more accurately restoring image details under low illumination conditions.

[0141] Table 5 Effect of different loss functions on model performance

[0142]

[0143] In the ablation experiment, we observed that when only VGGLoss was used, the model improved in terms of structural similarity from the perspective of SSIM score, but it was insufficient in terms of detail recovery from the perspective of PSNR score. On the contrary, when CharLoss was used alone, the model performed better in terms of detail recovery from the perspective of PSNR score, but the lower SSIM score indicated that the overall structure preservation was lacking when CharLoss was used alone. This shows that the combination of these two loss functions can complement each other to achieve a more comprehensive image quality improvement. In addition, the introduction of the lightweight edge loss function (EdgeLoss) further enhances the model's ability to perceive image edges. The ablation experiment results show that after removing EdgeLoss, although the overall performance did not decrease much, both detail recovery and structure recovery were affected to a certain extent from the perspective of PSNR and SSIM scores. This shows that EdgeLoss plays an important role in maintaining the clarity of image restoration, especially under low illumination conditions, where accurate recovery of image detail information is crucial to improving image quality.

[0144] Two-stage low-light image enhancement network: In order to verify the impact of the improved parts in the two-stage low-light image enhancement network on the model performance, ablation experiments were conducted on the shallow feature module (SF), the fine enhancement stage, the improved encoder-decoder, and the supervised attention module (SA). The experimental results are shown in Table 6. The experimental results show that these improved parts have a significant positive impact on the model performance.

[0145] Table 6 Impact of different modules on model performance

[0146]

[0147] Specifically, SF is responsible for extracting the primary features of the image, laying the foundation for the subsequent enhancement process. The ablation experiment results show that when SF is removed, the model performance drops significantly, specifically, the PSNR value drops to 23.22 and the SSIM value drops to 0.841. Compared with the baseline model of TSUNet (PSNR is 24.34 and SSIM is 0.861), this significant performance drop indicates that SF plays a key role in preserving the basic information of the image and providing high-quality input for subsequent processing. On the other hand, when the fine enhancement stage is removed, the performance of the model also drops significantly, with the PSNR value dropping to 23.46 and the SSIM value dropping to 0.849. Although the impact is smaller than that of the SF module, it still has a significant negative impact on the model performance, emphasizing the importance of the fine enhancement stage in the quality of image restoration. By carefully adjusting the local features of the image, the fine enhancement stage can effectively enhance the texture and edges of the image, thereby improving the overall visual quality. Furthermore, removing the improved encoder-decoder module caused the PSNR value to drop to 22.92 and the SSIM value to drop to 0.839, showing the largest performance degradation, which proves the core role of the improved encoder-decoder in the network. Its optimized up- and down-sampling architecture and the introduced EWFM module are crucial to improving the image restoration accuracy and visual effect. Finally, removing the supervised attention module (SA) caused a slight performance degradation, indicating that the slight performance degradation of the SA module indicates that its focus on key areas and its auxiliary role in detail enhancement, although important, have less impact than other modules, and provide assistance to the subsequent fine enhancement stage, thereby improving the overall visual perception quality of low-light image enhancement.

[0148] In summary, the present invention proposes a two-stage low-light image enhancement method based on wavelet transform. The method includes two stages, namely the primary restoration stage and the fine enhancement stage. The former extracts information and features through the synergy of components such as the redesigned shallow feature extraction module, realizes preliminary low-light image restoration, and provides more restoration features to the fine enhancement stage. The latter aims to reconstruct and generate high-quality output images. In the primary restoration stage, in order to improve the quality of low-light enhanced images, an enhanced wavelet domain feature fusion module is proposed, and a dual attention mechanism unit is designed, including a lightweight fusion curve attention module and a dynamic gated spatial attention module, to process image features in a more refined and flexible way. In order to better guide model training and obtain higher-quality low-light enhanced images, a fusion perception loss function and a lightweight edge loss are introduced in the loss function to assist the model in obtaining better enhancement effects. Experimental results show that the model proposed by the present invention is superior to existing advanced methods. Future work will continue to improve the loss function design to improve model performance and computational efficiency. At the same time, the applicability of each module proposed by the present invention on other models is tested and applied to the field of low-light video enhancement.

[0149] The above-described embodiments of the present invention do not limit the protection scope of the present invention.

Claims

1. A two-stage low-light image enhancement method based on wavelet transform, characterized in that: include: Step 1: Acquire a low-light image, and use a two-stage U-shaped low-light image enhancement network to perform deep extraction of image features and integration of multi-scale information on the low-light image; The two-stage U-shaped low-light image enhancement network includes a primary restoration stage and a fine enhancement stage, wherein the primary restoration stage includes: a first gradient denoising module, a first shallow feature module, an encoder, a decoder, and a supervised attention module; the fine enhancement stage includes a second gradient denoising module, a second shallow feature module, and a fusion module; Step 2: Input the image features into the wavelet domain feature fusion and enhancement module, use discrete wavelets to capture image features of different scales in the wavelet domain, and use dynamic gated spatial attention and lightweight fusion curve attention to enhance important features; Step three, construct a fused perception function and a lightweight edge loss function. The fused perception function takes into account the pixel-level error and the overall perceptual quality of the image. By combining the characteristics of the pixel loss and the perception network, it guides the model to generate more natural and realistic images; the lightweight edge loss function focuses on high-frequency texture structures, further optimizing the model's ability to reconstruct details.

2. The two-stage low-light image enhancement method based on wavelet transform as claimed in claim 1, characterized in that: In step 1, the primary restoration stage performs feature extraction, denoising and preliminary enhancement tasks on the input low-light image through the first gradient denoising module and the first shallow feature module; uses the combined encoder and decoder structure to perform feature extraction and preliminary enhancement processing on the image; and compares the feature image processed by the encoder and decoder with the feature X after extraction and denoising. in Jointly input the supervised attention module for feature separation; a part of it is used to output the preliminary enhanced feature image X out , and the other part is used to generate the auxiliary feature map X f ,These two parts of features are input into the fusion module of the fine enhancement stage to provide feature information for detail recovery; The fine enhancement stage converts the auxiliary feature map X recovered in the primary restoration stage f With the initial enhanced image X out Fusion; finally, the output of the fine enhancement stage is added pixel by pixel to the input image to generate the final enhanced image.

3. The two-stage low-light image enhancement method based on wavelet transform as claimed in claim 2, characterized in that: In step 1, the first gradient denoising module in the primary restoration stage processes the input image through two residual branches, and then adjusts the original image through the scale parameter to obtain the preliminary denoising and deblurring results; The first shallow feature module is used to capture the spatial detail information in the low-light image after the initial deblurring and denoising by the first gradient denoising module. The first shallow feature module consists of a convolutional layer, a channel attention module, and a dilated convolutional module. The channel attention module first undergoes a feature processing flow, which includes two convolutional layers and a ReLU activation function. Then, the channel attention layer is introduced to identify and focus on key information and enhance the importance of features. In addition, the module also integrates an enhanced wavelet domain feature fusion module to further optimize the feature representation; finally, the processed feature map is element-wise summed with the original input feature map to obtain the feature information map as the input of the dilated convolution layer; Subsequently, a dilated convolutional layer is introduced to expand the receptive field and thus capture spatial information more effectively; The codec architecture integrates enhanced wavelet domain feature fusion module, hybrid instance normalization and residual structure. First, the enhanced wavelet domain feature fusion module performs denoising and optimization processing on the features. Then, hybrid instance normalization can effectively extract multi-scale feature information. Then, the processed features are fused and enhanced with the original input features through the residual structure. Before downsampling, the enhanced features are divided into two processing branches: one branch is fed into the decoder through a skip connection as the fusion feature f res ; The other branch is downsampled and used as the input of the decoder; The processing flow of the decoder is similar to that of the encoder. It performs upsampling before wavelet domain feature fusion and performs residual fusion with the features returned by the decoder. The encoder and decoder work together to obtain decoding features that integrate multi-scale information. These features are then fed into a supervised attention module for further processing; The supervised attention module combines the decoded output features of the input with the feature image X after preliminary denoising. in Combine: a part of the decoded output features is passed into the feature image X after convolution operation in And add it to get the spatial attention enhanced feature representation X f The other part is convolved with the spatial attention enhanced feature representation X f Add to get the feature map X after preliminary processing out ; X f and X out It is then passed to the subsequent fine enhancement stage for further processing.

4. The two-stage low-light image enhancement method based on wavelet transform as claimed in claim 3, characterized in that: In step 1, the fine enhancement stage retains the feature X of the image’s detailed texture information f And the feature map X processed by the second gradient denoising module and the second shallow feature module out ′ are input into the fusion module together. The fusion module concatenates the two in the feature dimension and integrates these features through a series of convolutional layers to convert them into high-quality feature maps suitable for image reconstruction. Then, the fused feature map is further converted through another convolutional layer to match the size of the original input image. Finally, the residual of the converted feature map is added to the original image to obtain the final enhanced image.

5. The two-stage low-light image enhancement method based on wavelet transform as claimed in claim 4, characterized in that: In step 2, the wavelet domain feature fusion and enhancement module performs the input feature F in Perform dimensional transformation and divide the input channel into separation and fusion features F containing complete low-frequency information of the image identity And the input discrete wavelet transform feature map F inw ; F identity Used as the final fusion recovery part, F inw It is the part to be processed by discrete wavelet transform; After discrete wavelet transform decomposition, F inw Decompose into high-frequency components and low-frequency components; for the high-frequency components, a series of deep encoding is performed using depth-separable convolution and parameterized rectified linear units; The processed high-frequency components are learned through dual attention units to obtain deep feature information F inwd , input it into the inverse wavelet transform to obtain the weighted feature F inwt , and the separation fusion feature F containing the complete low-frequency information of the image in the identity domain identity Combine and splice, and use the original low-frequency information retained in the separation and fusion features to supplement the missing low-frequency information, and supplement the missing low-frequency information to obtain the splicing feature F r ; Finally, after convolution and PReLU activation function layer, the residual feature F is obtained out , and perform residual connection with the original input to get the final output.

6. A two-stage low-light image enhancement method based on wavelet transform as claimed in claim 5, characterized in that: Dynamic gated spatial attention first extracts spatial features from the input feature map through maximum pooling and average pooling operations, and then combines the two pooling results of maximum pooling and average pooling. Splice along the channel dimension, calculate the attention weight, and generate the initial spatial attention map F through a 2D 5×5 convolution layer and a Sigmoid activation function sa ; The gated weight is calculated through the gated convolution layer. This layer uses a 1×1 convolution kernel to adjust the feature response of each position. Its output is also processed by the Sigmoid function to generate the gated weight W G ; Set the gate weight W G With the spatial attention map F sa Multiply them together to get the attention weight F after gate modulation G ; Multiply the gated modulated attention weights by the input image F to obtain the feature map F to which the gated spatial attention is applied GP ; Residual connection applies gated spatial attention feature map F GP And the original input feature map F, through the learnable fusion ratio α to achieve adaptive weighted fusion of features, to obtain the processed feature map F P .

7. The two-stage low-light image enhancement method based on wavelet transform as claimed in claim 6, characterized in that: Lightweight fusion curve attention will fuse feature information X Fe The curve parameters are estimated pixel by pixel through curve parameter calculation to obtain the curve parameters L n(n＝1,2,3) ; The gradient amplification module will fuse the feature information X Fe The input is input into two ReLU activation functions for processing: one part remains unchanged, while the other part is reset to zero, and the difference between the two is calculated; the result of this operation is that some eigenvalues ​​are retained, while the remaining irrelevant eigenvalues ​​are reset to zero, thus forming a dynamically adjusted step curve adjusted feature information F a ; The feature information F processed by the gradient amplification module a and curve parameter L n Combined with the curve parameters, and introducing nonlinear transformation (1-F a ); Finally, the adjusted features and feature information processed by the gradient amplification module F a Add and update the features to get the processed feature map F L .

8. The two-stage low-light image enhancement method based on wavelet transform as claimed in claim 7, characterized in that: In step three, the fusion perception function includes the Charbonnier loss function and the VGG perception loss function. The VGG perception loss function uses the pre-trained VGG network to measure the differences between feature layers, thereby capturing the texture and structural information in the image. The weighted sum of the Charbonnier loss function and the VGG perception loss function is used as the total loss function. The lightweight edge loss function uses the standard 3×3 Laplace convolution kernel.

Images