Low-light image enhancement method based on gradient perception and intermediate frequency guide network

Abstract

The invention discloses a low-illumination image enhancement method based on gradient perception and intermediate frequency guidance, which mainly constructs a double-branch encoder-decoder network model in an HVI color space so as to effectively recover image texture details, suppress noise and improve color fidelity under a low-illumination condition. Firstly, an RGB image is converted into an HVI color space, and two branches of HV and I are separated; secondly, in the encoding process of an encoding layer, a gradient guide channel attention module is adopted to enhance detail perception, and meanwhile, deformable cross-branch fusion attention is adopted between middle-layer features and in a decoding layer to achieve cross-branch dynamic information interaction; in a bottleneck layer, an intermediate frequency guided multi-scale frequency domain enhancement module is adopted to perform intermediate frequency guided amplitude feature enhancement and phase maintenance in a frequency domain; and finally, reconstructing the enhanced HVI image through a symmetric decoder, carrying out residual connection on the HVI image and the initial HVI image, and inversely transforming the HVI image back to the RGB space to obtain a recovered image. Compared with the prior art, the method has better image visual quality.

Description

A Low-Light Image Enhancement Method Based on Gradient Sensing and Mid-Frequency Guiding Network Technical Field

[0001] This invention belongs to the field of computer vision and digital image processing technology, specifically relating to a method for image enhancement under low-light conditions. Background Technology

[0002] Images captured in low-light environments commonly suffer from low brightness, poor contrast, significant noise, and severe loss of detail, severely limiting their application in fields such as security monitoring, autonomous driving, and medical imaging. Traditional enhancement methods, such as histogram equalization and gamma correction, often increase overall brightness but lead to localized overexposure, color distortion, and noise amplification. Methods based on Retinex theory attempt to decompose the reflection and illumination components, but their physical assumptions often fail in complex real-world scenes, and the decomposition process is computationally intensive. In recent years, deep learning-based low-light enhancement methods have made significant progress. However, existing deep learning methods still face the following key challenges: (1) The contradiction between detail preservation and enhancement: Most methods focus on enhancing global or local illumination, lacking explicit modeling and targeted protection of high-frequency details (such as edges and textures) in the image, resulting in blurred details or artifacts in the enhanced image; (2) Insufficient utilization of frequency domain information: Images contain rich information structures in the frequency domain, but most existing methods only operate in the spatial domain and fail to make full use of the characteristics of the frequency domain, especially the mid-frequency information (main outline, main texture) which is crucial for visual perception; (3) Inefficiency of cross-branch feature interaction: When processing different feature branches (such as brightness and intensity) separated or derived from the original image, simple splicing or addition operations cannot achieve adaptive and refined fusion between features based on semantic content, which limits the efficiency of information utilization. Summary of the Invention

[0003] To overcome the shortcomings of the existing technologies, this invention proposes a low-light image enhancement method based on gradient perception and mid-frequency guided networks. This method employs a gradient perception mechanism, mid-frequency guided frequency domain enhancement, and deformable cross-branch attention, aiming to achieve a balance between illumination enhancement, detail sharpening, noise suppression, and color fidelity. The specific implementation steps of this invention are as follows:

[0004] 1. The steps for constructing a dual-branch system and decoding features are as follows:

[0005] (1a) Input low-light sRGB image Convert to HVI color space to obtain Then The hue (H) and lightness (V) channels are concatenated to form the initial features of the lightness-hue branch (HV branch). The initial feature of constructing the intensity (I) channel as a separate intensity branch (I branch) .

[0006] (1b) The HV branch and the I branch are each passed through an encoder consisting of multiple cascaded downsampling modules. Each module reduces the spatial resolution of the feature map and increases the number of channels, thereby extracting multi-scale features. and , where L is the number of coding layers.

[0007] 2. A gradient-aware context attention module is introduced after the first and second layers of the encoder. This gradient-aware mechanism enhances the ability to preserve edges and details while suppressing noise amplification. This module performs the following operations:

[0008] (2a) For input features First, feature enhancement is performed using a multi-level residual structure:

[0009]

[0010] in, As the basic residual unit, This represents the ReLU activation function.

[0011] (2b) Features The input is fed into the parallel channel-spatial attention module to generate channel attention weights. Spatial attention weights :

[0012]

[0013] in, For the Sigmoid function, For ReLU function, For element-wise multiplication, This is global average pooling. The final output of the main path is:

[0014]

[0015] (2c) Gradient-aware enhancement: The gradient-aware path calculates the spatial gradient G of the input feature X through the Scharr operator, and the convolution kernel of the Scharr operator... (level) and (Vertical) is defined as:

[0016]

[0017] Gradient magnitude map ,in Represents convolution. To prevent division by zero for small constants. To obtain richer gradient context, the module internally calculates the input feature X and the features processed by the parallel channel-spatial attention module. gradient plot and Then and The data is concatenated using a lightweight convolutional network. Fusion is performed to generate gradient-aware features. The final output is:

[0018]

[0019] The fusion process can be achieved through a learnable weight parameter. After adjustments, the final output is a feature map that enhances edge and texture details.

[0020] 3. In the bottleneck layer (the deepest feature layer) of the encoder, a multi-scale frequency enhancement module guided by intermediate frequency is applied to achieve adaptive modulation in the frequency domain. This module performs the following operations:

[0021] (3a) Input features Perform channel attention calibration and layer normalization:

[0022]

[0023] (3b) Frequency domain transformation: by performing a fast Fourier transform : Will Transform to the frequency domain to obtain the amplitude spectrum. and phase spectrum Phase spectrum The main focus is on preserving the structural information of the image, which is maintained during subsequent processing.

[0024]

[0025] (3c) Amplitude Spectrum Enhancement: Amplitude Spectrum With a learnable frequency domain filter Perform basic filtering by multiplying each channel:

[0026]

[0027] Then, It is fed into a mid-frequency guided multi-scale frequency domain attention subnetwork, which will It is decomposed into low, medium, and high frequency components through different paths. With information-rich mid-frequency components As a guide, a cross-attention mechanism is used to modulate the low-frequency and high-frequency components separately, enhancing their synergy with the mid-frequency components:

[0028]

[0029] in, This indicates lightweight cross-channel attention operation. Modulated components. They are spliced ​​and merged to generate the final enhanced amplitude spectrum. .

[0030] (3d) To further enhance the edges, the Scharr gradient feature map calculated in the spatial domain is also transformed to the frequency domain, and its amplitude spectrum is... With learnable weights Weighted and superimposed superior.

[0031] (3e) Inverse frequency domain transform: Finally, the enhanced amplitude spectrum Compared with the original phase spectrum Combine and perform inverse fast Fourier transform. And add it to the input through the residual connection:

[0032]

[0033] in These are learnable scaling parameters.

[0034] 4. A deformable cross-branch fusion attention module is used across multiple encoder levels and throughout the decoding layer to achieve adaptive feature alignment and information exchange between the HV and I branches. This module performs the following operations:

[0035] (4a) Migration prediction and deformable sampling: using features from a certain layer of a branch As a query, the same-level features of another branch As context. First, through a lightweight convolutional network. From context features Predict a set of two-dimensional sampled offset fields , where N is the number of sampling points.

[0036] (4b) Deformable sampling and attention computation: utilizing predicted offsets , for Key features obtained from projection Sum Value Characteristics Bilinear sampling is performed to obtain aligned features. and .

[0037] (4c) Subsequently, the query features are calculated. and Attention weights, and apply them to The formula is expressed as:

[0038]

[0039] in, This represents the learnable temperature parameter. After a 1×1 convolution, it is combined with the input features... Perform residual connections through strength enhancement layers Nonlinear enhancement is performed to obtain the final output features. :

[0040]

[0041] 5. Construct the decoder using an upsampling module symmetrical to the encoder. During decoding, features from corresponding levels of the encoder are fused via skip connections to recover spatial details. At the corresponding levels of the decoding path, the deformable cross-branch fusion attention module described in step 4 is used again to refine the current branch features using the decoding features from another branch, ensuring the consistency of the fused features. At the end of the decoder, convolutional layers are used to map the number of feature channels to the target dimension, resulting in enhanced HV branch features. and I branch features .

[0042] 6. Inverse color space transformation and output: This involves transforming the enhanced color space... and The tonal channels retained or acquired from the original input are combined to form an enhanced HVI image. Finally, the HVI inverse transform is used to... Convert back to RGB color space to obtain the final enhanced image. .

[0043] Beneficial effects:

[0044] Compared with the prior art, the beneficial effects of this invention are reflected in:

[0045] (1) Excellent detail enhancement and preservation capabilities: By explicitly modeling and incorporating gradient information through the gradient-aware context attention module, the network can simultaneously perceive and enhance high-frequency details such as edges and textures when making global illumination adjustments, effectively solving the contradiction between enhancement and detail preservation, and outputting images with clearer and sharper details.

[0046] (2) Comprehensive Frequency Domain Awareness and Optimization: The mid-frequency guided multi-scale frequency enhancement module extends the processing domain to the frequency domain. Through amplitude spectrum filtering and mid-frequency guided multi-scale attention modulation, it can specifically enhance the frequency band components most important for visual perception. At the same time, maintaining the phase spectrum ensures the stability of the image's main structure. This frequency-spatial domain collaborative processing method achieves multi-level enhancement from global illumination distribution to local texture details.

[0047] (3) Efficient and accurate cross-branch feature fusion: The deformable sampling mechanism introduced by the deformable cross-branch fusion attention module breaks the limitation of traditional attention in calculating similarity on a fixed grid, and realizes non-local and adaptive feature alignment and fusion based on image content. This enables the structural information of the HV branch and the illumination information of the I branch to interact more deeply and effectively, improving feature utilization. Attached Figure Description

[0048] To more clearly illustrate the technical solution of the present invention, a brief introduction is given to the accompanying drawings used in the present invention.

[0049] Figure 1 is a flowchart of the low-light image enhancement method based on gradient perception and mid-frequency guidance network of the present invention;

[0050] Figure 2 is a schematic diagram of the overall structure of the present invention based on gradient sensing and mid-frequency guidance network;

[0051] Figure 3 is a qualitative comparison of the low-light image enhancement model constructed in this invention with other existing algorithm models on the LOL-V1 dataset.

[0052] Figure 4 is a qualitative comparison of the low-light image enhancement model constructed in this invention with other existing algorithm models on the LOL-V2-Real dataset.

[0053] Figure 5 is a qualitative comparison of the low-light image enhancement model constructed in this invention with other existing algorithm models on the LOL-V2-Syn dataset. Detailed Implementation

[0054] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0055] This invention provides a low-light image enhancement method based on gradient perception and mid-frequency guiding network, as shown in Figures 1 and 2. The specific steps are as follows:

[0056] 1. Data preprocessing and input.

[0057] Obtain low-light-normal-light image pairs from the public datasets LOL-V1, LOL-V2-Real, or LOL-V2-Synthetic. Normalize the pixel values ​​of the input low-light RGB image to the [0,1] interval.

[0058] 2. HVI color space conversion.

[0059] Normalized RGB image The image is transformed to the HVI color space using a fixed, invertible linear transformation matrix, and then decomposed into a 2-channel HV color map. A 1-channel brightness map This conversion aims to effectively separate color information from luminance information.

[0060] 3. Dual-branch coding and feature enhancement.

[0061] (3a) will and The inputs are fed into the HV branch encoder and the I branch encoder, respectively. Both encoders contain three downsampling stages, each implemented by a convolutional layer with a stride of 2, and the number of channels is [64, 64, 72, 144], respectively.

[0062] (3b) After downsampling in the first and second stages of the encoder, gradient-aware contextual attention blocks are inserted. Features are extracted internally through residual structures, and gradients are calculated using a 3×3 Scharr operator. These gradient features are then fused with the backbone features to output a gradient-aware enhanced feature map. The gradient fusion network... It consists of a 1×1 convolution, a LeakyReLU (with a negative slope of 0.1), and a 3×3 convolution, with learnable gradient fusion weights. The initial value is set to 0.1.

[0063] (3c) At the deepest part of the encoder (bottleneck layer), a frequency-guided multi-scale frequency enhancement module is applied to the features of the HV and I branches respectively. First, an FFT is performed on the input features to obtain the frequency domain representation. The amplitude spectrum is then passed through a learnable frequency domain filter. (Initialized to a full 1 tensor) Scaled and enhanced by a mid-frequency guided multi-scale attention, while the Scharr gradient in the spatial domain is also transformed to the frequency domain with learnable weights (edge ​​enhancement parameters). The original phase spectrum is initialized to 0 and then fused in for edge enhancement. Finally, the original phase spectrum is combined with IFFT and output through residual connection.

[0064] 4. Cross-branch fusion and decoding reconstruction.

[0065] (4a) Deformable cross-branch fusion attention modules are deployed at all corresponding levels (6 locations in total) of the encoder and decoder. Features from one branch (e.g., HV) are used as query input X, and features from another branch (e.g., I) are used as context input y. The internal offset network... (Constituted by two 3×3 convolutional layers and the GELU activation function) Based on the predicted sampling offset of y, the key and value features derived from y are then deformably sampled using these offsets. The number of attention heads is matched with the number of feature channels and set to [1, 2, 4, 8]. Finally, cross-attention calculation is performed, which realizes content-based dynamic feature interaction between the two branches.

[0066] (4b) The decoder upsamples through symmetric transposed convolutions and fuses the features of the corresponding layers of the encoder (including features enhanced by the gradient-aware context attention module) through skip connections. Finally, the HV and I branch features output by the decoder are concatenated along the channel dimension.

[0067] 5. Image Output. The stitched features are reconstructed back to the RGB color space using HVI inverse transform to obtain the final enhanced image. .

[0068] Key technical details:

[0069] Training configuration: Using the Adam optimizer ( , The initial learning rate is Cosine annealing was used for attenuation. Training was performed on an NVIDIA RTX 4090D GPU using the PyTorch framework. For the LOL-V1 and LOL-V2-Synthetic datasets, images were randomly cropped. The dataset uses pixel blocks with a batch size of 8 pixels and is trained for 1000 epochs. For the LOL-V2-Real dataset, images are randomly cropped. The training uses pixel blocks with a batch size of 8 pixels and a training duration of 1000 epochs. Loss function: A composite loss function is used. ,in For L1 loss, For structural similarity loss, For the perceptual loss based on VGG network, the weights and Set them to 0.5 and 0.1 respectively.

[0070] Result:

[0071] As shown in Figures 3, 4, and 5, our method is compared with nine advanced methods on the LOL-V1, LOL-V2-Real, and LOL-V2-Synthetic datasets, including RetinexNet, ZeroDCE, KinD, LLFormer, Retinexformer, IGDFormer, CIDNet, RetinexMamba, and DPEC. Quantitative experimental results show that our embodiment effectively provides a high-performance low-light image enhancement solution, which not only improves the brightness and contrast of low-light images but also effectively suppresses noise while highlighting the texture details of the image, achieving the best results among all the compared methods.

[0072] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A low-light image enhancement method based on gradient perception and mid-frequency guided network, characterized in that, Includes the following steps: (1) Convert the input low-light sRGB image to HVI color space and separate it into lightness-hue HV branch features and intensity I branch features; (2) Multi-scale encoding is performed on the HV branch features and I branch features respectively. During the encoding process, gradient-aware contextual attention processing is applied to the feature map at at least one scale. The processing includes calculating the spatial gradient of the feature map and fusing the gradient features with the original feature map to enhance the detail representation; (3) Cross-attention operation based on deformable sampling is performed at multiple scales during the encoding process to realize dynamic information interaction between the HV branch features and I branch features. The interaction uses the feature prediction sampling offset of one branch to adaptively sample from the features of the other branch; (4) Mid-frequency guided multi-scale encoding is performed on the encoded deep features. Frequency domain enhancement processing, the processing includes: converting the features to the frequency domain to obtain the amplitude spectrum and phase spectrum, performing learnable filtering and mid-frequency guided attention modulation on the amplitude spectrum, combining the modulated amplitude spectrum and the phase spectrum and inversely transforming it back to the spatial domain; (5) decoding the features processed by steps (2)-(4) to restore the spatial resolution, fusing the features corresponding to the encoding stage during the decoding process, and continuing to perform the cross-attention operation based on deformable sampling to refine the features, obtaining the enhanced HV branch features and I branch features; (6) combining the enhanced HV branch features and I branch features with the original information, and inversely transforming them to the RGB color space to output the enhanced image.

2. The low-light image enhancement method based on gradient perception and mid-frequency guiding network according to claim 1, characterized in that, Step (1) specifically includes: (1a) inputting a low-light sRGB image Convert to HVI color space to obtain (1b) will The hue (H) channel and lightness (V) channel in the image are spliced ​​together to form the initial features of the HV branch. , Will The intensity I channel in the data is used as the initial feature of the I branch. 。 3. The low-light image enhancement method based on gradient perception and mid-frequency guiding network according to claim 1, characterized in that, Step (2) describes applying gradient-aware contextual attention processing to the feature map, which includes the following steps: (2a) processing the input feature map... Primary features are extracted through a backbone network containing multiple convolutional layers and activation functions, and then... The input is fed into a module that performs channel attention and spatial attention in parallel to calibrate and focus features, which are then compared with the initial input features. The main path output features are obtained by adding them together. ； (2b) Calculate the input feature map using the Scharr operator. gradient feature map and the main path output features gradient feature map The convolution kernel of the Scharr operator is defined as follows: The formula for calculating the gradient feature map is as follows: in This represents the convolution operation. (2c) The gradient feature map is a small constant; and After concatenation, it is processed through a lightweight convolutional network. Fusion is performed to generate gradient-aware features. The primary features are combined with the gradient-aware features. With learnable weights Combining these, we obtain the output features: in, Initially set to 0.1, and learning from the data as training progresses, the network can dynamically enhance or suppress structural elements based on gradient analysis.

4. The low-light image enhancement method based on gradient perception and mid-frequency guiding network according to claim 1, characterized in that, The cross-attention operation based on deformable sampling described in step (3) includes the following steps: (3a) using the features of one branch As a query source, based on the features of another branch (3b) As a key source; (3b) through an offset prediction network consisting of convolutional layers based on Predict a set of two-dimensional sampling offsets Using the offset To Key features obtained from mapping Sum value characteristics Bilinear resampling is performed to obtain the deformed key features. Sum value characteristics (3c) Calculate query features With deformable bond features The similarity is used as the attention weight, and this weight is applied to the deformable value feature. Resulting in attention output: in, This represents a learnable temperature parameter used to control the acuity of attention distribution.

5. The low-light image enhancement method based on gradient perception and mid-frequency guiding network according to claim 1, characterized in that, The intermediate frequency guided multi-scale frequency domain enhancement processing described in step (4) includes the following steps: (4a) processing the input features Performing a Fast Fourier Transform yields the frequency domain representation: Among them Amplitude spectrum The phase spectrum mainly preserves the structural information of the image and is maintained in subsequent processing; (4b) the amplitude spectrum Apply a learnable frequency domain filter The filtered amplitude spectrum is obtained as follows: Will Input a mid-frequency guided multi-scale frequency domain attention subnetwork and generate an attention weight map. And apply the attention weight map It is obtained by decomposing it into low, medium and high frequency components through different paths. With information-rich mid-frequency components As a guide, a cross-attention mechanism is used to modulate the low-frequency and high-frequency components separately, enhancing their synergy with the mid-frequency components: in, This indicates lightweight cross-channel attention operation. Modulated components. The spectra are spliced ​​and merged to generate an enhanced amplitude spectrum. (4c) Calculate input features Gradient magnitude in the spatial domain It is then transformed to the frequency domain using a fast Fourier transform and the amplitude spectrum is obtained. The gradient magnitude spectrum With learnable weights Weighted and modulated amplitude spectra The fusion yields the final enhanced amplitude spectrum: Will Compared with the original phase spectrum By combining these methods and performing an inverse fast Fourier transform back to the spatial domain, the frequency-enhanced features are obtained. and the original input Frequency domain enhancement features By combining residual connections, the final features are output: in, These are learnable scaling parameters.

6. The low-light image enhancement method based on gradient perception and mid-frequency guiding network according to claim 1, characterized in that, The decoding process described in step (5) adopts an upsampling structure symmetrical to the encoder. Each decoding block fuses the features of the corresponding level of the encoder through skip connections to restore spatial details. At the same time, cross-attention operation based on deformable sampling is continued to be used on the decoding path to refine the features of the current branch using the decoding features of another branch.

7. The low-light image enhancement method based on gradient perception and mid-frequency guiding network according to claim 1, characterized in that, The specific operation of step (6) is as follows: The enhanced HV branch features... and I-branch features Combined with the tone channels preserved from the original input, an enhanced HVI image is formed. By using HVI inverse transform Convert back to RGB color space to obtain the final enhanced image. 。

Images