Low-light image enhancement method based on multi-attention mechanism

Abstract

The invention provides a low-light image enhancement method based on a multi-attention mechanism. The method comprises the following steps: constructing an end-to-end neural network model named GLADFormer; the model comprises an illumination estimation module, an image restoration module and a pixel perception gating modulation module. The illumination estimation module extracts local and global illumination features, and introduces comparative learning to enhance feature discrimination; the image recovery module adopts a multi-scale U-shaped structure and is fused with an illumination guiding attention mechanism to realize structural detail recovery and noise suppression; and the pixel perception gating modulation module performs pixel-level semantic enhancement and structure optimization based on the codebook. The whole network is trained by adopting a joint loss function, and the reconstruction quality and illumination consistency are considered. The method can effectively improve the brightness balance and structural definition of the low-light image, is suitable for various actual scenes such as night vision and video monitoring, and has good adaptability and deployment efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a low-light image enhancement method based on a multi-attention mechanism, which can be applied to night imaging enhancement, video monitoring, low-light target detection and other visual tasks. BACKGROUND

[0002] In natural scenes, images taken under low-light conditions often have problems such as insufficient brightness, missing details, low contrast, and high noise. These problems not only reduce the subjective visual quality of the image, but also significantly affect the performance of downstream image recognition, target detection and other high-level visual tasks.

[0003] Traditional low-light image enhancement methods mainly rely on classic image processing algorithms such as histogram equalization, Gamma correction, Retinex theory, etc. Although these methods can improve image brightness in some specific scenarios, they often have problems such as uneven enhancement, loss of details, or amplification of noise, making it difficult to meet the needs of image enhancement under complex lighting conditions.

[0004] With the development of deep learning, end-to-end neural network methods have gradually become the mainstream of low-light image enhancement. Among them, the Transformer architecture is gradually introduced into the field of image enhancement due to its strong modeling ability, which is used to capture long-range dependencies and global features in images. However, the self-attention calculation in the standard Transformer has a complexity of O(N 2 )(where N is the number of pixels in the image), which brings significant computational overhead and memory burden in high-resolution images, severely limiting its practicality and deployment efficiency. To reduce the above complexity, the image is divided into local blocks for attention modeling, and the attention calculation is limited within a local window, significantly reducing the computational complexity to O(M 2 ·K)(where M is the window size and K is the number of windows), greatly improving the inference efficiency while maintaining the modeling ability. SUMMARY

[0005] The present application aims to overcome the shortcomings of existing low-light image enhancement methods in handling non-uniform lighting, detail restoration, noise suppression, and modeling efficiency, and proposes a low-light image enhancement method based on a multi-attention mechanism, which can effectively improve the brightness uniformity and structural clarity of the image, and has good robustness and deployment efficiency, and is widely applicable to night target detection, video monitoring, image preprocessing and other visual task scenarios.

[0006] Specifically, the present application provides a low-light image enhancement method based on a multi-attention mechanism, comprising the following steps:

[0007] Step 1, acquire a low-light image as input, splice the image and its channel mean as the input of the light estimation module, adopt a convolution structure to extract local and global light features, generate a light feature map and a brightness estimation map, and enhance the discriminability of the light feature through a contrast learning mechanism;

[0008] Step 2, generate a brightened image according to the brightness map as the basic input of the subsequent enhancement process;

[0009] Step 3, input the brightened image into the image restoration module, which adopts a multi-scale U-shaped encoding-decoding structure, introduces a light-guided attention block (LGAB) in each scale, fuses the light feature through local block division and mask attention mechanism, and realizes multi-scale structure restoration and noise suppression;

[0010] Step 4, input the restored feature into the pixel perception gate modulation module, which uses light guidance to generate a pixel-level soft allocation weight, extracts structural semantic features from a learnable codebook, and performs pixel-level fusion and nonlinear gating to further improve local detail performance;

[0011] Step 5, output the enhanced image, and jointly optimize the image reconstruction loss and the light contrast loss through an end-to-end training method to improve the image quality and enhance the robustness of the network to light changes.

[0012] Compared with the prior art, the present application has the following beneficial effects:

[0013] The present application adopts a combination of multi-attention mechanism and light perception strategy to improve the brightness of low-light images while effectively preserving structural details and suppressing noise, with significant overall performance advantages. By replacing the traditional global self-attention with a local window attention mechanism, the computational complexity is significantly reduced, improving the training and inference efficiency of the model; the introduction of a contrast learning-driven light estimation module can accurately model non-uniformly illuminated areas in the image, enabling adaptive brightness compensation; the image restoration process combines a light-guided attention block and a pixel-level gate modulation module, effectively enhancing the texture performance and semantic consistency of dark areas, avoiding over-enhancement and artifact problems; the overall structure adopts an end-to-end joint optimization method to ensure the collaborative effect and model stability between modules; in addition, this method is suitable for a variety of complex low-light scenes and has good adaptability and deployment value in night target detection, video monitoring preprocessing and other practical applications.

[0014] The parts not involved in the present application are the same as or can be realized by the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 The flowchart of the method described in the present application.

[0016] Figure 2 The structure diagram of the light-guided attention block (LGAB) proposed for the method. DETAILED DESCRIPTION

[0017] For better understanding and implementation of the present application by those skilled in the art, the technical solutions of the present application are further described below in combination with the drawings and one typical embodiment of the present application, but the present application is not limited to the described embodiments, and any equivalent improvement based on the concept of the present application shall be covered within the protection scope of the present application.

[0018] The present application provides a low-light image enhancement method based on a multi-attention mechanism, and an end-to-end neural network model named GLADFormer is constructed. The method mainly includes the following three functional modules:

[0019] An illumination estimation module E (Illumination Estimator);

[0020] An image restoration module R (Corruption Restorer);

[0021] A pixel-aware gated modulation module P (Pixel-Aware Gated Modulator, PAGM).

[0022] The specific implementation process is as follows:

[0023] Step 1: input processing

[0024] A low-light RGB image with a size of Bx3xHxW is input into the system, and the mean value of each channel is calculated. The original image and the channel mean value are spliced to form a 4-channel image, which is used as the input of the model.

[0025] Step 2: illumination estimation module E

[0026] This module is used to extract the illumination perception features of the image. First, the input is projected through a 1x1 convolution, and then a 5x5 deep convolution is used to extract local illumination features. Then, a global average pooling is used to generate a global context vector, which is broadcast to the spatial dimension and fused with the local features to output an illumination feature map Flu. Further, a 1x1 convolution is used to generate a brightness map Lmap, which is used to construct a brightened image Ilu. To enhance the illumination discrimination ability of the model, a contrast learning strategy is introduced, two variants (brightened sample and darkened sample) are constructed from the input image, and the contrast loss between their features is calculated to guide the model to learn a robust and discriminative illumination representation.

[0027] Step 3: image restoration module R

[0028] The bright image Iu is input into the image restoration module for structure enhancement and noise suppression. The module is a three-scale U-shaped structure, and multiple light guidance attention blocks (LGAB) are stacked at each resolution.

[0029] Each LGAB includes the following substructure:

[0030] LCMA (Local Chunked Masked Attention): The input is divided into non-overlapping local windows, self-attention calculation is performed, and the light feature Flu is introduced for modulation.

[0031] Global guidance fusion: Introduce tokens obtained by global pooling in each window to achieve local modeling and global consistency.

[0032] Feedforward network (FFN): Enhance the channel representation ability of the attention output.

[0033] The decoder output is an image residual, which is added to the brightness map Lmap to form a preliminary enhanced image.

[0034] Step 4: Pixel perception gating modulation module P

[0035] To further refine the image structure and semantic consistency, a PAGM module is used for pixel-level enhancement. The module includes:

[0036] Codebook construction: Predefine K vectors to form a learnable semantic codebook;

[0037] Soft assignment weight prediction: Predict the code assignment weight of each pixel from Lmap through a convolutional network;

[0038] Guided feature fusion: Gather semantic guide vectors from the codebook according to the pixel position, and concatenate them with the image restoration feature X to input into the gating mechanism;

[0039] Gating modulation output: Implement nonlinear modulation through sigmod and tanh activation functions to output the final enhanced image.

[0040] This module can effectively adaptively adjust the detail area under light perception, enhance brightness, contrast, and texture consistency.

[0041] Step 5: Training method

[0042] The model uses an end-to-end joint optimization strategy, and the loss function used includes:

[0043] Image reconstruction loss L1 loss of the enhanced image and the original image;

[0044] Light contrast loss Samples with similar illumination are encouraged to have similar representation.

[0045] The overall loss is:

[0046] where λ1=1.0, λ2=0.1.

[0047] Adam optimizer is used during training with an initial learning rate of 1e-4, a batch size of 8, and the training set can be selected from LOL, SID, SMID, and other low-light public datasets.

[0048] Step 6: Application scenarios

[0049] The trained model can be directly used for image enhancement applications such as night vision systems, video monitoring preprocessing, automatic driving perception systems, etc., with high precision, strong robustness, and good general deployment.

Claims

1. A low-light image enhancement method based on a multi-attention mechanism, characterized in that, The method comprises the following steps: Step 1, obtaining a low-light image as an input image; Step 2, inputting the input image into an illumination estimation module, the module extracts local and global illumination features through convolution after splicing the image and its channel mean, outputs a brightness map and an illumination feature map, and enhances the discriminability of the illumination feature by using a contrast learning strategy; Step 3, generating a brightened image based on the brightness map; Step 4, inputting the brightened image into an image restoration module, the module is a multi-scale U-shaped structure, and integrates an illumination-guided attention block in each scale, realizes structure restoration and noise suppression by local block division and mask attention calculation, and fuses the illumination feature; Step 5, inputting the restored feature into a pixel-aware gated modulation module, the module generates a pixel-level soft assignment weight based on the illumination map, guides the retrieval of structure semantic features from the codebook, and realizes the final image enhancement output; Step 6, the method adopts end-to-end joint training, and minimizes the weighted combination of image reconstruction loss and illumination contrast loss.

2. The method of claim 1, wherein, The illumination-guided attention block comprises local block division, self-attention calculation module and illumination modulation module; wherein the self-attention calculation adopts local mask mode, and the illumination feature is used for pixel-by-pixel adjustment of the value branch.

3. The method of claim 1, wherein, The pixel-aware gated modulation module comprises: A learnable structure semantic codebook; An illumination map driven pixel-level weight predictor; A gated fusion module for nonlinear fusion output of the restored feature and the codebook guided feature.

4. The method of claim 1, wherein, The method can be directly applied to low-light image enhancement, night target detection and video monitoring preprocessing and other computer vision scenes after training is completed.

Images