A method and system for low-light image enhancement based on illumination-guided convolution attention model

Abstract

The application discloses a low-light image enhancement method and system based on an illumination-guided convolution attention model, and belongs to the field of computer vision. The method and system solve the problems that the existing low-light image enhancement technology is difficult to establish global correlation, leading to noise amplification and detail loss, complex calculation and weakening of local details, slow reasoning and poor local recovery effect, and thus restricts the quality and efficiency of image enhancement. The method comprises the following steps: S1: inputting a low-light image and constructing illumination prior information; S2: using an illumination estimator to estimate an illumination feature map and an illumination map according to the illumination prior information, and constructing an illumination image; S3: using an illumination restorer to process the illumination image to generate an illumination restoration image; and S4: generating an enhanced image according to the illumination restoration image and the illumination image. The application is suitable for a low-light image enhancement scene.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to low-light image enhancement, a preprocessing step in image processing. More specifically, it is a low-light image enhancement method based on an illumination-guided convolutional attention model. Background Technology

[0002] Computer vision has undergone a significant transformation with the rise of deep learning technologies, which have played a central role in solving various visual challenges, including low-light image enhancement. Low-light image enhancement is a crucial aspect of computer vision and image processing research because it not only optimizes visual cognition but also improves the performance of subsequent applications, such as autonomous driving systems, object recognition, and image semantic segmentation. Images captured in low-light environments often suffer from various degradation problems, such as low visibility, weak contrast, and noise interference. Therefore, enhancing such images has become an indispensable preprocessing step.

[0003] With the advancement of deep learning technology, Convolutional Neural Networks (CNNs) have been widely used to improve image quality under low-light conditions. Currently, these techniques are mainly divided into two categories. The first category directly constructs a mapping relationship between low-light images and standard-light images; however, these methods lack interpretability and theoretical support. The second category of methods, based on Retinex theory and CNNs, divides the training process into three independent stages: image decomposition, reflection noise reduction, and illumination adjustment. Each stage requires separate fine-tuning, resulting in a complex and time-consuming process. CNNs have significant advantages in extracting local features and spatial information, but they are limited in capturing long-distance similarities, making them insufficient to fully grasp overall image information. This manifests as challenges in noise amplification and detail preservation during low-light image processing. In contrast, Transformer models, through self-attention mechanisms, can achieve global perception and long-distance dependency modeling; however, when processing high-resolution images or high-dimensional features, these methods generate significant computational complexity, leading to huge resource requirements. Furthermore, global operations often weaken the ability to preserve local texture and edge details, limiting their effectiveness in fine-grained visual tasks. Mamba models achieve global perception and long-range dependency modeling through spatial state models, which helps to more accurately recover details and structure of low-light images. However, they often introduce a large number of parameters, leading to slow inference, and still have limitations in feature matching between the encoder and decoder, resulting in poor performance in recovering image details, color transitions in local regions, and texture information in local regions. Summary of the Invention

[0004] This invention provides a solution to address the shortcomings of existing low-light image enhancement techniques, such as difficulty in establishing global correlations leading to noise amplification and loss of details, computational complexity that weakens local details, slow inference, and poor local restoration, which restrict the quality and efficiency of image enhancement.

[0005] In a first aspect, the present invention aims to provide a low-light image enhancement method based on an illumination-guided convolutional attention model, comprising the following steps: S1: Input a dark light image and construct prior lighting information; S2: Employ an illumination estimator to estimate illumination feature maps and illumination maps based on prior illumination information, and construct illumination images; S3: Use the illumination restorer to process the illumination image and generate an illumination restore map; S4: Generate an enhanced image based on the illumination restoration map and the illumination image.

[0006] Furthermore, a preferred solution is provided: the prior illumination information is obtained by calculating the average value of the channel dimension and then adding it to the HSV spatial degree component.

[0007] Furthermore, a preferred embodiment is provided: the illumination estimator includes a first... layer, Convolutional layer and second layer.

[0008] Furthermore, a preferred embodiment is provided: S2 includes: S21: Concatenate the low-light image with the prior lighting information in the channel dimension as input; S22: Through the first The layer fuses prior lighting information into the low-light image; S23: Pass The convolutional layer performs upsampling and further integrates prior lighting information to obtain a lighting feature map with 40 dimensions; S24: Through the second The layer downsamples the output to obtain the lighting map; S25: Perform element-wise multiplication on the illumination map and the dark image to obtain the illumination image.

[0009] Furthermore, a preferred embodiment is provided: the illumination restorer is based on an illumination-guided convolutional attention framework, which includes an encoder and a decoder, the encoder being responsible for downsampling operations and the decoder being responsible for upsampling operations.

[0010] Furthermore, a preferred embodiment is provided: S3 includes: S31: The illumination image is downsampled by the encoder to obtain a feature map and extract image features; S32: Upsample the image features to obtain an illumination restoration map.

[0011] Furthermore, a preferred embodiment is provided: the enhanced image is obtained by adding the illumination restoration map and the illumination image pixel by pixel.

[0012] Secondly, the purpose of this invention is to propose a low-light image enhancement system based on an illumination-guided convolutional attention model. The system is implemented based on a low-light image enhancement method based on an illumination-guided convolutional attention model as described in any one or more of the above-mentioned schemes. The system includes: Prior construction module: used to input a dark light image and construct prior lighting information; Illumination estimation module: Used to employ an illumination estimator to estimate illumination feature maps and illumination maps based on prior illumination information, and to construct illumination images; Lighting restoration module: Used to process lighting images using a lighting restorer to generate lighting restoration maps; Lighting enhancement module: Used to generate enhanced images based on lighting restoration maps and lighting images.

[0013] Thirdly, the present invention aims to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, and when the processor runs the computer program stored in the memory, the processor executes a low-light image enhancement method based on an illumination-guided convolutional attention model according to any one or more of the above-described schemes.

[0014] Fourthly, the present invention aims to provide a computer-readable storage medium for storing a computer program that executes a low-light image enhancement method based on an illumination-guided convolutional attention model as described in any one or more of the above-described schemes.

[0015] The beneficial effects of this invention are: This invention proposes a low-light image enhancement method based on an illumination-guided convolutional attention model. First, it proposes a novel illumination prior that integrates the channel mean of the RGB color space with the Value (luminance) component of the HSV color space, effectively combining physical statistical measures with human perceptual cues. This prior exhibits stronger robustness in capturing non-uniform illumination distributions, thus significantly reducing color bias and detail loss. Second, addressing the issues of high computational complexity and resource consumption in existing attention mechanisms when processing high-resolution images or high-dimensional features, and the tendency for global operations to weaken local detail preservation, this invention proposes a Bidirectional Convolutional Attention Module (BCAM). The core innovation of this module lies in abandoning the traditional query-key-value interaction mechanism and instead using dynamically generated convolutional kernels to achieve feature modulation. BCAM achieves bidirectional adaptive fusion between illumination features and content features through grouped convolution operations. Its computational complexity is linearly related to the spatial size and dimensionality of the input feature map, thus significantly reducing the computational overhead in high-resolution image or high-dimensional data processing. Finally, addressing the issues of inconsistent feature alignment, insufficient information utilization, and inadequate robustness of feature fusion during encoder-decoder feature fusion in low-light image enhancement, this invention proposes a spatial channel-weighted feature fusion module. By generating channel-specific spatial interaction maps, it enhances the model's adaptability and stability under different lighting conditions and noise levels. This invention significantly improves the detail preservation and brightness enhancement of low-light images, effectively reducing noise and artifacts while maintaining natural color transitions, thus improving the overall visual quality and naturalness of the image.

[0016] The significance of this invention lies in its significant improvement in the overall visual quality and naturalness of low-light images, enabling images captured under complex lighting conditions to present details more clearly and realistically.

[0017] This invention is applicable to low-light image enhancement scenarios. Attached Figure Description

[0018] Figure 1 This is a flowchart of the low-light image enhancement method based on the illumination-guided convolutional attention model according to a specific embodiment of the present invention; Figure 2 This is a flowchart illustrating the internal workings of the illumination estimator according to a specific embodiment of the present invention. Figure 3 This is a structural diagram of the illumination-guided bidirectional convolutional attention block according to a specific embodiment of the present invention; Figure 4 This is a flowchart illustrating the fusion strategy described in a specific embodiment of the present invention; Figure 5 This is a schematic diagram of the bidirectional convolutional attention module according to a specific embodiment of the present invention; Figure 6 This is a schematic diagram of the content-guided attention module as described in a specific embodiment of the present invention; Figure 7 (a) is the input dark image, (b) is the ground truth image, and (c) is the dark enhancement result image. Detailed Implementation

[0019] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0020] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0022] Implementation Method 1 A low-light image enhancement method based on an illumination-guided convolutional attention model includes the following steps: S1: Input a low-light image Illumination prior information is constructed by averaging along the channel dimension and adding it to the luminance component in the HSV space. ; Specifically, in this embodiment, the prior lighting information of the image is obtained by calculating the average value of the three channel dimensions and adding it to the luminance component in the HSV space, as shown in the following formula: , (1) , (2) in, This represents a non-linear mapping from the RGB to the HSV color space; , and These represent the values ​​of the red, green, and blue channels of the image, respectively. This represents the brightness channel obtained from the HSV conversion.

[0023] RGB mean provides stability in color distribution, while HSV's V channel is consistent with human brightness perception, thus enabling more accurate mapping of light intensity.

[0024] S2: Employ an illumination estimator to estimate the illumination feature map based on prior illumination information. and lighting mapping And construct a lighting image ,include: S21: Concatenate the low-light image with the prior lighting information in the channel dimension as input; S22: Through the first The layer fuses prior lighting information into the low-light image; S23: Pass The convolutional layer performs upsampling and further integrates prior lighting information to obtain a lighting feature map with 40 dimensions; S24: Through the second The layer downsamples the output to obtain the lighting map; S25: Perform element-wise multiplication on the illumination map and the dark image to obtain the illumination image.

[0025] S3: Use the illumination restorer to process the illumination image and generate an illumination restore map. ,include: S31: The illumination image is downsampled by the encoder to obtain a feature map and extract image features; S32: Upsample the image features to obtain an illumination restoration map.

[0026] S4: Based on the illumination restoration diagram and lighting images Generate enhanced images Specifically, the enhanced image is obtained by adding the illumination restoration map and the illumination image pixel by pixel.

[0027] The effect of processing a low-light image using the method described in this embodiment is shown in the figure below. Figure 7 As shown.

[0028] Implementation Method 2 This embodiment is a further illustrative example of S2 in the low-light image enhancement method based on the illumination-guided convolutional attention model described in Embodiment 1.

[0029] The internal workflow of the illumination estimator is shown in the diagram below. Figure 2 As shown, it includes two Layers and Convolutional layers convert dark images With prior information about illumination Concatenation along the channel dimension is used as input. Next, features are extracted through three convolution operations: the first... The layer will concatenate the previously spliced ​​input Merging primarily serves to combine prior information about illumination. Blended into low-light images In the middle. The second convolution is a depthwise separable one. Convolution, its main function is to perform upsampling and further integrate illumination priors to obtain an illumination feature map with 40 dimensions. The last one The layer downsamples the output to produce a three-channel illumination map. Then by mapping the light... Low-light images Perform element-wise multiplication to obtain the illumination image. .

[0030] Among them, input Illumination feature map Light mapping and lighting images The results can be obtained through the following calculations: , (3) , (4) , (5) , (6) in, This indicates a concatenation operation along the channel dimension. This indicates element-wise multiplication.

[0031] Implementation Method 3 This embodiment is a further illustrative example of S3 in the low-light image enhancement method based on the illumination-guided convolutional attention model described in Embodiment 1.

[0032] The illumination restorer is based on an illumination-guided convolutional attention framework. This framework comprises an encoder and a decoder, where the encoder handles downsampling and the decoder handles upsampling. Both processes are symmetrical and each consists of two stages. First, the illumination image from the illumination estimator... pass Downsampling is performed with a step size of 2, and its dimensions are compared with the illumination feature map. Alignment obtained This is for subsequent processing. The downsampling operation is divided into two stages, each stage including an illumination-guided bidirectional convolutional attention block (IBCAB), followed by a convolution with a stride of 2 and a kernel size of [missing information]. The convolutional layers. With each convolutional layer, the width and height of the image are halved, while the feature dimension doubles. After two levels of downsampling, the feature map dimension will be... After extracting image features, upsampling is performed to reconstruct the image. The upsampling process is similar to the downsampling process, also consisting of two stages. Each stage includes one... (Stride = 2), and an illumination-guided bidirectional convolutional attention block (IBCAB). With each deconvolution layer, the image width and height increase, while the feature dimension decreases. To optimize the fusion of shallow and deep features and address the feature matching problem between the encoder and decoder, the outputs of the deconvolution layers and the corresponding levels of IBCAB outputs are processed through a spatial channel weighted feature fusion (SCWFusion) module. Finally, a... (Stride=2) to reduce dimensionality, restore the image to RGB format, and generate an illumination restoration map. .

[0033] The core component of the illumination restorer is an illumination-guided bidirectional convolutional attention block, such as... Figure 3 As shown, it features a bidirectional convolutional attention module (BCAM), layer-normalization (LN), and a feed-forward network (FFN).

[0034] like Figure 5 As shown, the bidirectional convolutional attention module (BCAM) is designed to facilitate the processing of illumination features. (Derived from the illumination estimator) and input features Efficient and aligned feature fusion between features (such as those from illuminated images). Unlike treating features as tokens for dot-product attention, BCAM adaptively generates convolution kernels from two features, achieving a bidirectional modulation that mimics the mutual guidance mechanism in cross-attention. This implicitly incorporates global information into the generation process of the convolution kernels, while retaining the locality and efficiency of convolution.

[0035] The overall architecture of the BCAM module is implemented using the following algorithm: Input: Image features Illumination characteristics ; Output: Enhanced image features ; Step 1: Dynamic Convolution Kernel Generation Computing light-sensing convolution kernels: ; Computation of content-aware convolution kernels: ; Step 2: Bidirectional Feature Modulation Modulation content characteristics: ; Modulated illumination characteristics: ; Step 3: Feature Fusion and Enhancement Fusion characteristics: This represents a nonlinear feature enhancement network.

[0036] The BCAM module consists of three steps: dynamic convolutional kernel generation, bidirectional feature modulation, and feature fusion and enhancement. During dynamic convolutional kernel generation, the convolutional kernel... and Generation is achieved through a lightweight network that captures the global context using adaptive average pooling. This design enables convolutional operations to mimic the global receptive field of the attention mechanism, achieving long-range dependency modeling while maintaining computational efficiency.

[0037] in, Represents the features of the input image. Indicates enhanced image features, This indicates a global pooling operation, which retrieves global information about the features. This means using a convolution with a kernel size of 1 to reduce the feature channel dimension from... Mapped to ; This indicates that the Gaussian error linear unit is used as the activation function; This means using a convolution with a kernel size of 1 to reduce the feature channel dimension from... Mapped to , This represents a nonlinear feature enhancement network.

[0038] During bidirectional feature modulation, the generated convolutional kernels are applied to the feature streams in opposite directions through grouped convolutions, achieving feature interaction and modulation. Specifically, illumination-aware convolutional kernels are used to convolve image features, while content-aware convolutional kernels are used to convolve illumination features. For feature fusion and enhancement, the two modulated feature streams are summed and fused, and the fused result is input into a nonlinear enhancement unit for further processing to improve the feature representation capability.

[0039] like Figure 4 and Figure 6 As shown, the core idea of ​​the SCWFusion module is to guide the fusion of feature maps in both spatial and channel dimensions by generating channel-specific spatial interaction graphs (SIMs). Specifically, the input features... First, after a series of processing steps, SIMs with the same dimensions are generated. This provides guidance for subsequent weighted fusion.

[0040] First, channel attention and spatial attention maps are computed using global average pooling (GAP) and global max pooling (GMP) operations. and .

[0041] , (7) , (8) in express Activation function Represent a Convolutional layers with kernel size, This indicates a channel splicing operation. In this context, GAP represents global average pooling, and CA represents processing features along the channel dimension. In this context, SA indicates that features are processed along the spatial dimension. In this context, GMP stands for Global Max Pooling Operation. This represents a channel attention map, where CA indicates feature processing along the channel dimension and SA indicates feature processing along the spatial dimension. Represents a spatial attention map.

[0042] To reduce the number of parameters and limit model complexity, the first Convolution reduces the channel dimension from Reduce to (in (Refers to the reduction ratio), the second one Convolution expands it back This implementation method selects to set... for This reduces the channel dimension to a fixed value.

[0043] Then, through simple addition operations, following the broadcast rules, ... and Combined together to obtain a rough spatial attention map .

[0044] , (9) After obtaining the initial SIMs ( Following this, the SCWFusion module further fuses the features using an adaptive weighting method. Specifically, the SIMs of each channel are alternately rearranged with the input features to generate the final channel-specific SIMs. This process, through a channel shuffle operation, ensures that each channel feature can be effectively combined with its corresponding SIMs, thereby enhancing the key region features in the image.

[0045] , (10) in, This represents the sigmoid operation. This indicates a channel shuffling operation. Represent a Grouped convolutional layers with varying kernel sizes.

[0046] To further improve feature fusion performance and ensure training stability, the SCWFusion module does not simply employ skip connection mechanisms. Instead, it performs weighted fusion of encoder and decoder features. This strategy avoids the vanishing gradient problem, ensuring the model can better transmit gradient information during training, promoting more efficient learning, and making low-light image enhancement more efficient and robust.

[0047] (11) in, A spatial interaction diagram representing a specific channel. Indicates fusion features, This represents the characteristics of the encoder output. This represents the characteristics of the decoder output.

[0048] The SCWFusion module combines feature representations from different levels through weighted fusion of encoder and decoder features. This feature fusion enables the network to learn image details and brightness information at multiple scales, while avoiding the feature information loss problem in traditional methods. After feature fusion, concatenating the encoder features with the fused features along the channel dimension effectively balances the influence of local and global information. Dimensionality reduction through 1×1 convolutions allows the network to focus on detail processing of local features without losing global information. This balance between global and local information is crucial for enhancing low-light images, especially in detail recovery and brightness enhancement.

[0049] Implementation Method 4 This embodiment is a further illustrative example of S4 in the low-light image enhancement method based on the illumination-guided convolutional attention model described in Embodiment 1.

[0050] S4 restores the illumination map and lighting images Perform pixel-by-pixel addition to generate an enhanced image. The formula is as follows: , (12) It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A low-light image enhancement method based on an illumination-guided convolutional attention model, characterized in that, Includes the following steps: S1: Input a dark light image and construct prior lighting information; S2: Employ an illumination estimator to estimate illumination feature maps and illumination maps based on prior illumination information, and construct illumination images; S3: Use the illumination restorer to process the illumination image and generate an illumination restore map; S4: Generate an enhanced image based on the illumination restoration map and the illumination image.

2. The low-light image enhancement method based on an illumination-guided convolutional attention model according to claim 1, characterized in that, The illumination prior information is obtained by calculating the average value of the channel dimension and then adding it to the HSV spatial degree component.

3. The low-light image enhancement method based on an illumination-guided convolutional attention model according to claim 1, characterized in that, The illumination estimator includes a first layer, Convolutional layer and second layer.

4. The low-light image enhancement method based on an illumination-guided convolutional attention model according to claim 3, characterized in that, S2 includes: S21: Concatenate the low-light image with prior lighting information in the channel dimension as input; S22: Through the first The layer fuses prior lighting information into the low-light image; S23: Pass The convolutional layer performs upsampling and further integrates prior lighting information to obtain a lighting feature map with 40 dimensions; S24: Through the second The layer downsamples the output to obtain the lighting map; S25: Perform element-wise multiplication on the illumination map and the dark image to obtain the illumination image.

5. The low-light image enhancement method based on an illumination-guided convolutional attention model according to claim 1, characterized in that, The illumination restorer is based on an illumination-guided convolutional attention framework, which includes an encoder and a decoder. The encoder is responsible for downsampling operations, and the decoder is responsible for upsampling operations.

6. The low-light image enhancement method based on an illumination-guided convolutional attention model according to claim 5, characterized in that, S3 includes: S31: The illumination image is downsampled by the encoder to obtain a feature map and extract image features; S32: Upsample the image features to obtain an illumination restoration map.

7. The low-light image enhancement method based on an illumination-guided convolutional attention model according to claim 6, characterized in that, The enhanced image is obtained by adding the illumination restoration image and the illumination image pixel by pixel.

8. A low-light image enhancement system based on an illumination-guided convolutional attention model, characterized in that, The system is implemented based on a low-light image enhancement method based on an illumination-guided convolutional attention model as described in any one of claims 1-7, and the system includes: Prior construction module: used to input a dark light image and construct prior lighting information; Illumination estimation module: Used to employ an illumination estimator to estimate illumination feature maps and illumination maps based on prior illumination information, and to construct illumination images; Lighting restoration module: Used to process lighting images using a lighting restorer to generate lighting restoration maps; Lighting enhancement module: Used to generate enhanced images based on lighting restoration maps and lighting images.

9. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes a low-light image enhancement method based on an illumination-guided convolutional attention model according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that executes a low-light image enhancement method based on an illumination-guided convolutional attention model according to any one of claims 1-7.

Images