Low-light image color restoration method and system based on multi-scale attention decomposition and adaptive chromaticity correction

Abstract

The invention discloses a low-light image color restoration method and system based on multi-scale attention decomposition and adaptive chromaticity correction, and belongs to the technical field of image processing. The method comprises the following steps: extracting brightness information of a low-illumination image and performing exposure compensation to generate a reference image; inputting and constructing the low-illumination image and the reference image through a scale attention decomposition network to obtain a low-light reflection image, a low-light illumination image, a reference reflection image and a reference illumination image; inputting the obtained low-light reflection image into a self-adaptive chromaticity correction module to adjust the chromaticity to obtain an enhanced reflection image; obtaining an enhanced image through the low-light illumination image and the enhanced reflection image; and performing element-by-element multiplication on the low-illumination image and the enhanced image to obtain a final enhanced image. According to the method, image reflection and illumination information are separated, and balance between image details and the overall illumination effect is ensured through joint optimization training, so that the finally enhanced image is more natural and clear visually.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a low-light image color restoration method and system based on multi-scale attention decomposition and adaptive chromaticity correction. Background Art

[0002] Under low illumination conditions, due to insufficient light, images often exhibit problems such as high noise, blurred details, low contrast, and color distortion. This not only seriously affects the visual quality of the images but also poses great challenges to subsequent target recognition, monitoring, and analysis tasks. Traditional image enhancement methods mainly rely on global or local statistical processing means such as histogram equalization, Gamma correction, and contrast stretching. Although these methods improve the image brightness to a certain extent, they have obvious limitations in detail restoration and color reproduction. Specifically, traditional methods often cannot balance local detail restoration and overall color fidelity in low-light images, and are prone to introducing artifacts or over-enhancement phenomena, thus affecting the realism of the images and subsequent application effects.

[0003] In recent years, the development of deep learning technology has brought new breakthroughs to low-light image processing. Image enhancement methods based on convolutional neural networks (CNNs) have shown excellent performance in image decomposition and reconstruction, especially image decomposition methods based on the Retinex theory. The Retinex theory decomposes an image into a reflectance map and an illumination map, and by processing these two parts of information separately, it improves the local details and overall illumination balance of the image. However, existing Retinex-based decomposition networks still have some problems in practical applications: (1) Detail loss and color bleeding: Due to the limitations of traditional decomposition networks in multi-scale feature extraction and fusion, local detail restoration in the reflectance map is often insufficient, and color bleeding is prone to occur in the illumination map. (2) Limitations in RGB space processing: When directly performing color restoration in the RGB color space, due to the coupling effect between channels, color shift and distortion problems are easily caused, and accurate color reproduction is difficult to achieve. Summary of the Invention

[0004] To address the above problems, the present invention proposes a low-light image color restoration method and system based on multi-scale attention decomposition and adaptive chromaticity correction, aiming to improve the detail retention ability and color fidelity of low-light images and overcome the deficiencies of existing methods.

[0005] To achieve the above technology, specifically as follows:

[0006] A low-light image color restoration method based on multi-scale attention decomposition and adaptive chromaticity correction, comprising the following steps:

[0007] S1. Extract the luminance information of the low-illumination image S from the luminance L channel in the Lab color space, and perform exposure compensation on the pixels with luminance information lower than the exposure threshold using non-linear mapping; in the compensation process, use bilateral filtering technology to generate a reference image S l ; h ;

[0008] Perform histogram analysis on the luminance (extracted from the L channel in the Lab color space) of the low-illumination image, and statistically analyze the luminance distribution. According to the luminance statistical data of the image, adaptively determine the exposure threshold T (set T to 1.2 times the average luminance of a normal image); for pixels below the threshold T, perform exposure compensation using non-linear mapping (logarithmic function) to appropriately increase their luminance while keeping the highlight area from being overexposed. In the compensation process, use bilateral filtering technology so that the edge and texture information is not damaged, and finally generate a reference image that is closer to the target low-illumination image in terms of structure, noise, and color.

[0009] S2. Construct a multi-scale attention decomposition network (MSAD-Net), and input the low-illumination image S l and the reference image S h into the constructed multi-scale attention decomposition network to obtain the low-light reflection map R l , the low-light illumination map L l , the reference reflection map R h and the reference illumination map L h ;

[0010] The multi-scale attention decomposition network (MSAD-Net) includes: 4 cascaded channel-spatial dual attention modules and 3 multi-scale dilated convolutional pyramid layers. Among them, the multi-scale dilated convolutional pyramid layer uses 3 parallel dilated convolutions with different dilation rates; the input image passes through 4 cascaded channel-spatial dual attention modules and 3 multi-scale dilated convolutional pyramid layers in sequence to obtain the decomposition result;

[0011] The channel-spatial dual attention module is used to enhance the feature separation between the reflection map and the illumination map, and the expression is:

[0012]

[0013] In the formula, F all is the input feature map, including: the low-illumination image S l and the reference image S h ; GAP is global average pooling; F grad is the gradient feature map; σ is the Sigmoid function; Conv is the convolution operation; Cat represents channel concatenation;

[0014] The multi-scale dilated convolutional pyramid fusion layer is used to extract and dynamically fuse multi-scale features, and the expression is:

[0015]

[0016] Wherein, DConv d＝i represents dilated convolution with a dilation rate of i; the weight α i is generated by a lightweight multi-layer perceptron (MLP), and α i is a data-dependent parameter related to the input F att-all which will be dynamically adjusted according to the input and is not a fixed constant;

[0017] In a single channel-spatial dual attention module, after the module performs global average pooling operations on the input original feature map respectively, it uses a multi-layer perceptron (MLP) to generate channel attention. After passing through the Sigmoid activation function, at the same time, it extracts the gradient information of the image with channel attention through the Sobel operator to generate gradient features, and then splices the result extracted by the Sobel operator with the original feature map. After passing through a 3×3 convolution and the Sigmoid activation function, the results after the two Sigmoid activation functions are multiplied element by element to form an attention enhanced map F att-all to ensure that image reflection and illumination information are accurately separated;

[0018] When the low-light image S l or the reference image S h is input into the multi-scale attention decomposition network (MSAD-Net), the image passes through the first channel-spatial dual attention module to obtain the first attention enhanced feature F att1 and F att1 is input into the second identical channel-spatial dual attention module to obtain the second attention enhanced feature F att2 and F att2 is input into the third identical channel-spatial dual attention module to obtain the third attention enhanced feature F att3 and F att3 is input into the fourth identical channel-spatial dual attention module to obtain the fourth attention enhanced feature F att4 ; Through the cascade structure, the network can further strengthen the attention to key details and local structures at each layer, so as to obtain a richer and more refined feature representation;

[0019] The fourth attention enhanced feature F att4 is input into the first multi-scale dilated convolution pyramid layer, and three dilated convolutions with different dilation rates of 1, 2, and 3 are used to process it in parallel. The dynamic weight α i of each branch is calculated through a lightweight MLP, and the features of the three scales are weighted and summed to generate the first layer fusion feature F fuse1 ; The first layer fusion feature Ffuse1 The second identical pyramid layer is input to obtain the second-layer fused feature F fuse2 ; The second-layer fused feature F fuse2 is input to the third identical pyramid layer to obtain the third-layer fused feature F fuse3 ;

[0020] The third-layer fused feature F fuse3 passes through a convolutional kernel of size 3×3 to capture local details (such as edges and textures). After passing through the ReLU activation function, an image that separates the inherent color and texture of the object is obtained through the output channels, namely the reflection decomposition result map, including: the low-light reflection map R l and the reference reflection map R h ; Among them, the number of output channels is the same as the number of input channels;

[0021] The third-layer fused feature F fuse3 passes through a convolutional kernel of size 5×5 to smooth the noise and retain the illumination distribution. After passing through the Sigmoid activation function, the output range is limited to [0,1]. An image that extracts the global illumination intensity distribution of the scene is obtained through the output channels, namely the illumination decomposition result map, including: the low-light illumination map L l and the reference illumination map L h ; Among them, the number of output channels is a 1-channel grayscale image because the illumination intensity is independent of color.

[0022] S3. Input the obtained low-light reflection map R l into the adaptive chromaticity correction module to adjust the chromaticity and obtain the enhanced reflection map R' l , and the steps are as follows:

[0023] S3.1. The obtained low-light reflection map R l is passed through the Lab color space to obtain the luminance component L, the red-green chromaticity component a, and the yellow-blue chromaticity component b;

[0024] The red-green chromaticity component a passes through a 3×3 convolutional operation and then passes through the ReLU activation function for output. The result output by the ReLU activation function passes through a 1×1 convolutional operation and then passes through the Sigmoid activation function to output a dynamic mask

[0025] The yellow-blue chromaticity component b passes through a 3×3 convolutional operation and then passes through the ReLU activation function for output. The result output by the ReLU activation function passes through a 1×1 convolutional operation and then passes through the Sigmoid activation function to output a dynamic mask

[0026] The red-green chromaticity component a and the yellow-blue chromaticity component b are corrected using the dynamic mask and the learnable bias. The expression is as follows:

[0027]

[0028] In the formula, is the mask generated for the a-channel attention of the red-green chromaticity component; is the mask generated for the b-channel attention of the yellow-blue chromaticity component; is the learnable bias term of the red-green chromaticity component a; is the learnable bias term of the yellow-blue chromaticity component b; is the element-wise multiplication (Hadamard product), that is, the mask is multiplied by the chromaticity component pixel by pixel;

[0029] S3.2. Use the gamut protection mechanism to constrain the corrected red-green chromaticity component a' and yellow-blue chromaticity component b' to obtain the constrained red-green chromaticity component a' clip and the constrained yellow-blue chromaticity component b' clip , to prevent over-saturation, and the expression is as follows:

[0030] a′ clip = tanh(a′)·γ, γ ∈ [0.8, 1.2]

[0031] b′ clip = tanh(b′)·γ, γ ∈ [0.8, 1.2]

[0032] In the formula, γ is an adjustable scaling factor;

[0033] The role of the constraint is: to protect the gamut boundary and avoid the corrected color exceeding the representable range of the display device (such as the sRGB gamut);

[0034] S3.3. Convert the luminance component L, the constrained red-green chromaticity component a' clip and the constrained yellow-blue chromaticity component b' clip to the RGB space to obtain the enhanced reflection map R' l .

[0035] Conversion process: By calculating the intermediate variables and XYZ components from the luminance component L, the constrained red-green chromaticity component a' clip and the constrained yellow-blue chromaticity component b' clip , the XYZ color space is obtained; then the XYZ color space is converted to the RGB space through a conversion matrix to obtain the enhanced reflection map R' l .

[0036] S4. Multiply the low-light illumination map L l and the enhanced reflection map R' l element-wise to obtain the enhanced image I enhanced ; Multiply the low-illumination image S lAfter the 1×1 convolution operation, it is combined with the enhanced image I enhanced The final enhanced image I is obtained by element-wise multiplication final ;

[0037] The enhanced image I is obtained enhanced The expression of I is as follows:

[0038]

[0039] The final enhanced image I is obtained final The expression of I is as follows:

[0040]

[0041] S5. Through the reference image S h Perform backpropagation training and construct a joint optimization loss function for the collaborative work of each module of the entire system;

[0042] The joint optimization loss function includes: cross-modal reflection consistency loss function, illumination smoothness constraint function, multi-scale contrast learning loss function, and color fidelity loss function; among them, the cross-modal reflection consistency loss function is used to capture the global structural information in the image, so that the reflection map after low-light processing is consistent with the reference reflection map in the overall structure; the illumination smoothness constraint function is used to achieve a smooth transition between the low-light image and the reference image illumination map; the multi-scale contrast learning loss function is used to improve the discrimination ability of different-scale features; the color fidelity loss function is used to limit the difference between the corrected chromaticity and the reference chromaticity;

[0043] The steps are as follows:

[0044] S5.1. Through the low-light reflection map R l and the reference reflection map R h , construct the cross-modal reflection consistency loss function, and the expression is as follows:

[0045]

[0046] In the formula, SSIM represents the Structural Similarity Index Measure;

[0047] S5.2. Through the low-light illumination map L l and the reference illumination map L h , construct the illumination smoothness constraint function, and the expression is as follows:

[0048]

[0049] In the formula, represents the Fourier transform;

[0050] S5.3. Construct a multi-scale contrast learning loss function for local feature extraction of the low-light image reflection map and the reference image reflection map. The expression is as follows:

[0051]

[0052] In the formula, S' represents all scales for local feature extraction of the low-light image reflection map and the reference image reflection map, S' = 4; s represents the scale for local feature extraction of the low-light image reflection map and the reference image reflection map, s ∈ S'; v s represents the anchor feature vector at scale s, that is, the currently selected image; represents the positive sample feature vector that is semantically similar to the anchor feature vector v s ; K' represents the set of negative sample feature vectors, K' = 30; k represents the k-th negative sample feature vector, k ∈ K'; represents the k-th negative sample feature vector at scale s; τ represents the proportionality coefficient used to adjust the smoothness of the contrast loss;

[0053] S5.4. Use the corrected red-green chromaticity component a' and yellow-blue chromaticity component b' and the red-green chromaticity component a h and yellow-blue chromaticity component b h of the reference image S h to construct a color fidelity loss function. The expression is as follows:

[0054] L color = ||a' - a h ||1 + ||b' - b h ||1;

[0055] S5.5. Construct a joint optimization loss function through the cross-modal reflection consistency loss function, the illumination smoothness constraint function, the multi-scale contrast learning loss function, and the color fidelity loss function. The expression is as follows:

[0056] L total = L cm + λ1L smooth + λ2L color + λ3L mscl

[0057] In the formula, λ1 represents the weight coefficient of the illumination smoothness constraint function; λ2 represents the weight coefficient of the color fidelity loss function; λ3 represents the weight coefficient of the multi-scale contrast learning loss function; among them, λ1 = 0.2, λ2 = 0.6, λ3 = 0.3.

[0058] A low - illumination image color restoration system based on multi - scale attention decomposition and adaptive chromaticity correction, comprising: an adaptive exposure compensation processing module, a multi - scale attention decomposition network module (MSAD - Net), an adaptive chromaticity correction module (ACC), a light restoration module, and a joint loss calculation module;

[0059] The adaptive exposure compensation processing module is used to perform the following steps:

[0060] S1. Extract the luminance information of the low - illumination image S from the luminance L channel in the Lab color space, and perform exposure compensation on the pixels whose luminance information is lower than the exposure threshold using non - linear mapping; during the compensation process, use the bilateral filtering technique to generate the reference image S l ; h ;

[0061] The multi - scale attention decomposition network module (MSAD - Net) is used to perform the following steps:

[0062] S2. Construct a multi - scale attention decomposition network (MSAD - Net), input the low - illumination image S l and the reference image S h into the constructed multi - scale attention decomposition network to obtain the low - light reflection map R l , the low - light illumination map L l , the reference reflection map R h and the reference illumination map L h ;

[0063] The adaptive chromaticity correction module (ACC) is used to perform the following steps:

[0064] S3. Input the obtained low - light reflection map R l into the adaptive chromaticity correction module to adjust the chromaticity and obtain the enhanced reflection map R' l ;

[0065] The light restoration module is used to perform the following steps:

[0066] S4. Multiply the low - light illumination map L l and the enhanced reflection map R' l element - by - element to obtain the enhanced image I enhanced ; After the low - illumination image S l passes through a 1×1 convolution operation, multiply it with the enhanced image I enhanced element - by - element to obtain the final enhanced image I final ;

[0067] The joint loss calculation module is used to perform the following steps:

[0068] S5. Through the reference image S hPerform backpropagation training and construct a joint optimization loss function for the collaborative work of each module in the entire system.

[0069] Advantages of the present invention:

[0070] The present invention separates image reflection and illumination information, thus enabling better restoration of details in low-illumination images; independently performs chromaticity correction in the Lab color space, significantly reducing color deviation caused by RGB channel coupling; at the same time, through global residual connection, that is, the method of extracting high-dimensional features after 1×1 convolution processing of the original low-illumination image and superimposing them with the original low-illumination image in an element-wise addition manner to generate the final enhanced image and joint optimization training, ensuring the balance between image details and overall illumination effect, making the final enhanced image more natural and clear visually.

[0071] Due to the adoption of a lightweight network structure and module design in the present invention, this method has high computational efficiency and meets the requirements of mobile terminals and real-time processing. Brief Description of the Drawings

[0072] Figure 1 is the flowchart of the steps of the present invention;

[0073] Figure 2 is the schematic diagram of the overall system architecture of the present invention;

[0074] Figure 3 is the schematic diagram of the channel-spatial dual attention module structure of the present invention;

[0075] Figure 4 is the schematic diagram of the multi-scale dilated convolution pyramid layer structure of the present invention;

[0076] Figure 5 is the schematic diagram of the adaptive chromaticity correction module (ACC) of the present invention;

[0077] Figure 6 is the schematic diagram of image fusion of the present invention;

[0078] Figure 7 is the comparison chart of the image color restoration results of the target area 1 in the embodiment of the present invention. Among them, part a is the low-illumination image, part b is the color restoration result of the present invention, and part c is the color restoration result of the Retinex algorithm;

[0079] Figure 8 is the comparison chart of the image color restoration results of the target area 2 in the embodiment of the present invention. Among them, part a is the low-illumination image, part b is the color restoration result of the present invention, and part c is the color restoration result of the Retinex algorithm. Detailed Description of the Embodiment

[0080] The present invention will be further described in detail below with reference to specific embodiments.

[0081] As Figure 1 and Figure 2 shown, a low - illumination image color restoration method based on multi - scale attention decomposition and adaptive chromaticity correction includes the following steps:

[0082] S1. Extract the luminance information of the low - illumination image S from the luminance L channel in the Lab color space, and perform exposure compensation on the pixels whose luminance information is lower than the exposure threshold using non - linear mapping; during the compensation process, bilateral filtering technology is used to generate the reference image S l ; h ;

[0083] Perform histogram analysis on the luminance (extracted from the L channel in the Lab color space) of the low - illumination image, and statistically analyze the luminance distribution. According to the luminance statistical data of the image, adaptively determine the exposure threshold T (set T to 1.2 times the average luminance of a normal image); for the pixels below the threshold T, perform exposure compensation using non - linear mapping (logarithmic function) to appropriately increase their luminance while keeping the highlight area from being over - exposed. During the compensation process, bilateral filtering technology is used so that the edge and texture information are not damaged, and finally a reference image closer to the target low - illumination image in terms of structure, noise, and color is generated.

[0084] S2. Construct a multi - scale attention decomposition network (MSAD - Net), and input the low - illumination image S l and the reference image S h into the constructed multi - scale attention decomposition network to obtain the low - light reflection map R l , the low - light illumination map L l , the reference reflection map R h and the reference illumination map L h ;

[0085] The multi - scale attention decomposition network (MSAD - Net) includes: 4 cascaded channel - spatial dual - attention modules and 3 multi - scale dilated convolutional pyramid layers. Among them, the multi - scale dilated convolutional pyramid layer uses 3 parallel dilated convolutions with different dilation rates; the input image passes through 4 cascaded channel - spatial dual - attention modules and 3 multi - scale dilated convolutional pyramid layers in sequence to obtain the decomposition result;

[0086] The channel - spatial dual - attention module is used to enhance the feature separation between the reflection map and the illumination map, and the expression is:

[0087]

[0088] In the formula, F all is the input feature map, including: the low - illumination image S l and the reference image S h ; GAP is global average pooling; F gradis the gradient feature map; σ is the Sigmoid function; Conv is the convolution operation; Cat represents channel concatenation;

[0089] The multi-scale dilated convolution pyramid fusion layer is used to extract and dynamically fuse multi-scale features, and the expression is:

[0090]

[0091] In the formula, DConv d＝i represents the dilated convolution with a dilation rate of i; the weight α i is generated by a lightweight multi-layer perceptron (MLP), and α i is a data-dependent parameter related to the input F att-all They will be dynamically adjusted according to the input and are not fixed constants;

[0092] As Figure 3 shown, in a single channel-spatial dual attention module, after the module performs global average pooling operations on the input original feature map respectively, a channel attention is generated using a multi-layer perceptron (MLP). After passing through the Sigmoid activation function, the gradient information of the image with channel attention is extracted using the Sobel operator to generate the gradient feature. Then, the result extracted by the Sobel operator is concatenated with the original feature map. After 3×3 convolution and the Sigmoid activation function, the results after the two Sigmoid activation functions are subjected to element-wise multiplication to form the attention-enhanced map F att-all , thus ensuring that the image reflection and illumination information are accurately separated;

[0093] When the low-light image S l or the reference image S h is input into the multi-scale attention decomposition network (MSAD-Net), the image passes through the first channel-spatial dual attention module to obtain the first attention-enhanced feature F att1 , and F att1 is input into the second identical channel-spatial dual attention module to obtain the second attention-enhanced feature F att2 , and F att2 is input into the third identical channel-spatial dual attention module to obtain the third attention-enhanced feature F att3 , and F att3 is input into the fourth identical channel-spatial dual attention module to obtain the fourth attention-enhanced feature F att4 ; through the cascade structure, the network can further strengthen the attention to key details and local structures at each layer, thereby obtaining a richer and more refined feature representation;

[0094] As Figure 4 shown, the fourth attention-enhanced feature F att4Input the first multi-scale dilated convolutional pyramid layer. Use three dilated convolutions with different dilation rates of 1, 2, and 3 to process it in parallel, and calculate the dynamic weight α of each branch through a lightweight MLP. i Weightedly sum the features of the three scales to generate the first-layer fused feature F. fuse1 ; Input the first-layer fused feature F fuse1 into the second identical pyramid layer to obtain the second-layer fused feature F. fuse2 ; Input the second-layer fused feature F fuse2 into the third identical pyramid layer to obtain the third-layer fused feature F. fuse3 ;

[0095] The third-layer fused feature F fuse3 passes through a 3×3 convolutional kernel to capture local details (such as edges and textures). After passing through the ReLU activation function, an image that separates the inherent color and texture of the object is obtained through the output channels, that is, the reflection decomposition result map, including: the low-light reflection map R l and the reference reflection map R h ; Among them, the number of output channels is the same as the number of input channels;

[0096] The third-layer fused feature F fuse3 passes through a 5×5 convolutional kernel to smooth the noise and retain the illumination distribution. After passing through the Sigmoid activation function, the output range is restricted to [0,1]. An image that extracts the global illumination intensity distribution of the scene is obtained through the output channels, that is, the illumination decomposition result map, including: the low-light illumination map L l and the reference illumination map L h ; Among them, the number of output channels is a 1-channel grayscale image because the illumination intensity is independent of color.

[0097] S3. As Figure 5 shown, input the obtained low-light reflection map R l into the adaptive chromaticity correction module to adjust the chromaticity and obtain the enhanced reflection map R'. l , the steps are as follows:

[0098] S3.1. Pass the obtained low-light reflection map R l through the Lab color space to obtain the luminance component L, the red-green chromaticity component a, and the yellow-blue chromaticity component b;

[0099] The red-green chromaticity component a passes through a 3×3 convolutional operation and then outputs through the ReLU activation function, and the result output by the ReLU activation function passes through a 1×1 convolutional operation and then outputs through the Sigmoid activation function to obtain the dynamic mask.

[0100] The yellow - blue chrominance component b is output through the ReLU activation function after a 3×3 convolution operation, and the result output by the ReLU activation function is output through a 1×1 convolution operation and then through the Sigmoid activation function to obtain a dynamic mask.

[0101] The red - green chrominance component a and the yellow - blue chrominance component b are corrected using the dynamic mask and the learnable bias, and the expression is as follows:

[0102]

[0103] In the formula, is the mask generated by the a - channel attention of the red - green chrominance component a; is the mask generated by the b - channel attention of the yellow - blue chrominance component b; is the learnable bias term of the red - green chrominance component a; is the learnable bias term of the yellow - blue chrominance component b; is the element - wise multiplication (Hadamard product), that is, the mask is multiplied by the chrominance component pixel - by - pixel;

[0104] S3.2. Use the gamut protection mechanism to constrain the corrected red - green chrominance component a' and yellow - blue chrominance component b' to obtain the constrained red - green chrominance component a' clip and the constrained yellow - blue chrominance component b' clip , to prevent over - saturation, and the expression is as follows:

[0105] a′ clip =tanh(a′)·γ, γ ∈ [0.8, 1.2]

[0106] b′ clip =tanh(b′)·γ, γ ∈ [0.8, 1.2]

[0107] In the formula, γ is an adjustable scaling factor;

[0108] The role of the constraint is to protect the gamut boundary and prevent the corrected color from exceeding the representable range of the display device (such as the sRGB gamut);

[0109] S3.3. Convert the luminance component L, the constrained red - green chrominance component a' clip and the constrained yellow - blue chrominance component b' clip to the RGB space to obtain the enhanced reflection map R' l .

[0110] The conversion process: By combining the luminance component L, the constrained red - green chrominance component a' clip and the constrained yellow - blue chrominance component b' clipCalculate intermediate variables and XYZ components to obtain the XYZ color space; then convert the XYZ color space to the RGB space through a conversion matrix to obtain the enhanced reflection map R'. l .

[0111] S4. As Figure 6 shown, perform element-wise multiplication on the low-light illumination map L l and the enhanced reflection map R' l to obtain the enhanced image I enhanced ; after subjecting the low-illumination image S l to a 1×1 convolution operation, perform element-wise multiplication with the enhanced image I enhanced to obtain the final enhanced image I final ;

[0112] Obtain the enhanced image I enhanced The expression is as follows:

[0113]

[0114] Obtain the final enhanced image I final The expression is as follows:

[0115]

[0116] S5. Through the reference image S h perform backpropagation training and construct a joint optimization loss function for the collaborative work of each module of the entire system;

[0117] The joint optimization loss function includes: a cross-modal reflection consistency loss function, an illumination smoothness constraint function, a multi-scale contrast learning loss function, and a color fidelity loss function; among them, the cross-modal reflection consistency loss function is used to capture the global structural information in the image, so that the reflection map after low-illumination processing is consistent with the reference reflection map in the overall structure; the illumination smoothness constraint function is used to achieve a smooth transition between the low-light image and the reference image illumination map; the multi-scale contrast learning loss function is used to improve the discrimination ability of different-scale features; the color fidelity loss function is used to limit the difference between the corrected chromaticity and the reference chromaticity;

[0118] The steps are as follows:

[0119] S5.1. Through the low-light reflection map R l and the reference reflection map R h , construct a cross-modal reflection consistency loss function, and the expression is as follows:

[0120]

[0121] In the formula, SSIM represents the Structural Similarity Index Measure;

[0122] S5.2. Construct a lighting smoothness constraint function through the low-light illumination map L l and the reference illumination map L h , and the expression is as follows:

[0123]

[0124] In the formula, F represents the Fourier transform;

[0125] S5.3. Construct a multi-scale contrast learning loss function for local feature extraction of the low-light image reflection map and the reference image reflection map, and the expression is as follows:

[0126]

[0127] In the formula, S' represents all scales for local feature extraction of the low-light image reflection map and the reference image reflection map, S' = 4; s represents the scale for local feature extraction of the low-light image reflection map and the reference image reflection map, s ∈ S'; v s represents the anchor feature vector at scale s, that is, the current image taken; represents the positive sample feature vector that is semantically similar to the anchor feature vector vs; K' represents the set of negative sample feature vectors, K' = 30; k represents the k-th negative sample feature vector, k ∈ K'; represents the k-th negative sample feature vector at scale s; τ represents the proportionality coefficient used to adjust the smoothness of the contrast loss;

[0128] S5.4. Use the corrected red-green chromaticity component a' and yellow-blue chromaticity component b' and the red-green chromaticity component a h of the reference image S h and the yellow-blue chromaticity component b h to construct a color fidelity loss function, and the expression is as follows:

[0129] L color = ||a′ - a h ||1 + ||b′ - b h ||1;

[0130] S5.5. Construct a joint optimization loss function through the cross-modal reflection consistency loss function, the lighting smoothness constraint function, the multi-scale contrast learning loss function, and the color fidelity loss function, and the expression is as follows:

[0131] L total = L cm + λ1L smooth+λ2L color +λ3L mscl

[0132] Wherein, λ1 represents the weight coefficient of the illumination smoothness constraint function; λ2 represents the weight coefficient of the color fidelity loss function; λ3 represents the weight coefficient of the multi-scale contrast learning loss function; among them, λ1 = 0.2, λ2 = 0.6, λ3 = 0.3.

[0133] A low-light image color restoration system based on multi-scale attention decomposition and adaptive chromaticity correction, comprising: an adaptive exposure compensation processing module, a multi-scale attention decomposition network module (MSAD-Net), an adaptive chromaticity correction module (ACC), an illumination restoration module, and a joint loss calculation module;

[0134] The adaptive exposure compensation processing module is used to perform the following steps:

[0135] S1. Extract the luminance information of the low-light image S from the luminance L channel in the Lab color space, and perform exposure compensation on the pixels with luminance information lower than the exposure threshold using non-linear mapping; during the compensation process, use the bilateral filtering technique to generate the reference image S l ; h ;

[0136] The multi-scale attention decomposition network module (MSAD-Net) is used to perform the following steps:

[0137] S2. Construct a multi-scale attention decomposition network (MSAD-Net), and input the low-light image S l and the reference image S h into the constructed multi-scale attention decomposition network to obtain the low-light reflection map R l , the low-light illumination map L l , the reference reflection map R h and the reference illumination map L h ;

[0138] The adaptive chromaticity correction module (ACC) is used to perform the following steps:

[0139] S3. Input the obtained low-light reflection map R l into the adaptive chromaticity correction module to adjust the chromaticity and obtain the enhanced reflection map R' l ;

[0140] The illumination restoration module is used to perform the following steps:

[0141] S4. Multiply the low-light illumination map L l and the enhanced reflection map R' l element-wise to obtain the enhanced image I enhanced ; The low-light image S lAfter the 1×1 convolution operation, it is combined with the enhanced image I enhanced The final enhanced image I is obtained by element-wise multiplication final ;

[0142] The joint loss calculation module is used to perform the following steps:

[0143] S5. Through the reference image S h Backpropagation training is carried out and a joint optimization loss function is constructed for the collaborative work of each module of the whole system.

[0144] To verify the results of the present invention, a result comparison is made with the Retinex algorithm, as Figure 7 and Figure 8 shown. In Figure 7 and Figure 8 , the a part is the low-illumination image, the b part is the color restoration result of the present invention, and the c part is the color restoration result of the existing Retinex algorithm. It can be seen from Figure 7 and Figure 8 that the effect of the present invention is closer to the normal picture in color restoration than the result of the existing Retinex algorithm.

[0145] The specific embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the knowledge scope of those of ordinary skill in the art, various changes can be made without departing from the gist of the present invention.​​​​

Claims

1. A low-light image color restoration method based on multi-scale attention decomposition and adaptive chromaticity correction, characterized in that Including the following steps: S1. Extract the luminance information of the low-illumination image S from the luminance L channel in the Lab color space, and perform exposure compensation on the pixels whose luminance information is lower than the exposure threshold using non-linear mapping; generate a reference image S using bilateral filtering technology during the compensation process l ; h ; S2. Construct a multi-scale attention decomposition network and input the low-illumination image S l and the reference image S h into the constructed multi-scale attention decomposition network to obtain the low-light reflection map R l , the low-light illumination map L l , the reference reflection map R h and the reference illumination map L h ; S3. Input the obtained low-light reflection map R l into the adaptive chromaticity correction module to adjust the chromaticity and obtain an enhanced reflection map R'. l ; S4. Multiply the low-light illumination image L l and the enhanced reflection image R' l element-wise to obtain the enhanced image I enhanced ; After subjecting the low-illumination image S l to a 1×1 convolution operation, multiply it with the enhanced image I enhanced element-wise to obtain the final enhanced image I final ; S5. Through the reference image S h Perform backpropagation training and construct a joint optimization loss function for the collaborative work of each module of the entire system.

2. A low-light image color restoration method based on multi-scale attention decomposition and adaptive chromaticity correction according to claim 1, characterized in that The multi-scale attention decomposition network includes: 4 cascaded channel-spatial dual attention modules and 3 multi-scale dilated convolutional pyramid layers; among them, the multi-scale dilated convolutional pyramid layer adopts 3 parallel dilated convolutions with different dilation rates of 1, 2, and 3.

3. A low - illumination image color restoration method based on multi - scale attention decomposition and adaptive chromaticity correction according to claim 1, characterized in that, The obtained low-light reflection map R l is input into an adaptive chromaticity correction module to adjust the chromaticity, and an enhanced reflection map R' is obtained l , and the steps are as follows: S3.

1. Obtain the resulting low-light reflection map R l In the Lab color space, obtain the luminance component L, the red-green chrominance component a, and the yellow-blue chrominance component b; The red-green chrominance component a is output through the ReLU activation function after a 3×3 convolution operation, and the result output by the ReLU activation function is output as a dynamic mask through a 1×1 convolution operation followed by the Sigmoid activation function The yellow-blue chrominance component b is output through a ReLU activation function after a 3×3 convolution operation, and the result output by the ReLU activation function is output as a dynamic mask through a 1×1 convolution operation followed by a Sigmoid activation function. Using a dynamic mask and a learnable bias to correct the red-green chromaticity component a and the yellow-blue chromaticity component b; S3.

2. Use the color gamut protection mechanism to constrain the corrected red-green chromaticity component a' and yellow-blue chromaticity component b' to obtain the constrained red-green chromaticity component a' clip and the constrained yellow-blue chromaticity component b' clip to prevent oversaturation; S3.

3. Convert the luminance component L, the constrained red-green chrominance component a', clip and the constrained yellow-blue chrominance component b' clip to the RGB space to obtain the enhanced reflection map R'. l ; Conversion process: By using the luminance component L, the constrained red-green chrominance component a' clip and the constrained yellow-blue chrominance component b' clip calculate the intermediate variables and XYZ components to obtain the XYZ color space; Then, the XYZ color space is converted into the RGB space through a conversion matrix to obtain an enhanced reflection map R'. l .

4. A low-light image color restoration method based on multi-scale attention decomposition and adaptive chromaticity correction according to claim 1, characterized in that, The above-mentioned reference image S h is used for backpropagation training to construct a joint optimization loss function for the optimal collaborative work of each module of the entire system. The joint optimization loss function includes: a cross-modal reflection consistency loss function, a lighting smoothness constraint function, a multi-scale contrast learning loss function, and a color fidelity loss function; The steps for constructing the joint optimization loss function are as follows: S5.

1. Construct a cross-modal reflection consistency loss through the low-light reflection map R l and the reference reflection map R h . S5.

2. Construct a lighting smoothness constraint function through the low-light illumination map L l and the reference illumination map L h . S5.

3. Construct a multi-scale contrast learning loss function for local feature extraction of the low-light image reflection map and the reference image reflection map; S5.

4. Use the corrected red-green chromaticity component a' and yellow-blue chromaticity component b' and the red-green chromaticity component a h of the reference image S h and the yellow-blue chromaticity component b h to construct a color fidelity loss function; S5.

5. Construct a joint optimization loss function through a cross-modal reflection consistency loss function, a lighting smoothness constraint function, a multi-scale contrast learning loss function, and a color fidelity loss function.

5. A low-light image color restoration system based on multi-scale attention decomposition and adaptive chromaticity correction, characterized in that, The system includes: an adaptive exposure compensation processing module, a multi-scale attention decomposition network module, an adaptive chromaticity correction module, a lighting restoration module, and a joint loss calculation module; The adaptive exposure compensation processing module is used to perform the following steps: S1. Extract the luminance information of the low-illumination image S from the luminance L channel in the Lab color space, and perform exposure compensation on the pixels with luminance information lower than the exposure threshold using non-linear mapping; use the bilateral filtering technique to generate a reference image S during the compensation process l ; h ; The multi-scale attention decomposition network module is used to perform the following steps: S2. Construct a multi-scale attention decomposition network and input the low-illumination image S l and the reference image S h into the constructed multi-scale attention decomposition network to obtain the low-light reflection map R l , the low-light illumination map L l , the reference reflection map R h and the reference illumination map L h ; The adaptive chromaticity correction module is used to perform the following steps: S3. Input the obtained low-light reflection map R l into the adaptive chromaticity correction module to adjust the chromaticity and obtain an enhanced reflection map R'. l ; The lighting restoration module is used to perform the following steps: S4. Multiply the low-light illumination image L l and the enhanced reflection image R' l element-wise to obtain the enhanced image I enhanced ; After subjecting the low-illumination image S l to a 1×1 convolution operation, multiply it element-wise with the enhanced image I enhanced to obtain the final enhanced image I final ; The joint loss calculation module is used to perform the following steps: S5. Through the reference image S h Perform backpropagation training and construct a joint optimization loss function for the optimal collaborative work of each module of the entire system.

Images