Low-illumination image enhancement method based on Retinex theory and guided by color prior

Abstract

The invention discloses a low-illumination image enhancement method based on a Retinex theory and guided by color priori, a low-illumination image enhancement network model based on the Retinex theory and guided by the color priori is constructed for low-illumination image enhancement, the model is based on the Retinex theory, the color priori is generated through a color priori generation module, and the low-illumination image enhancement network model is used for enhancing the low-illumination image. The color priori and the original image are spliced to form a combined image to guide low-illumination image enhancement, the pixel value of the image is improved through an image illumination enhancement network, and an illumination image and a feature image are generated. The illumination image is input into a denoising network to generate a denoising component, the denoising network is provided with an attention module guided by illumination, and the attention module introduces a multi-scale double-gating feed-forward network, so that the denoising network realizes image denoising on different scales, and key content of the image is highlighted. According to the low-illumination image enhancement method provided by the invention, the problems that part of the enhanced region in the enhanced image deviates from the original color and the noise is amplified in the process of enhancing the low-illumination image are solved.

Description

Technical Field

[0001] The invention belongs to the technical field of low-light image enhancement, and in particular relates to a low-light image enhancement method guided by color prior based on Retinex theory. Background Art

[0002] Images taken at night are usually low-light images, with problems such as insufficient illumination, missing details and noise. These problems not only affect the visual quality of the image, but also affect subsequent computer vision tasks such as target tracking and face recognition. Therefore, low-light image enhancement technology has increasingly important theoretical and application value, and low-light image enhancement technology has become an important research direction in the field of computer vision. The purpose of low-light image enhancement technology is to effectively improve image brightness, enhance image contrast, eliminate the influence of image noise, improve image details, maintain image color balance, and ultimately improve the overall quality of the image.

[0003] Traditional low-light image enhancement technology based on Retinex theory often improves the pixel value of the image through histogram equalization or gamma correction to achieve the purpose of low-light image enhancement. The Chinese invention with publication number CN117994155A discloses a dual-branch low-light image enhancement method based on Retinex theory, which decomposes the low-light image into an illumination component and a reflection component, initializes the feature extraction of the input low-light image through parallel hole convolutions with different expansion rates, and performs deeper feature extraction on the initialized image through the enhancement unit in the iterative learning module and the U-Net network. While denoising the reflection component, the illumination component is improved. After three iterative learnings, the illumination component obtained is fused with the reflection component to achieve the purpose of denoising during the enhancement process. The disadvantage of this method is that the actual meaning of the image content is ignored when enhancing the low-light image, resulting in the deviation of the enhanced area of ​​the image from the original color. The Chinese invention with publication number CN118247164A discloses a low-light image enhancement network and an image enhancement method. After decomposing the low-light image into illumination components and reflection components, the method uses a coding-decoding structure to process the noise in the reflection map, and finally fuses the illumination map with the denoised reflection map as the input of the enhancement network. The spatial information lost in the pooling layer is transferred using CTrans, and the complete spatial resolution is restored through the coding-decoding process. Upsampling or downsampling is achieved through 4 consecutive RDBs, and each upsampling or downsampling operation is bypassed by an additional jump connection to bypass redundant low-frequency information, so that the network focuses on learning high-frequency information. The downsampling process uses stride convolution to convert the noisy image from a high-resolution scale to a low-resolution scale, and the upsampling process uses sub-pixel convolution to convert the low-resolution scale back to a high-resolution scale, thereby removing noise and retaining important features of the input image. The disadvantage of this method is that the actual meaning of the image content is ignored when enhancing the low-light image, resulting in the enhanced area deviating from the original color.

[0004] Analysis of the existing technology shows that the deep learning low-light image enhancement method based on Retinex theory usually divides the low-light image into an illumination component and a reflection component. At the beginning, the reflection component is directly used as the enhancement effect. Later, the illumination distribution of the illumination component is improved, and the noise in the reflection component is removed, and finally the enhancement effect is achieved by fusion. The method of directly increasing the pixel value is often accompanied by the problems of overexposure and high image noise. Specifically, due to the lack of understanding of the image content during processing, the algorithm often enhances the dark area in the image, which may cause some problems. For example, a person's black hair is treated as a low-light area, which causes the problem that some enhanced areas of the image deviate from the original color. In addition, when some methods enhance low-light images, denoising is performed first and then enhancement is performed, resulting in the problem of missing details in the enhanced image. In summary, the current low-light image enhancement method based on deep learning has the problem that some enhanced areas in the enhanced image deviate from the original color, and the noise problem that occurs during the low-light image enhancement process is not well solved. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention is implemented by the following technical solutions:

[0006] The present invention proposes a low-light image enhancement method guided by color prior based on Retinex theory, comprising the following steps:

[0007] S20, obtaining a test sample set A and a training sample set B, wherein both the test sample set A and the training sample set B are provided with a plurality of original low-light images from different data sources;

[0008] S40. Construct a low-light image enhancement network model guided by color prior based on Retinex theory, including:

[0009] S402, constructing a color priori generation module, wherein the color priori generation module can re-divide the received input image I according to the RGB pixel values ​​based on the preset proportional coefficients α, β, and γ to generate a color priori I p , the color prior I p Concatenate with input image I to form combined image I R ;

[0010] S404: construct an image illumination enhancement network D, which can extract the combined image I R The global features of the feature image F lu , and based on the feature image F lu Get the illumination map Based on Retinex theory, the input image I and the illumination mapping Perform pixel-by-pixel multiplication to output the illumination image I lu To the denoising network;

[0011] S406, constructing a denoising network, wherein the denoising network includes a plurality of illumination-guided attention modules (IGABs), wherein the IGABs are provided with a regularization layer, an illumination-guided multi-head self-attention module, a regularization layer, and a multi-scale dual-gated feedforward network connected in sequence, wherein the illumination-guided multi-head self-attention module can enhance the long-range dependency of the received image, and the multi-scale dual-gated feedforward network can copy the received image and input it into two branches respectively, wherein one branch can extract detail information of the received image, and the other branch can extract information of adjacent pixels of the received image, highlighting important pixel information in both branches, performing feature enhancement on the images of the two branches and then performing feature fusion, and outputting a fused image after fusion;

[0012] S60, using training sample set B to train the model, calculating the loss of the enhanced image and the real image obtained after the original low-light image is trained by the model according to the mean absolute error loss function, and updating the weight of the model according to the loss until the preset training condition is met;

[0013] S80. Obtain enhancement results of the original low-light images in the test sample set A through the model.

[0014] In one embodiment of the present invention, S402, constructing a color priori generation module, includes:

[0015] The color prior generation module includes an input layer and a color prior generation network. The input layer sorts the pixels of the input image I in descending order according to the RGB pixel values; the color prior generation network re-divides the pixels of the input image I in descending order according to preset proportional coefficients α, β, and γ to generate a color prior I p ; Set the color prior I p After the input image I is connected in series through the channel, the output combined image I R , wherein the preset proportional coefficients α, β, and γ correspond to the maximum value, the middle value, and the minimum value of the RGB pixel values ​​in descending order, respectively.

[0016] In one embodiment of the present invention, the color prior generation network re-divides the pixels of the input image I in descending order according to the preset proportional coefficients α, β, and γ to generate the color prior I p ,include:

[0017] The preset proportional coefficients α, β, and γ of the highest pixel value, the middle pixel value, and the minimum pixel value in the RGB channel of the pixel point at the random position are determined by experimental analysis. The pixels of the input image I are re-divided according to the preset proportional coefficients α, β, and γ to generate a new color prior I with a size of H×W×3 p:

[0018] I p (x,y,F(x,y))=αI(x,y,F(x,y)) (4)

[0019] I p (x,y,S(x,y))=βI(x,y,S(x,y)) (5)

[0020] I p (x,y,L(x,y))=γI(x,y,L(x,y)) (6)

[0021] Among them, I(x,y,F(x,y)), I(x,y,S(x,y)), and I(x,y,L(x,y)) represent the highest, middle, and minimum pixel values ​​in the RGB channels of the pixel at a random position of the input image I, respectively. p (x,y,F(x,y)),I p (x,y,S(x,y)),I p (x, y, L(x, y)) is the pixel value after the highest, middle, and minimum pixel values ​​in the RGB channels of the pixel point at the random position are re-divided based on the preset proportional coefficients α, β, and γ respectively;

[0022] The pixel values ​​of all pixels in the input image I are re-divided into a new image space of size H×W×3 as the color prior I p , where H represents the height of the input image I and W represents the width of the input image I.

[0023] In one embodiment of the present invention, S404, constructing an image illumination enhancement network D, includes:

[0024] The image illumination enhancement network D includes a 1×1 convolutional layer and a 9×9 depth-separable convolutional layer, wherein the combined image I R As input, the image I is enhanced by a 1×1 convolution layer. R The 9×9 depthwise separable convolution receives the output of the 1×1 convolution layer and extracts the combined image I R The global features in get the feature image F lu ;

[0025] The extracted feature image F lu Copy it into two feature images F lu , one of the feature images F lu Output to the denoising network;

[0026] The image illumination enhancement network D also includes a 1×1×3 convolution kernel, and another feature image F luAs the input of the 1×1×3 convolution kernel, we get the illumination map

[0027] Based on the Retinex theory, the input image I is decomposed into a reflection component R and an illumination component L. Multiply the input image I pixel by pixel to generate and output the illumination image I lu , the illumination image I lu It is characterized by the reflection component R and all interference factors Z.

[0028] In one embodiment of the present invention, the input image I is decomposed into a reflection component R and an illumination component L based on the Retinex theory, and the illumination mapping Multiply the input image I pixel by pixel to generate and output the illumination image I lu ,include:

[0029] Step 1: Decompose the input image I into the reflection component R and the illumination component L according to the Retinex theory:

[0030] I=R×L (7)

[0031] Step 2: According to formula (7), considering the interference factors in the reflection component R and the illumination component L of the input image I, the reflection component R and the illumination component L of the input image I considering the interference factors are obtained:

[0032]

[0033] in, and They represent the interference factors of the reflection component R and the interference factors of the illumination component L respectively;

[0034] Step 3: Map the input image I with the interference factor Multiply them to get the illumination image I lu :

[0035]

[0036] Among them, according to The illumination image I lu It is characterized by two parts consisting of the reflection component R and all interference factors Z:

[0037]

[0038] In one embodiment of the present invention, S406, constructing a denoising network, includes:

[0039] S4062, construct a multi-scale dual-gated feedforward network;

[0040] S4064, constructing an illumination-guided attention module (IGAB) based on a multi-scale dual-gated feed-forward network;

[0041] S4066, build a denoising network based on IGAB;

[0042] The denoising network includes three downsampling convolution layers, the convolution kernel sizes of the three downsampling convolution layers are 3×3, 4×4, and 4×4, respectively, and the step sizes are 1, 2, and 2, respectively. lu As the input of the denoising network, the input of the first downsampling convolutional layer receives the illumination image I as the input of the denoising network. lu , the first IGAB module is connected between the first downsampling convolution layer and the second downsampling convolution layer, the second IGAB module is connected between the second downsampling convolution layer and the third downsampling convolution layer, and the third IGAB module is connected after the third downsampling convolution layer. The result is input to the deconvolution layer with a size of 2×2 and a step size of 2 to obtain the first deconvolution result. The result obtained by the second IGAB module is spliced ​​with the first deconvolution result. After the splicing is completed, the first spliced ​​image is obtained. The first spliced ​​image is input to the convolution layer with a convolution kernel size of 1×1. The denoising network also includes a fourth IGAB module, a 1×1 convolution kernel. The result of the convolution layer output is used as the input of the fourth IGAB module, and the output result of the fourth IGAB module is input to the deconvolution layer with a size of 2×2 and a step size of 2 to obtain a second deconvolution result. The result obtained by the first IGAB module is spliced ​​with the second deconvolution result to obtain a second spliced ​​image, and the second spliced ​​image is input to the convolution layer with a convolution kernel size of 1×1. The denoising network also includes a fifth IGAB module, and the result of the second spliced ​​image output by the convolution layer with a convolution kernel size of 1×1 is used as the input of the fifth IGAB module, and the fifth IGAB module is output to the convolution layer with a size of 3×3 and a step size of 1 to form a denoising component I re , the denoised component I re The illumination image I generated by the image illumination enhancement network D lu The final enhanced image I is obtained by summing each element en , wherein the first IGAB module includes one IGAB, and the second to fifth IGAB modules are each provided with two connected IGABs.

[0043] In one embodiment of the present invention, S4062, constructing a multi-scale dual-gated feedforward network, includes:

[0044] The multi-scale dual-gated feedforward network includes a 1×1 high-dimensional convolution kernel, and also includes a first branch and a second branch. The first branch includes a 5×5 depth-separable convolution and a GELU activation function, and the second branch includes a 3×3 depth-separable convolution and a GELU activation function. The input end of the 1×1 high-dimensional convolution kernel is used as the input end of the multi-scale dual-gated feedforward network. The image received by the input end of the multi-scale dual-gated feedforward network is copied after passing through the 1×1 high-dimensional convolution kernel and input into the first branch and the second branch respectively, wherein a first convolution result is obtained through the 5×5 depth-separable convolution, a first filtering result is obtained through the GELU activation function of the first convolution result, a second convolution result is obtained through the 3×3 depth-separable convolution, and a second filtering result is obtained through the GELU activation function of the second convolution result;

[0045] The first filtering result is multiplied pixel by pixel by the second convolution result to obtain the first branch filtering result B1:

[0046] B1=GELU(DW 5×5 (Conv 1×1 (I in )))×DW 3×3 (Conv 1×1 (I in )) (11)

[0047] Among them, DW 3×3 Denotes the 3×3 depthwise separable convolution, DW 5×5 Represents the 5×5 depth-wise separable convolution, Conv 1×1 represents a convolution operation of size 1×1, and GELU represents the GELU activation function;

[0048] The second filtering result is multiplied pixel by pixel by the first convolution result to obtain the second branch filtering result B2:

[0049] B2=GELU(DW 3×3 (Conv 1×1 (I in )))×DW 5×5 (Conv 1×1 (I in )) (12)

[0050] The multi-scale dual-gated feedforward network also includes a 1×1 low-dimensional convolution kernel, which can keep the dimension of the image after the fusion processing of the first branch filtering result B1 and the second branch filtering result B2 consistent with the dimension of the image received by the input end of the multi-scale dual-gated feedforward network, and obtain the fusion result S of the two branches:

[0051] S=Conv 1×1 (B1+B2) (13)

[0052] The fusion result S is added to the image received at the input of the multi-scale dual-gated feedforward network to obtain the denoised image I M And output.

[0053] In one embodiment of the present invention, S4064, constructing an illumination-guided attention module (IGAB) based on a multi-scale dual-gated feedforward network, includes:

[0054] The IGAB module consists of the first regularization layer, the illumination-guided multi-head self-attention module, the second regularization layer, and the multi-scale dual-gated feedforward network (MDGFN) from front to back, and a residual connection is established before the first regularization layer and before the second regularization layer, and a residual connection is established before the second regularization layer and after the multi-scale dual-gated feedforward network, where

[0055] The image received by the first regularization layer is used as the first input of the IGAB. The first regularization layer can normalize the first input. The feature image F generated by the image illumination enhancement network D lu As the second input of IGAB, it is directly input into the illumination-guided multi-head self-attention module, which consists of two branches, one of which reshapes the first input into a first reshaped image X, and divides the first reshaped image X into k groups of heads:

[0056] X=[X1,X2,...,X i ,...,X k ] (15)

[0057]

[0058] Where k is an integer greater than or equal to 1, i∈{1,2,3…,k}, d k Indicates the number of channels allocated to each group, X i The size is HW×d k , H represents the height of the input image I, W represents the width of the input image I, C represents the number of channels of the image, and C is an integer of [32-512];

[0059] Another branch, the feature image F lu Reshape into the second reshaped image Y so that the feature image F lu Keep the same shape as the first reshaped image X and divide it into k groups of heads:

[0060] Y=[Y1,Y2,...,Y i ,...,Y k ] (18)

[0061] Among them, Yi The size is also HW×d k , the self-attention of each group head is expressed as:

[0062]

[0063] Among them, Q i Indicates based on X i The generated query vector, K i Indicates based on X i The generated key vector, V i Indicates that K i The corresponding value vector, represents block-by-block multiplication, softmax represents the activation function, k groups of heads are connected and position encoded to generate the output image X out .

[0064] In one embodiment of the present invention, α>β>γ.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] The low-light image enhancement method based on Retinex theory and guided by color prior provided by the present invention utilizes a low-light image enhancement network model guided by color prior based on Retinex theory. Firstly, based on Retinex theory, the pixel values ​​of the three channels R, G, and B of the original image are re-divided, and a color prior I is generated through a color prior generation module. p , the color prior I p Splice with the original image I to form a combined image I R To guide low-light image enhancement, it effectively solves the problem that the enhanced area of ​​some algorithms deviates from the original color of the object. The image illumination enhancement network D is used to increase the pixel value of the image and generate the illumination image I lu and feature image F lu . The illumination image I lu The image is input into the denoising network and denoised through a three-layer U-shaped network formed by multiple illumination-guided attention modules and convolutional layers. The illumination-guided attention module realizes image denoising at different scales by introducing a multi-scale dual-gated feedforward network. The dual-gated feedforward network first extracts image information of different scales, filters part of the noise at one scale through the GELU gating function, and then multiplies the filtered image with the image before filtering at another scale pixel by pixel to highlight the key content of the image. Finally, the results obtained at different scales are summed element by element to obtain an enhanced image, thereby improving the quality of the normal area of ​​the image and forming the final enhanced image. The problem that some enhanced areas in the enhanced image deviate from the original color and the noise is amplified in the process of enhancing low-light images is solved.

[0067] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a flowchart of a low-light image enhancement method guided by color prior based on Retinex theory provided by an embodiment of the present invention;

[0069] Figure 2 It is a structural schematic diagram of a color priori generation module in a low-light image enhancement network model provided by an embodiment of the present invention;

[0070] Figure 3 is a structural schematic diagram of an illumination enhancement network in a low illumination image enhancement network model provided by an embodiment of the present invention;

[0071] Figure 4 yes Figure 1 A schematic diagram of the process of constructing a denoising network in the provided low-light image enhancement method;

[0072] Figure 5 It is a schematic diagram of the structure of a multi-scale dual-gated feedforward network in a low-light image enhancement network model provided by an embodiment of the present invention;

[0073] Figure 6 is a structural schematic diagram of an IGAB module in a low-illumination image enhancement network model provided by an embodiment of the present invention;

[0074] Figure 7 is a schematic diagram of the structure of IG-MSA in a low-illumination image enhancement network model provided by an embodiment of the present invention;

[0075] Figure 8 is a schematic diagram of the structure of a denoising network in a low-light image enhancement network model provided by an embodiment of the present invention;

[0076] Fig. 9 (a) to Fig. 9 (d) are schematic diagrams showing the comparison of four different low-light images after being enhanced using the low-light image enhancement method provided by the present invention and the traditional Retinexformer method;

[0077] Fig.10 (a) to Fig.10 (c) are schematic diagrams showing the comparison of three different low-light images after being enhanced using the low-light image enhancement method provided by the present invention and the traditional Retinexformer method. DETAILED DESCRIPTION

[0078] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the scheme according to the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0079] The above and other technical contents, features and effects of the present invention are clearly presented in the following detailed description of the specific implementation modes in conjunction with the accompanying drawings. Through the description of the specific implementation modes, the technical means and effects adopted by the present invention to achieve the predetermined purpose can be more deeply and specifically understood. However, the attached drawings are only for reference and explanation purposes and are not used to limit the technical solutions of the present invention.

[0080] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants are intended to cover non-exclusive inclusion, so that an article or device including a series of elements includes not only those elements, but also other elements that are not explicitly listed. In the absence of more restrictions, the elements defined by the statement "including one..." do not exclude the existence of other identical elements in the article or device including the elements.

[0081] Please refer to Figure 1 , Figure 1 1 is a flow chart of a low-light image enhancement method guided by color prior based on Retinex theory provided by an embodiment of the present invention. The method is referred to as the low-light image enhancement method hereinafter. The low-light image enhancement method includes the following steps:

[0082] S20, obtaining a test sample set A and a training sample set B;

[0083] The test sample set A and the training sample set B are both provided with a plurality of original low-light images from different data sources, and the image data of the test sample set A and the training sample set B do not overlap. Specifically, a number of low-light images are randomly selected from the public low-light image data set as the sample image data set, and images accounting for 10% of the total number are randomly selected from the sample image data set as the test sample set A, and the remaining 90% of the images are used as the training sample set B. Specifically, in this embodiment, images of three data subsets of LOL-v2-syn, SMID and SDSD-out are selected as the sample image data sets, 90% of the images in each data subset are used as the training sample set B, and the remaining 10% of the reference images are used as the test sample set B. This arrangement ensures that there is no image data with the same image scene and content in the training set and the test set.

[0084] S40. Construct a low-light image enhancement network model guided by color prior based on Retinex theory;

[0085] The low-light image enhancement network model guided by color prior based on Retinex theory is hereinafter referred to as the enhancement network model or the low-light image enhancement network model. The construction of the low-light image enhancement network model includes the following steps:

[0086] S402: construct a color prior generation module, which can re-divide the received input image I according to the RGB pixel values ​​based on the preset proportional coefficients α, β, and γ to generate a color prior I p , the color prior I p Concatenate with input image I to form combined image I R ;

[0087] S404: construct an image illumination enhancement network D, which receives the combined image I R , it is possible to extract the combined image I R The global features of the feature image F lu , the feature image F lu Output, and based on the feature image F lu Get the illumination map Specifically, copy the feature image F lu , for the copied feature image F lu Perform convolution kernel operation to obtain illumination map Based on Retinex theory, the input image I and the illumination mapping Perform pixel-by-pixel multiplication to achieve image enhancement and output illumination image I lu , the illumination image I lu Characterized as consisting of the reflection component R and all interference factors Z;

[0088] S406. Construct a denoising network, which includes multiple illumination-guided attention modules (IGABs). The IGAB is provided with a regularization layer, an illumination-guided multi-head self-attention module, a regularization layer and a multi-scale dual-gated feedforward network connected in sequence. The illumination-guided multi-head self-attention module can enhance the long-range dependency of the received image. The multi-scale dual-gated feedforward network can copy the received image and input it into two branches respectively, one of which can extract the detail information of the received image, and the other branch can extract the information of adjacent pixels of the received image, highlighting the important pixel information in both branches, performing feature enhancement on the images of the two branches and then performing feature fusion, and outputting the fused fused image.

[0089] For further information, please refer to Figure 1 ,The specific steps of building a low-light image enhancement network model are:

[0090] S402, constructing a color priori generation module;

[0091] Please refer to Figure 2 , Figure 2 The present invention is a schematic diagram of the structure of a color priori generation module in a low-light image enhancement network model provided by an embodiment of the present invention. Specifically, the color priori generation module includes an input layer and a color priori generation network. The input layer sorts the pixels of the input image I in descending order according to the RGB pixel values; the color priori generation network re-divides the pixels of the input image I in descending order according to preset proportional coefficients α, β, and γ to generate a color priori I. p The preset proportional coefficients α, β, and γ correspond to the maximum, middle, and minimum values ​​of the RGB pixel values ​​in descending order, respectively, α>β>γ; the color prior I p After the input image I is connected in series through the channel, the output combined image I R .

[0092] Furthermore, the input layer can sort the pixels of the input image in descending order according to the RGB three-channel pixel values. Specifically, H represents the height of the input image I, W represents the width of the input image I, and the number of channels of the input image I is 3. Taking the input image I with a size of H×W×3 as an example, in the color prior generation module, the pixels at a random position of the input image I are sorted in descending order according to the size of the RGB three-channel pixel values, and are sorted in sequence as F(x,y), S(x,y), and L(x,y), which correspond to the highest pixel value, middle pixel value, and minimum pixel value of the pixel at the random position, respectively. The calculation formulas are as follows:

[0093] F(x,y)=MAX(R(x,y),G(x,y),B(x,y)) (1)

[0094] S(x,y)=SECOND(R(x,y),G(x,y),B(x,y)) (2)

[0095] L(x,y)=MIN(R(x,y),G(x,y),B(x,y)) (3)

[0096] In formulas (1), (2), and (3), (x, y) represents the position information of the pixel at the random position in the two-dimensional image, and MAX(), SECOND(), and MIN() represent the channel numbers of the highest, middle, and lowest pixel values, respectively. Taking a pixel at position (x, y) as an example, the channel numbers of the highest, middle, and lowest pixel values ​​in the three channels are R, G, and B, respectively. Then F(x, y) = R, S(x, y) = G, and L(x, y) = B. The same process is performed on the pixel points at other positions according to this rule until the pixel points at all positions have the highest pixel value, the middle pixel value, and the minimum pixel value, respectively, and F(x, y), S(x, y), and L(x, y) are all assigned RGB numbers.

[0097] The color prior generation network re-divides the pixels of the input image I by presetting the scale coefficient to generate a color prior I with a size of H×W×3. p .

[0098] Specifically, the preset proportional coefficients α, β, and γ of the highest pixel value, the middle pixel value, and the minimum pixel value in the RGB three channels of the pixel point at the random position are determined through experimental analysis. The pixels of the input image I are re-divided according to the preset proportional coefficients α, β, and γ to generate a color prior I with a size of H×W×3 p , combined with the original image I to form a combined image I with a size of H×W×6 R , used to guide subsequent image enhancement. Specifically, based on the preset scale coefficients α, β, and γ, the following formula is used to generate a color prior I with a size of H×W×3 p :

[0099] I p (x,y,F(x,y))=αI(x,y,F(x,y)) (4)

[0100] I p (x,y,S(x,y))=βI(x,y,S(x,y)) (5)

[0101] I p (x,y,L(x,y))=γI(x,y,L(x,y)) (6)

[0102] In formulas (4), (5), and (6), I(x,y,F(x,y)), I(x,y,S(x,y)), and I(x,y,L(x,y)) represent the highest, middle, and lowest pixel values ​​in the RGB channels of a pixel at a random position of the input image I, respectively. p (x,y,F(x,y)),I p (x,y,S(x,y)),I p(x,y,L(x,y)) is the pixel value after multiplying I(x,y,F(x,y)), I(x,y,S(x,y)), and I(x,y,L(x,y)) by the proportional coefficients α, β, and γ, respectively, that is, I p (x,y,F(x,y)),I p (x,y,S(x,y)),I p (x, y, L(x, y)) is the pixel value after the highest, middle, and minimum pixel values ​​in the RGB channels of the pixel point at the random position are re-divided based on the preset proportional coefficients α, β, and γ respectively.

[0103] The pixel values ​​I after all pixels of the input image I are re-divided p (x,y,F(x,y)),I p (x,y,S(x,y)),I p (x, y, L(x, y)) is put into a new image space of size H×W×3 as the color prior I p , the color prior I p Concatenate with input image I to form combined image I R As output, specifically, it is output to the image illumination enhancement network D.

[0104] The low-light image enhancement network model provided in this embodiment is to sort the pixels of the input image I in descending order according to the RGB pixel values, and the color prior generation network re-divides the pixels of the input image I in descending order according to the preset proportional coefficient to generate the color prior I p . The color prior I p After the input image I is connected in series through the channel, the output combined image I R , which solves the problem of overexposure during subsequent illumination enhancement, because the pixel values ​​of an entire low-illuminance area are close, resulting in the entire pixel values ​​being still relatively close after enhancement. The color prior generation module of this embodiment can further divide the parts with relatively close pixel values ​​in the low-illuminance area, thereby highlighting the differences between pixels at different positions with similar pixel values, reducing the probability of overexposure during the image enhancement process and improving the accuracy of image illumination enhancement.

[0105] S404, constructing an image illumination enhancement network D;

[0106] Please refer to Figure 3 , Figure 3 : is a schematic diagram of the structure of an illumination enhancement network in a low illumination image enhancement network model provided by an embodiment of the present invention. The image illumination enhancement network D includes a 1×1 convolution layer, a combined image I R As input, the combined image I is enhanced by a 1×1 convolution layer R Dimension.

[0107] Furthermore, the image illumination enhancement network D also includes a 9×9 depth-separable convolution layer, which receives the output of the 1×1 convolution layer and extracts the combined image I R The global features in the image are used to obtain the dimension-enhanced H×W×C feature image F lu , H represents the height of the input image I, W represents the width of the input image I, C represents the number of channels, and C is an integer of [32-512].

[0108] The extracted feature image F lu Duplicate it into two copies, one of which has the characteristic image F lu As the first output, it is output to the denoising network. Specifically, the feature image F lu Serves as input to the illumination-guided attention module included in the subsequent denoising network.

[0109] Furthermore, the image illumination enhancement network D also includes a 1×1×3 convolution kernel, which transforms another feature image F lu As the input of the 1×1×3 convolution kernel, we get the illumination map

[0110] Furthermore, based on the Retinex theory, the input image I is decomposed into a reflection component R and an illumination component L, and the illumination mapping Multiply the input image I pixel by pixel to generate the illumination image I lu As the second output, the illumination image I lu It is characterized by the reflection component R and all interference factors Z.

[0111] Specifically, since most of the noise exists in the reflection component, the input image I is decomposed into the reflection component R and the illumination component L based on the Retinex theory. The process of enhancing the image is actually the process of enhancing the illumination component L. Separating the illumination component L from the reflection component R is beneficial to processing noise in subsequent steps. Specifically, the illumination mapping Multiply the input image I pixel by pixel to generate the illumination image I lu As the second output, the specific calculation steps are as follows:

[0112] Step 1: Decompose the input image I into the reflection component R and the illumination component L according to the Retinex theory:

[0113] I=R×L (7)

[0114] Step 2: According to formula (7), considering the interference factors in the reflection component R and the illumination component L of the input image I, the reflection component R and the illumination component L of the input image I considering the interference factors are obtained:

[0115]

[0116] Specifically, and They represent the interference factors of the reflection component R and the interference factors of the illumination component L respectively.

[0117] Step 3: Map the input image I with the interference factor Multiply them to get the illumination image I lu :

[0118]

[0119] Specifically, according to The illumination image I lu It is characterized by two parts consisting of the reflection component R and all interference factors Z:

[0120]

[0121] The low-light image enhancement network model provided in this embodiment extracts global features from the input image I and estimates the illumination map. According to the Retinex theory, the input image I is decomposed into the reflection component R and the illumination component L, and the illumination mapping is transformed into Multiply the input image I pixel by pixel to achieve image enhancement, and process the input image I that needs to be retained and the interference factors that need to be processed separately, and use the convolutional network to estimate the illumination map Instead of directly estimating the illumination image I lu Compared with the traditional processing, which ignores the process of separating the noise part and directly estimates the illumination image, it reduces the noise amplification.

[0122] S406: Construct a denoising network.

[0123] Please refer to Figure 4 , Figure 4 yes Figure 1 A schematic diagram of a process for constructing a denoising network in the provided low-light image enhancement method. Specifically, constructing the denoising network includes the following steps:

[0124] S4062, construct a multi-scale dual-gated feedforward network;

[0125] Please refer to Figure 5 , Figure 5It is a structural schematic diagram of a multi-scale dual-gated feed-forward network in a low-illumination image enhancement network model provided by an embodiment of the present invention. The multi-scale dual-gated feed-forward network (MDGFN) can copy the received image and input it into two branches respectively, one of which can extract detail information of the received image, and the other branch can extract information of adjacent pixels of the received image, highlighting important pixel information in both branches, performing feature enhancement on the images of the two branches and then performing feature fusion, and outputting the fused fused image.

[0126] Specifically, the multi-scale dual-gated feedforward network includes a 1×1 high-dimensional convolution kernel, and the input end of the 1×1 high-dimensional convolution kernel serves as the input end of the multi-scale dual-gated feedforward network. The image received by the input end of the multi-scale dual-gated feedforward network undergoes a convolution operation through the 1×1 high-dimensional convolution kernel. Exemplarily, the input image received by the input end of the multi-scale dual-gated feedforward network is the input image I in .

[0127] The multi-scale dual-gated feedforward network also includes a first branch and a second branch. The output result obtained by the 1×1 high-dimensional convolution kernel is copied and input into the two branches respectively. Further, the first branch includes a 5×5 depthwise separable convolution (DSC) and a Gaussian Error Linear Unit (GELU) activation function. The 5×5 depthwise separable convolution can perceive the image in a larger area and extract more detail information of the image received by the multi-scale dual-gated feedforward network to obtain the first convolution result. The image detail information can be, for example, some edge pixel values ​​of the image and the connection between some pixel values. At the same time, the use of the 5×5 depthwise separable convolution can reduce the amount of parameters introduced into the model, making the low-light image enhancement network model as simple as possible. Then, the GELU activation function can filter out a small amount of unimportant information and part of the noise in the image to obtain the first filtering result.

[0128] Furthermore, the second branch includes a 3×3 depthwise separable convolution, which can perceive the image in a smaller area, extract and focus on the information of adjacent pixels, and reduce the amount of parameters introduced into the model compared to a 5×5 depthwise separable convolution. The second branch also includes a GELU activation function, which extracts and focuses on the information of adjacent pixels in the image in a smaller area, obtains a second convolution result, and uses the GELU activation function to filter out a small amount of unimportant information and part of the noise in the adjacent pixel area to obtain a second filtering result. The unimportant small amount of information in the two branches mainly refers to the information of some areas with lower pixel values, such as black noise.

[0129] like Figure 5 As shown, the first filtering result of the first branch and the second convolution result of the second branch are multiplied pixel by pixel to obtain the first branch filtering result B1:

[0130] B1=GELU(DW 5×5 (Conv 1×1 (I in )))×DW 3×3 (Conv 1×1 (I in )) (11)

[0131] Specifically, D.W. 3×3 Denotes a depthwise separable convolution of size 3×3, i.e., the 3×3 depthwise separable convolution, DW 5×5 Represents a depthwise separable convolution of size 5×5, that is, the 5×5 depthwise separable convolution, Conv 1×1 represents a convolution operation of size 1×1, and GELU represents the GELU activation function.

[0132] The second filtering result of the second branch is multiplied pixel by pixel by the first convolution result of the first branch to obtain the second branch filtering result B2:

[0133] B2=GELU(DW 3×3 (Conv 1×1 (I in )))×DW 5×5 (Conv 1×1 (I in )) (12)

[0134] Furthermore, the multi-scale dual-gated feedforward network also includes a 1×1 low-dimensional convolution kernel, and the sum of the first branch filtering result B1 and the second branch filtering result B2 is used as the input of the 1×1 low-dimensional convolution kernel. The 1×1 low-dimensional convolution kernel can keep the dimension of the image after the first branch filtering result B1 and the second branch filtering result B2 are added and processed consistent with the image dimension received at the input end of the multi-scale dual-gated feedforward network, and obtain the fusion result S of the two branches:

[0135] S=Conv 1×1 (B1+B2) (13)

[0136] Then add the obtained fusion result S to the image received at the input of the multi-scale dual-gated feedforward network to obtain the denoised image I M , denoised image I M As the output of this multi-scale dual-gated feedforward network:

[0137] I M =S+I in (14)

[0138] The multi-scale dual-gated feedforward network sets a first branch including a 5×5 depthwise separable convolution and a GELU activation function, so as to perceive the image received by the branch in a larger area, extract more detailed information of the image and perform noise filtering. At the same time, a second branch including a 3×3 depthwise separable convolution and a GELU activation function is set to extract the image received by the second branch in a smaller area, focus on the information of adjacent pixels, and perform noise filtering. The first filtering result of the first branch is multiplied pixel by pixel with the second convolution result of the second branch to obtain the first branch filtering result B1, and the second filtering result of the second branch is multiplied pixel by pixel with the first convolution result of the first branch to obtain the second branch filtering result B2, thereby highlighting the important pixel information in both branches and suppressing the important information in only one branch, thereby suppressing the noise of the enhanced image.

[0139] S4064, constructing an illumination-guided attention block (IGAB) based on a multi-scale dual-gated feed-forward network;

[0140] Please refer to Figure 6 , Figure 6 It is a structural schematic diagram of the IGAB module in the low-illumination image enhancement network model provided by an embodiment of the present invention. IGAB can denoise the received image and enhance the long-range dependency relationship between image pixels.

[0141] The IGAB consists of a first regularization layer, an illumination-guided multi-head self-attention module (IG-MSA), a second regularization layer and the multi-scale dual-gated feedforward network from front to back, and a residual connection is established before the first regularization layer and before the second regularization layer, and a residual connection is established before the second regularization layer and after the multi-scale dual-gated feedforward network.

[0142] Specifically, the image received by the first regularization layer is used as the first input of the IGAB. For example, the first input may be the denoised image I M , the first regularization layer can normalize the first input, and convert the feature image F generated by the image illumination enhancement network D in step S404 lu As the second input of the IGAB, and the second input is directly input into the illumination-guided multi-head self-attention module, the IG-MSA module can extract the connection between pixel values ​​from the entire image globally for the normalized first input. The IG-MSA consists of two branches, one of which reshapes the first input into a two-dimensional image. Specifically, the first input is reshaped into a first reshaped image X with a size of HW×C, and then the first reshaped image X is divided into k groups of heads. The calculation formula includes formula (15) and formula (16):

[0143] X=[X1,X2,...,X i ,...,X k ] (15)

[0144]

[0145] Specifically, k is the number of groups into which the channels are divided, k is an integer greater than or equal to 1, i∈{1,2,3…,k}, X i represents the i-th group head of the first reshaped image X, X i The size is HW×d k , d k Represents the number of channels in each group, H represents the height of the input image I, W represents the width of the input image I, C represents the number of channels of the image, and C is an integer of [32-512].

[0146] Use three unbiased fully connected layers X i Linear projection to query vector Q i , key vector K i , value vector V i As shown in formula (17):

[0147]

[0148] Specifically, Q i Indicates based on Xi The generated query vector, K i Indicates based on X i The generated key vector, V i Indicates that K i The corresponding value vector, T represents the matrix transpose.

[0149] Another branch of the IG-MSA uses the feature image F generated by the image illumination enhancement network D in step S404 lu As the input feature, the feature image F lu Reshape into a two-dimensional image. Specifically, the feature image F lu Reshape into the second reshaped image Y of size HW×C so that the feature image F lu Keep the shape of the first reshaped image X consistent and divide it into k groups of heads. The formula is as follows:

[0150] Y=[Y1,Y2,...,Y i ,...,Y k ] (18)

[0151] Specifically, k is an integer greater than or equal to 1, i∈{1,2,3…,k}, Y i represents the i-th group head of the second reshaped image Y, Y i The size is also HW×d k ,Right now H represents the height of the input image I, W represents the width of the input image I, C represents the number of channels of the image, and C is an integer between [32-512]. The self-attention of each group head is expressed as:

[0152]

[0153] Specifically, α i represents the learnable parameters of the adaptive scaling matrix multiplication, i∈{1,2,3…,k}, represents block-by-block multiplication, and softmax represents the activation function. Then the k groups of heads are connected and positional encoding is added to generate the output image X out .

[0154] In this embodiment, k=1, such as Figure 7 As shown, the query vector Q i Here is an example of a query vector Q and a key vector K. i Sum value vector V i The key vector K and the value vector V are respectively exemplified. The first reshaped image X is linearly projected to the query vector Q, the key vector K, the value vector V, and the feature image F lu The first input of IGAB is processed by the above formula (19) to obtain the output image X out .

[0155] The illumination-guided attention module improves the long-range dependencies of the received image through the set IG-MSA. For example, when two pixels are far apart in the image, it is difficult to establish a connection through convolution operation, but the IG-MSA module can enhance the connection between pixel values ​​at distant locations.

[0156] S4066. Construct a denoising network based on the illumination-guided attention module.

[0157] Please refer to Figure 8 , Figure 8 : is a schematic diagram of the structure of a denoising network in a low-illumination image enhancement network model provided by an embodiment of the present invention. The denoising network includes three downsampling convolution layers. The convolution kernel sizes of the three downsampling convolution layers are 3×3, 4×4, and 4×4, respectively, and the step sizes are 1, 2, and 2, respectively. The illumination image I lu As the input of the denoising network, the input end of the first downsampling convolution layer is used as the input end of the denoising network, the first IGAB module is connected between the first downsampling convolution layer and the second downsampling convolution layer, and the first IGAB module includes 1 IGAB, and the second IGAB module is connected between the second downsampling convolution layer and the third downsampling convolution layer. The third IGAB module is connected after the third downsampling convolution layer, and the result obtained is input to the deconvolution layer with a size of 2×2 and a step size of 2 to obtain the first deconvolution result, and the result obtained by the second IGAB module is spliced ​​with the first deconvolution result. After the splicing is completed, the first spliced ​​image is obtained, and the first spliced ​​image is input to the convolution layer with a convolution kernel size of 1×1. The denoising network also includes a fourth IGAB module, the output result of the 1×1 convolution layer is used as the input of the fourth IGAB module, the output result of the fourth IGAB module is input to the deconvolution layer with a size of 2×2 and a step size of 2 to obtain a second deconvolution result, the result obtained by the first IGAB module is spliced ​​with the second deconvolution result to obtain a second spliced ​​image, and the second spliced ​​image is input to the convolution layer with a convolution kernel size of 1×1. The denoising network also includes a fifth IGAB module, the output result of the second spliced ​​image through the convolution layer with a convolution kernel size of 1×1 is used as the input of the fifth IGAB module, and the fifth IGAB module is output to the convolution layer with a size of 3×3 and a step size of 1 to form a denoising component I re , the denoised component I re The illumination image I generated by the image illumination enhancement network D lu The final enhanced image I is obtained by summing each element en . Thus, the illumination image I is offset luPart of the interference factors are removed to achieve the purpose of denoising. Specifically, the second IGAB module, the third IGAB module, the fourth IGAB module and the fifth IGAB module are each provided with two connected light-guided attention modules.

[0158] S60, using training sample set B to train a low-light image enhancement network model guided by color prior based on Retinex theory;

[0159] The training sample set B is used to train the low-light image enhancement network model guided by color prior based on Retinex theory. The loss of the enhanced image and the real image obtained after the original low-light image is trained by the low-light image enhancement network model guided by color prior based on Retinex theory is calculated according to the mean absolute error loss function. The weight of the model is updated according to the loss until the preset training conditions are met.

[0160] Specifically, a training sample set is used to train a low-light image enhancement network model guided by color prior based on Retinex theory, which specifically includes the following steps:

[0161] Step 1: Set the initialization iteration number to t, the maximum iteration number to T, and set t = 1;

[0162] In this example, T=150,000.

[0163] Step 2: randomly select b training samples from the training sample set B without replacement as the original image and input them into the low-light image enhancement network model guided by the color prior based on Retinex theory to form an enhanced image I of the original image. en ;

[0164] Specifically, the color prior generation module generates the color prior I p , the generated color prior I p Combined with the original image I to generate the combined image I R , the original image I comes from b training samples, and the illumination image I is generated by the image illumination enhancement network D lu , and then the illumination image I lu Use the denoising network to generate the denoised component I re Finally, the denoised component I re With the illumination image I lu Add pixel by pixel to form the enhanced image I of the original image en .

[0165] Step 3: Select the mean absolute error loss function (MAE) and calculate the enhanced image I according to the mean absolute error loss function. en And the loss MAE of the real image, update the weight of the model;

[0166] Specifically, the enhanced image I is calculated according to the mean absolute error loss function en And the loss MAE of the real image is as follows:

[0167]

[0168] Specifically, W represents the width of the real image, H represents the height of the real image, Re(x,y) represents the pixel value at position (x,y) in the real image, and I en (x,y) represents the pixel value at position (x,y) in the enhanced image.

[0169] Step 4: Repeat steps 2 to 3 until all original images in sample set B have completed traversal training;

[0170] Step 5, determine whether t=T is established. When t=T, obtain the trained low-light image enhancement network model guided by color prior based on Retinex theory. Otherwise, let t=t+1, repeat steps 2 to 3 until t=T, and obtain the low-light image enhancement network model guided by color prior based on Retinex theory.

[0171] S80. Obtain the enhancement result of the original low-light image in the test sample set A through the low-light image enhancement network model guided by color prior based on Retinex theory.

[0172] The test sample set A is used as the input of the trained low-light image enhancement network model guided by color prior based on Retinex theory. Forward reasoning is performed to obtain the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) of the test sample set A to verify the model effect.

[0173] The following is a description of the technical effects of the present invention in combination with simulation experiments:

[0174] In order to verify the effectiveness of the proposed method in the low-light image enhancement task, experiments were carried out using the method of this application on the LOL-v2-syn, SMID and SDSD-out public datasets, and the experimental results were compared with those of other algorithms in terms of PSNR and SSIM evaluation indicators. Table 1 is a comparison of the effects of the method provided by this application and the general method for low-light image enhancement:

[0175] Table 1

[0176]

[0177] In Table 1, IPT, RetinexNet, Sparse, EnGAN, RUAS, FIDE, DRBN, KinD, Restormer, MIRNet, SNR-Net, and Retinexformer are general methods for low-light image enhancement, LOL-v2-syn, SMID, and SDSD-out are three different data sources, and the intersection of columns and rows represents the PSNR and SSIM values ​​of different methods for the same low-light photo from the same data source. The last row is the PSNR and SSIM values ​​of the method of this application for the same low-light photo from the same data source. It can be seen from the table that the PSNR and SSIM values ​​of the method of this application have reached the highest on multiple data sets, and the SSIM value has increased ↑ relative to the PSNR value of the general method for low-light image enhancement.

[0178] Please refer to Fig. 9 and Fig.10 , Fig. 9 (a) to Fig. 9 (d) are schematic diagrams showing the comparison of four different low-light images after being enhanced using the low-light image enhancement method provided by the present invention and the traditional Retinexformer method. Fig.10 (a) to Fig.10 (c) are schematic diagrams comparing three different low-light images after being enhanced using the low-light image enhancement method provided by the present invention and the traditional Retinexformer method. It can be clearly seen from the figure that, compared with the traditional Retinexformer method, the color of the objects in the image enhanced by the method of the present application is closer to the real color, the image details are clearer, the image is smoother, the noise processing effect is better, and the background details are better preserved. It can be seen from the figure that the texture characteristics of the sky and the lake in the enhanced image of the present application are fully reflected.

[0179] The low-light image enhancement method based on Retinex theory and guided by color prior provided by the present invention utilizes a low-light image enhancement network model guided by color prior based on Retinex theory. Firstly, based on Retinex theory, the pixel values ​​of the three channels R, G, and B of the original image are re-divided, and a color prior I is generated through a color prior generation module. p , the color prior I p Splice with the original image I to form a combined image I R To guide low-light image enhancement, it effectively solves the problem that the enhanced area of ​​some algorithms deviates from the original color of the object. The image illumination enhancement network D is used to increase the pixel value of the image and generate the illumination image I lu and feature image F lu . The illumination image Ilu The image is input into the denoising network and denoised through a three-layer U-shaped network formed by multiple illumination-guided attention modules and convolutional layers. The illumination-guided attention module realizes image denoising at different scales by introducing a multi-scale dual-gated feedforward network. The dual-gated feedforward network first extracts image information of different scales, filters part of the noise at one scale through the GELU gating function, and then multiplies the filtered image with the image before filtering at another scale pixel by pixel to highlight the key content of the image. Finally, the results obtained at different scales are summed element by element to obtain an enhanced image, thereby improving the quality of the normal area of ​​the image and forming the final enhanced image. The denoising effect and image quality are improved, and the problem of some enhanced areas in the enhanced image deviating from the original color and the problem of noise amplification in the process of enhancing low-illuminance images is solved.

[0180] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A low-light image enhancement method based on Retinex theory and guided by color prior, characterized in that: The following steps are involved: S20, obtaining a test sample set A and a training sample set B, wherein both the test sample set A and the training sample set B are provided with a plurality of original low-light images from different data sources; S40. Construct a low-light image enhancement network model guided by color prior based on Retinex theory, including: S402, constructing a color priori generation module, wherein the color priori generation module can re-divide the received input image I according to the RGB pixel values ​​based on the preset proportional coefficients α, β, and γ to generate a color priori I p , the color prior I p Concatenate with input image I to form combined image I R ; S404: construct an image illumination enhancement network D, which can extract the combined image I R The global features of the feature image F lu , and based on the feature image F lu Get the illumination map L, and multiply the input image I and the illumination map L pixel by pixel based on the Retinex theory to output the illumination image I lu To the denoising network; S406, constructing a denoising network, wherein the denoising network includes a plurality of illumination-guided attention modules (IGABs), wherein the IGABs are provided with a regularization layer, an illumination-guided multi-head self-attention module, a regularization layer, and a multi-scale dual-gated feedforward network connected in sequence, wherein the illumination-guided multi-head self-attention module can enhance the long-range dependency of the received image, and the multi-scale dual-gated feedforward network can copy the received image and input it into two branches respectively, wherein one branch can extract detail information of the received image, and the other branch can extract information of adjacent pixels of the received image, highlighting important pixel information in both branches, performing feature enhancement on the images of the two branches and then performing feature fusion, and outputting a fused image after fusion; S60, using training sample set B to train the model, calculating the loss of the enhanced image and the real image obtained after the original low-light image is trained by the model according to the mean absolute error loss function, and updating the weight of the model according to the loss until the preset training condition is met; S80. Obtain enhancement results of the original low-light images in the test sample set A through the model.

2. The low-light image enhancement method based on Retinex theory and guided by color prior according to claim 1, characterized in that: S402, constructing a color priori generation module, including: The color prior generation module includes an input layer and a color prior generation network. The input layer sorts the pixels of the input image I in descending order according to the RGB pixel values; the color prior generation network re-divides the pixels of the input image I in descending order according to preset proportional coefficients α, β, and γ to generate a color prior I p ; Set the color prior I p After the input image I is connected in series through the channel, the output combined image I R , wherein the preset proportional coefficients α, β, and γ correspond to the maximum value, the middle value, and the minimum value of the RGB pixel values ​​in descending order, respectively.

3. The low-light image enhancement method based on Retinex theory and guided by color prior according to claim 2, characterized in that: The color prior generation network re-divides the pixels of the input image I in descending order according to the preset proportional coefficients α, β, and γ to generate the color prior I p ,include: The preset proportional coefficients α, β, and γ of the highest pixel value, the middle pixel value, and the minimum pixel value in the RGB channel of the pixel point at the random position are determined by experimental analysis. The pixels of the input image I are re-divided according to the preset proportional coefficients α, β, and γ to generate a new color prior I with a size of H×W×3 p : I p (x,y,F(x,y))=αI(x,y,F(x,y)) (4) I p (x,y,S(x,y))=βI(x,y,S(x,y)) (5) I p (x,y,L(x,y))=γI(x,y,L(x,y)) (6) Among them, I(x,y,F(x,y)), I(x,y,S(x,y)), and I(x,y,L(x,y)) represent the highest, middle, and minimum pixel values ​​in the RGB channels of the pixel at a random position of the input image I, respectively. p (x,y,F(x,y)),I p (x,y,S(x,y)),I p (x, y, L(x, y)) is the pixel value after the highest, middle, and minimum pixel values ​​in the RGB channels of the pixel point at the random position are re-divided based on the preset proportional coefficients α, β, and γ respectively; The pixel values ​​of all pixels in the input image I are re-divided into a new image space of size H×W×3 as the color prior I p , where H represents the height of the input image I and W represents the width of the input image I.

4. The low-light image enhancement method based on Retinex theory and guided by color prior according to claim 1, characterized in that: S404, constructing an image illumination enhancement network D, including: The image illumination enhancement network D includes a 1×1 convolutional layer and a 9×9 depth-separable convolutional layer, wherein the combined image I R As input, the image I is enhanced by a 1×1 convolution layer. R The 9×9 depthwise separable convolution receives the output of the 1×1 convolution layer and extracts the combined image I R The global features in get the feature image F lu ; The extracted feature image F lu Copy it into two feature images F lu , one of the feature images F lu Output to the denoising network; The image illumination enhancement network D also includes a 1×1×3 convolution kernel, and another feature image F lu As the input of the 1×1×3 convolution kernel, we get the illumination map Based on the Retinex theory, the input image I is decomposed into a reflection component R and an illumination component L. Multiply the input image I pixel by pixel to generate and output the illumination image I lu , the illumination image I lu It is characterized by the reflection component R and all interference factors Z.

5. The low-light image enhancement method based on Retinex theory and guided by color prior according to claim 4, characterized in that: Based on the Retinex theory, the input image I is decomposed into a reflection component R and an illumination component L. Multiply the input image I pixel by pixel to generate and output the illumination image I lu ,include: Step 1: Decompose the input image I into the reflection component R and the illumination component L according to the Retinex theory: I=R×L (7) Step 2: According to formula (7), considering the interference factors in the reflection component R and the illumination component L of the input image I, the reflection component R and the illumination component L of the input image I considering the interference factors are obtained: in, and They represent the interference factors of the reflection component R and the interference factors of the illumination component L respectively; Step 3: Map the input image I with the interference factor Multiply them to get the illumination image I lu : Among them, according to The illumination image I lu It is characterized by two parts consisting of the reflection component R and all interference factors Z:

6. The low-light image enhancement method based on Retinex theory and guided by color prior according to claim 1, characterized in that: S406, constructing a denoising network, including: S4062, construct a multi-scale dual-gated feedforward network; S4064, constructing an illumination-guided attention module (IGAB) based on a multi-scale dual-gated feed-forward network; S4066, build a denoising network based on IGAB; The denoising network includes three downsampling convolution layers, the convolution kernel sizes of the three downsampling convolution layers are 3×3, 4×4, and 4×4, respectively, and the step sizes are 1, 2, and 2, respectively. lu As the input of the denoising network, the input of the first downsampling convolutional layer receives the illumination image I as the input of the denoising network. lu , the first IGAB module is connected between the first downsampling convolution layer and the second downsampling convolution layer, the second IGAB module is connected between the second downsampling convolution layer and the third downsampling convolution layer, and the third IGAB module is connected after the third downsampling convolution layer. The result is input to the deconvolution layer with a size of 2×2 and a step size of 2 to obtain the first deconvolution result. The result obtained by the second IGAB module is spliced ​​with the first deconvolution result. After the splicing is completed, the first spliced ​​image is obtained. The first spliced ​​image is input to the convolution layer with a convolution kernel size of 1×1. The denoising network also includes a fourth IGAB module, a 1×1 convolution kernel. The result of the convolution layer output is used as the input of the fourth IGAB module, and the output result of the fourth IGAB module is input to the deconvolution layer with a size of 2×2 and a step size of 2 to obtain a second deconvolution result. The result obtained by the first IGAB module is spliced ​​with the second deconvolution result to obtain a second spliced ​​image, and the second spliced ​​image is input to the convolution layer with a convolution kernel size of 1×1. The denoising network also includes a fifth IGAB module, and the result of the second spliced ​​image output by the convolution layer with a convolution kernel size of 1×1 is used as the input of the fifth IGAB module, and the fifth IGAB module is output to the convolution layer with a size of 3×3 and a step size of 1 to form a denoising component I re , the denoised component I re The illumination image I generated by the image illumination enhancement network D lu The final enhanced image I is obtained by summing each element en , wherein the first IGAB module includes one IGAB, and the second to fifth IGAB modules are each provided with two connected IGABs.

7. The low-light image enhancement method based on Retinex theory and guided by color prior according to claim 6, characterized in that: S4062. Construct a multi-scale dual-gated feedforward network, including: The multi-scale dual-gated feedforward network includes a 1×1 high-dimensional convolution kernel, and also includes a first branch and a second branch. The first branch includes a 5×5 depth-separable convolution and a GELU activation function, and the second branch includes a 3×3 depth-separable convolution and a GELU activation function. The input end of the 1×1 high-dimensional convolution kernel is used as the input end of the multi-scale dual-gated feedforward network. The image received by the input end of the multi-scale dual-gated feedforward network is copied after passing through the 1×1 high-dimensional convolution kernel and input into the first branch and the second branch respectively, wherein a first convolution result is obtained through the 5×5 depth-separable convolution, a first filtering result is obtained through the GELU activation function of the first convolution result, a second convolution result is obtained through the 3×3 depth-separable convolution, and a second filtering result is obtained through the GELU activation function of the second convolution result; The first filtering result is multiplied pixel by pixel by the second convolution result to obtain the first branch filtering result B1: B1=YELLOW(DW 5×5 (Conv. 1×1 (I in )))×DW 3×3 (Conv. 1×1 (I in )) (11) Among them, DW 3×3 Denotes the 3×3 depthwise separable convolution, DW 5×5 Represents the 5×5 depth-wise separable convolution, Conv 1×1 represents a convolution operation of size 1×1, and GELU represents the GELU activation function; The second filtering result is multiplied pixel by pixel by the first convolution result to obtain the second branch filtering result B2: B2=GELU(DW 3×3 (Conv 1×1 (I in )))×DW 5×5 (Conv 1×1 (I in )) (12) The multi-scale dual-gated feedforward network also includes a 1×1 low-dimensional convolution kernel, which can keep the dimension of the image after the fusion processing of the first branch filtering result B1 and the second branch filtering result B2 consistent with the dimension of the image received by the input end of the multi-scale dual-gated feedforward network, and obtain the fusion result S of the two branches: S=Conv 1×1 (B1+B2) (13) The fusion result S is added to the image received at the input of the multi-scale dual-gated feedforward network to obtain the denoised image I M And output.

8. The low-light image enhancement method based on Retinex theory and guided by color prior according to claim 6, characterized in that: S4064, constructing an illumination-guided attention module (IGAB) based on a multi-scale dual-gated feedforward network, including: The IGAB module consists of the first regularization layer, the illumination-guided multi-head self-attention module, the second regularization layer, and the multi-scale dual-gated feedforward network (MDGFN) from front to back, and a residual connection is established before the first regularization layer and before the second regularization layer, and a residual connection is established before the second regularization layer and after the multi-scale dual-gated feedforward network, where The image received by the first regularization layer is used as the first input of the IGAB. The first regularization layer can normalize the first input. The feature image F generated by the image illumination enhancement network D lu As the second input of IGAB, it is directly input into the illumination-guided multi-head self-attention module, which consists of two branches, one of which reshapes the first input into a first reshaped image X, and divides the first reshaped image X into k groups of heads: X=[X1,X2,…,X i ,…,X k ] (15) Where k is an integer greater than or equal to 1, i∈{1,2,3…,k}, d k Indicates the number of channels allocated to each group, X i The size is HW×d k , H represents the height of the input image I, W represents the width of the input image I, C represents the number of channels, and C is an integer of [32-512]; Another branch, the feature image F lu Reshape into the second reshaped image Y so that the feature image F lu Keep the same shape as the first reshaped image X and divide it into k groups of heads: Y=[Y1,Y2,...,Y i ,...,Y k ] (18) Among them, Y i The size is also HW×d k , the self-attention of each group head is expressed as: Among them, Q i Indicates based on X i The generated query vector, K i Indicates based on X i The generated key vector, V i Indicates that K i The corresponding value vector, represents block-by-block multiplication, softmax represents the activation function, k groups of heads are connected and position encoded to generate the output image X out .

9. The low-light image enhancement method based on Retinex theory and guided by color prior according to any one of claims 1 to 8, characterized in that: α>β>γ.

Images