Low-light image enhancement method guided by color prior based on retinex theory

Abstract

The application discloses a low-illumination image enhancement method based on a color prior guided Retinex theory, and constructs a low-illumination image enhancement network model based on the color prior guided Retinex theory to perform low-illumination image enhancement. The model is based on the Retinex theory, generates a color prior through a color prior generation module, splices the color prior and an original image to form a combined image to guide the low-illumination image enhancement, improves pixel values of the image through an image illumination enhancement network, and generates an illumination image and a feature image. The illumination image is input into a denoising network to generate a denoising component, and the denoising network is provided with a light-guided attention module. The module introduces a multi-scale double-gated feedforward network, so that the denoising network realizes image denoising at different scales and highlights key contents of the image. The low-illumination image enhancement method provided by the application solves the problems that part of the enhanced region deviates from the original color in the enhanced image and noise amplification in the process of enhancing the low-illumination image.

Description

Technical Field

[0001] This invention belongs to the field of low-light image enhancement technology, specifically relating to a low-light image enhancement method based on Retinex theory and guided by color prior. Background Technology

[0002] Images captured at night are typically low-light images, suffering from insufficient illumination, loss of detail, and noise. These problems not only affect the visual quality of the image but also subsequent computer vision tasks, such as object tracking and face recognition. Therefore, low-light image enhancement techniques are increasingly valuable in both theory and application, becoming an important research direction in the field of computer vision. The goal of low-light image enhancement techniques is to effectively increase image brightness, enhance image contrast, eliminate the effects of image noise, improve image details, maintain image color balance, and ultimately improve the overall quality of the image.

[0003] Traditional low-light image enhancement techniques based on Retinex theory often use histogram equalization or gamma correction to increase pixel values ​​and achieve the desired effect. Chinese invention patent CN117994155A discloses a two-branch low-light image enhancement method based on Retinex theory. This method decomposes a low-light image into illuminance and reflectance components. It performs initial feature extraction on the input low-light image using dilated convolutions with different dilation rates in parallel, and then uses an enhancement unit in an iterative learning module with a U-Net network to perform deeper feature extraction on the initialized image. While denoising the reflectance component, it enhances the illuminance component. After three iterations of learning, the resulting illuminance and reflectance components are fused, achieving denoising during the enhancement process. The drawback of this method is that it ignores the actual meaning of the image content during low-light image enhancement, causing some enhanced areas to deviate from the original colors. Chinese invention patent CN118247164A discloses a low-light image enhancement network and method. This method decomposes a low-light image into illumination and reflectance components, uses an encoder-decoder structure to process noise in the reflectance image, and finally fuses the illumination image with the denoised reflectance image as input to the enhancement network. It utilizes CTrans to transmit spatial information lost in the pooling layer and restores the complete spatial resolution through the encoder-decoder process. Upsampling or downsampling is achieved through four consecutive RDBs, with additional skip connections between each upsampling or downsampling operation to bypass redundant low-frequency information, allowing the network to focus on learning high-frequency information. The downsampling process uses stride convolution to convert the noisy image from a high-resolution scale to a low-resolution scale, while the upsampling process uses sub-pixel convolution to convert the low-resolution scale back to a high-resolution scale, thereby removing noise and preserving important features of the input image. The drawback of this method is that it ignores the actual meaning of the image content when enhancing low-light images, causing the enhanced region to deviate from the original color.

[0004] Analysis of existing technologies reveals that deep learning-based low-light image enhancement methods, grounded in Retinex theory, typically divide low-light images into illuminance and reflectance components. Initially, the reflectance component is directly enhanced. Later, the illuminance distribution of the illuminance component is improved, and noise in the reflectance component is removed. Finally, these components are fused to achieve the desired enhancement. Directly increasing pixel values ​​often results in overexposure and increased image noise. Specifically, due to a lack of understanding of image content during processing, algorithms often enhance dark areas, potentially leading to problems. For example, treating black hair as a low-light area can cause the enhanced region to deviate from its original color. Furthermore, some methods denoise before enhancing, resulting in a loss of detail in the enhanced image. In summary, current deep learning-based low-light image enhancement methods suffer from issues such as some enhanced regions deviating from their original colors and do not effectively address the noise problem that arises during low-light image enhancement. Summary of the Invention

[0005] To address the aforementioned problems in the prior art, the present invention is achieved through the following technical solution:

[0006] This invention proposes a low-light image enhancement method based on Retinex theory and guided by color prior, comprising the following steps:

[0007] S20. Obtain test sample set A and training sample set B, wherein both test sample set A and training sample set B contain multiple original low-light images from different data sources.

[0008] S40. Construct a low-light image enhancement network model based on Retinex theory and guided by color priors, including:

[0009] S402. Construct a color prior generation module, which can re-divide the received input image I according to RGB pixel values ​​based on preset scaling coefficients α, β, and γ to generate a color prior I. p Color prior I p Combined with input image I to form composite image I R ;

[0010] S404. Construct an image illumination enhancement network D. The image illumination enhancement network D can extract combined images I. R The global features are used to obtain the feature image F. lu And based on the feature image F lu Obtain illumination mapping Based on Retinex theory, the input image I and illumination are mapped... Perform pixel-by-pixel multiplication to output illumination image I lu To the noise reduction network;

[0011] S406. Construct a denoising network, which includes multiple illumination-guided attention modules (IGAB). Each IGAB is configured with a regularization layer, an illumination-guided multi-head self-attention module, a regularization layer, and a multi-scale dual-gated feedforward network connected in sequence. The illumination-guided multi-head self-attention module can enhance the long-range dependencies of the received image. The multi-scale dual-gated feedforward network can copy the received image and input it into two branches. One branch can extract the detailed information of the received image, and the other branch can extract the information of the adjacent pixels of the received image, highlighting the important pixel information in both branches. After feature enhancement of the images in the two branches, feature fusion is performed, and the fused image is output.

[0012] S60. Use training sample set B to train the model. Calculate the loss between the enhanced image and the real image obtained after training the original low-light image based on the mean absolute error loss function. Update the model weights based on the loss until the preset training conditions are met.

[0013] S80. Obtain the enhancement results of the original low-light images in the test sample set A through the model.

[0014] In one embodiment of the present invention, S402, constructing a color prior generation module, including:

[0015] The color prior generation module includes an input layer and a color prior generation network. The input layer sorts the pixels of the input image I in descending order according to their RGB pixel values. The color prior generation network re-divides the sorted pixels of the input image I according to preset ratio coefficients α, β, and γ to generate the color prior I. p ; Color prior I p The combined image I is output by concatenating the input image I with other channels. R The preset scaling coefficients α, β, and γ correspond to the maximum, median, and minimum values ​​of RGB pixel values ​​sorted in descending order, respectively.

[0016] In one embodiment of the present invention, the color prior generation network re-divides the pixels of the input image I, which are sorted in descending order, according to preset proportional coefficients α, β, and γ, to generate a color prior I. p ,include:

[0017] Through experimental analysis, preset scaling factors α, β, and γ were determined for the highest, middle, and lowest pixel values ​​in the RGB channels of pixels at random locations. The pixels of the input image I were then re-divided according to these preset scaling factors α, β, and γ to generate a new color prior image I with dimensions H×W×3. p:

[0018] I p (x,y,F(x,y))=αI(x,y,F(x,y)) (4)

[0019] I p (x,y,S(x,y))=βI(x,y,S(x,y)) (5)

[0020] I p (x,y,L(x,y))=γI(x,y,L(x,y)) (6)

[0021] Where I(x,y,F(x,y)), I(x,y,S(x,y)), and I(x,y,L(x,y)) represent the highest, middle, and lowest pixel values ​​in the RGB channels of a pixel at a random location in the input image I, respectively. p (x,y,F(x,y)),I p (x,y,S(x,y)),I p (x,y,L(x,y)) are the pixel values ​​after re-dividing the highest, middle, and lowest pixel values ​​in the RGB channels of the pixel at the random position based on preset scaling factors α, β, and γ, respectively.

[0022] The pixel values ​​of all pixels in the input image I, after being re-divided, are placed into a new image space of size H×W×3, serving as the color prior I. p Where H represents the height of input image I, and W represents the width of input image I.

[0023] In one embodiment of the present invention, S404, constructing an image illumination enhancement network D, includes:

[0024] The image illumination enhancement network D includes 1×1 convolutional layers and 9×9 depthwise separable convolutional layers, wherein the combined image I R Image I is input and then boosted by a 1×1 convolutional layer. R The 9×9 depth-separable convolution receives the output of a 1×1 convolutional layer to extract the combined image I. R The global features in the image are used to obtain the feature image F. lu ;

[0025] The extracted feature image F lu Copy into two feature images F lu One of the feature images F lu Output to the denoising network;

[0026] The image illumination enhancement network D also includes a 1×1×3 convolutional kernel, and another feature image F luAs input to a 1×1×3 convolution kernel, an illumination map is obtained.

[0027] Based on Retinex theory, the input image I is decomposed into a reflectance component R and an illuminance component L, and the illuminance is mapped... Multiply the input image I pixel by pixel to generate and output the illumination image I. lu The illumination image I lu It is characterized as being composed of the reflection component R and all interference factors Z.

[0028] In one embodiment of the present invention, the input image I is decomposed into a reflectance component R and an illuminance component L based on Retinex theory, and the illuminance is mapped... Multiply the input image I pixel by pixel to generate and output the illumination image I. lu ,include:

[0029] Step 1: Decompose the input image I into a reflectance component R and an illuminance component L according to Retinex theory:

[0030] I = R × L (7)

[0031] Step 2: According to formula (7), considering the interference factors in the reflection component R and illuminance component L of the input image I, we obtain the reflection component R and illuminance component L of the input image I considering the interference factors:

[0032]

[0033] in, and These represent the interference factors for the reflection component R and the illuminance component L, respectively.

[0034] Step 3: Map the input image I, considering interference factors, to the illumination. Multiplying them together yields the illuminance image I. lu :

[0035]

[0036] Among them, according to Illuminance image I lu Characterized as two parts consisting of the reflection component R and all interfering factors Z:

[0037]

[0038] In one embodiment of the present invention, S406, constructing a denoising network includes:

[0039] S4062. Construct a multi-scale dual-gated feedforward network;

[0040] S4064. Constructing an illumination-guided attention module (IGAB) based on a multi-scale dual-gated feedforward network;

[0041] S4066, Constructing a denoising network based on IGAB;

[0042] The denoising network comprises three downsampling convolutional layers with kernel sizes of 3×3, 4×4, and 4×4, and strides of 1, 2, and 2, respectively. The illumination image I... lu As input to this denoising network, the input of the first downsampling convolutional layer receives the illumination image I. lu The first downsampling convolutional layer is connected to the second downsampling convolutional layer via a first IGAB module. The second downsampling convolutional layer is connected to the third downsampling convolutional layer via a second IGAB module. The third downsampling convolutional layer is then connected to a third IGAB module. The result is input to a 2×2 deconvolutional layer with a stride of 2 to obtain the first deconvolution result. The result from the second IGAB module is then concatenated with the first deconvolution result to obtain the first concatenated image. This first concatenated image is input to a 1×1 convolutional layer. The denoising network also includes a fourth IGAB module, a 1×1 convolutional layer. The output of the first IGAB module is used as the input to the fourth IGAB module. The output of the fourth IGAB module is then fed into a 2×2 deconvolution layer with a stride of 2 to obtain the second deconvolution result. The result obtained from the first IGAB module is concatenated with the second deconvolution result to obtain the second stitched image. The second stitched image is then fed into a convolutional layer with a kernel size of 1×1. The denoising network also includes a fifth IGAB module. The output of the second stitched image from the 1×1 convolutional layer is used as the input to the fifth IGAB module. The output of the fifth IGAB module is then fed into a 3×3 convolutional layer with a stride of 1 to form the denoising component I. re , denoise component I re Illumination image generated by image illumination enhancement network D lu The final enhanced image I is obtained by summing element by element. en The first IGAB module includes one IGAB, and the second to fifth IGAB modules each have two connected IGABs.

[0043] In one embodiment of the present invention, S4062, constructing a multi-scale dual-gated feedforward network includes:

[0044] The multi-scale dual-gated feedforward network includes a 1×1 high-dimensional convolutional kernel, as well as a first branch and a second branch. The first branch includes a 5×5 depthwise separable convolution and a GELU activation function, and the second branch includes a 3×3 depthwise separable convolution and a GELU activation function. The input end of the 1×1 high-dimensional convolutional kernel serves as the input end of the multi-scale dual-gated feedforward network. The image received by the input end of the multi-scale dual-gated feedforward network is copied after passing through the 1×1 high-dimensional convolutional kernel and input to the first branch and the second branch respectively. Specifically, the first convolution result is obtained by passing through the 5×5 depthwise separable convolution, and the first convolution result is obtained by passing through the GELU activation function to obtain the first filtering result. The second convolution result is obtained by passing through the 3×3 depthwise separable convolution, and the second convolution result is obtained by passing through the GELU activation function to obtain the second filtering result.

[0045] The first filtering result B is obtained by multiplying the first filtering result by the second convolution result pixel by pixel. 1：

[0046] B1 = GELU(DW 5×5 (Conv 1×1 (I in )))×DW 3×3 (Conv 1×1 (I in (11)

[0047] Among them, DW 3×3 This indicates the 3×3 depthwise separable convolution, DW 5×5 This indicates the 5×5 depthwise separable convolution, Conv 1×1 This represents a convolution operation of size 1×1, and GELU represents the GELU activation function.

[0048] The second filtering result is multiplied pixel-by-pixel with the first convolution result to obtain the second branch filtering result B2:

[0049] B2 = GELU(DW 3×3 (Conv 1×1 (I in )))×DW 5×5 (Conv 1×1 (I in (12)

[0050] The multi-scale dual-gated feedforward network also includes a 1×1 low-dimensional convolutional kernel. This 1×1 low-dimensional convolutional kernel can ensure that the dimension of the image after fusing the filtering results B1 of the first branch and B2 of the second branch is consistent with the dimension of the image received at the input of the multi-scale dual-gated feedforward network, thus obtaining the fusion result S of the two branches.

[0051] S = Conv 1×1(B1+B2) (13)

[0052] The fused result S is added to the image received at the input of the multi-scale dual-gated feedforward network to obtain the denoised image I. M And output it.

[0053] In one embodiment of the present invention, S4064, constructing an illumination-guided attention module (IGAB) based on a multi-scale dual-gated feedforward network, includes:

[0054] The IGAB module, from front to back, consists of a first regularization layer, an illumination-guided multi-head self-attention module, a second regularization layer, and a multi-scale dual-gated feedforward network (MDGFN). A residual connection is established before the first and second regularization layers, and another residual connection is established before the second regularization layer and after the MDGFN.

[0055] The image received by the first regularization layer serves as the first input to the IGAB. The first regularization layer can normalize the first input, and the feature image F generated by the image illumination enhancement network D... lu As the second input to IGAB, it is directly input to the illumination-guided multi-head self-attention module, which consists of two branches. One branch reshapes the first input into a first reshaped image X, and divides the first reshaped image X into k groups of heads.

[0056] X = [X1, X2, ..., X i ,...,X k (15)

[0057]

[0058] Where k is an integer greater than or equal to 1, i∈{1,2,3…,k}, d k X represents the number of channels assigned to each group. i The dimensions are HW×d k H represents the height of the input image I, W represents the width of the input image I, and C represents the number of channels of the image, where C is an integer in the range of 32-512.

[0059] Another branch will use the feature image F lu Reconstruct the image into a second reconstructed image Y, such that the feature image F... lu Maintaining the shape of the first reshaped image X, and dividing it into k groups of heads:

[0060] Y = [Y1, Y2, ..., Y] i ,...,Y k (18)

[0061] Among them, Y i The dimensions are also HW×d k The self-attention of each head group is represented as:

[0062]

[0063] Among them, Q i Indicates based on X i The generated query vector, K i Indicates based on X i The generated key vector, V i Indicates K i The corresponding value vector, This represents block-by-block multiplication, where softmax represents the activation function. The k groups of headers are concatenated and positional encoding is added to generate the output image X. out .

[0064] In one embodiment of the present invention, α>β>γ.

[0065] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0066] The low-light image enhancement method based on Retinex theory and guided by color prior provided in this invention utilizes a low-light image enhancement network model guided by color prior based on Retinex theory. First, based on Retinex theory, the pixel values ​​of the R, G, and B channels of the original image are re-divided. Then, a color prior generation module generates a color prior I. p Color prior I p This image is then stitched together with the original image I to form a composite image I. R This approach guides low-light image enhancement, effectively addressing the issue of some algorithms deviating the original color of the object from the enhanced region. The image illumination enhancement network D enhances the pixel values ​​of the image to generate an illumination image I. lu and feature image F lu Illuminance image I lu The input is fed into a denoising network, where it is denoised using a three-layer U-shaped network consisting of multiple illumination-guided attention modules and convolutional layers. The illumination-guided attention modules introduce a multi-scale dual-gated feedforward network to denoise the image at different scales. This dual-gated feedforward network first extracts image information at different scales, filters some noise at one scale using the GELU gating function, then multiplies the filtered image pixel-by-pixel with the unfiltered image at another scale to highlight key content. Finally, the results obtained at different scales are summed element-by-element to obtain the enhanced image, improving the quality of the normal areas of the image and forming the final enhanced image. This solves the problems of some enhanced areas deviating from the original color in the enhanced image and the amplification of noise during the enhancement of low-light images.

[0067] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0068] Figure 1 This is a flowchart illustrating a low-light image enhancement method based on Retinex theory and guided by color prior, provided in an embodiment of the present invention.

[0069] Figure 2 This is a schematic diagram of the color prior generation module in a low-light image enhancement network model provided in an embodiment of the present invention;

[0070] Figure 3 This is a schematic diagram of the illumination enhancement network structure in a low-light image enhancement network model provided in an embodiment of the present invention;

[0071] Figure 4 yes Figure 1 A schematic diagram illustrating the process of constructing the denoising network in the provided low-light image enhancement method;

[0072] Figure 5 This is a schematic diagram of the structure of a multi-scale dual-gated feedforward network in a low-light image enhancement network model provided in an embodiment of the present invention;

[0073] Figure 6 This is a schematic diagram of the IGAB module in a low-light image enhancement network model provided in an embodiment of the present invention;

[0074] Figure 7 This is a schematic diagram of the IG-MSA structure in a low-light image enhancement network model provided in an embodiment of the present invention;

[0075] Figure 8 This is a schematic diagram of the denoising network structure in a low-light image enhancement network model provided in an embodiment of the present invention;

[0076] Figure 9 (a) to Figure 9 (d) are schematic diagrams comparing the enhancement of four low-light images using the low-light image enhancement method provided by this invention with the traditional Retinexformer method.

[0077] Figure 10 (a) to Figure 10 (c) are schematic diagrams comparing the enhancement of three low-light images using the low-light image enhancement method provided by this invention with the traditional Retinexformer method. Detailed Implementation

[0078] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following detailed description of the solution according to the present invention is provided in conjunction with the accompanying drawings and specific embodiments.

[0079] The foregoing and other technical contents, features, and effects of the present invention will be clearly presented in the following detailed description of specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a more in-depth and concrete understanding can be gained of the technical means and effects adopted by the present invention to achieve its intended purpose. However, the accompanying drawings are for reference and illustration only and are not intended to limit the technical solutions of the present invention.

[0080] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that an article or apparatus comprising a list of elements includes not only those elements but also other elements not expressly listed. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or apparatus that includes said element.

[0081] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a low-light image enhancement method based on Retinex theory and guided by color prior, provided by an embodiment of the present invention. This method will be referred to as the low-light image enhancement method, which includes the following steps:

[0082] S20. Obtain test sample set A and training sample set B;

[0083] Both the test sample set A and the training sample set B contain multiple original low-light images from different data sources, and the image data in the test sample set A and the training sample set B do not overlap. Specifically, a number of low-light images are randomly selected from publicly available low-light image datasets as the sample image dataset. From the sample image dataset, 10% of the images are randomly selected as the test sample set A, and the remaining 90% of the images are used as the training sample set B. Specifically, in this embodiment, images from three data subsets—LOL-v2-syn, SMID, and SDSD-out—are selected as the sample image dataset. 90% of the images in each data subset are used as the training sample set B, and the remaining 10% of the reference images are used as the test sample set B. This setup ensures that there are no image data sets with the same scene and content in the training and test sets.

[0084] S40. Construct a low-light image enhancement network model based on Retinex theory and guided by color prior.

[0085] This low-light image enhancement network model, based on Retinex theory and guided by color prior (hereinafter referred to as the enhancement network model or low-light image enhancement network model), includes the following steps in its construction:

[0086] S402. Construct a color prior generation module. The color prior generation module can re-divide the received input image I according to the RGB pixel values ​​based on preset scaling coefficients α, β, and γ to generate a color prior I. p Color prior I p Combined with input image I to form composite image I R ;

[0087] S404. Construct an image illumination enhancement network D. The image illumination enhancement network D receives the combined image I. R It can extract combined images I R The global features are used to obtain the feature image F. lu , to feature image F lu Output, and based on the feature image F lu Obtain illumination mapping Specifically, copy the feature image F lu For the copied feature image F lu Perform convolution kernel operations to obtain illuminance mapping Based on Retinex theory, the input image I and illumination are mapped... Image enhancement is achieved through pixel-by-pixel multiplication, outputting an illumination image I. lu The illumination image I lu Characterized as consisting of the reflection component R and all interfering factors Z;

[0088] S406. Construct a denoising network. The denoising network includes multiple illumination-guided attention modules (IGAB). Each IGAB is configured with a regularization layer, an illumination-guided multi-head self-attention module, a regularization layer, and a multi-scale dual-gated feedforward network connected in sequence. The illumination-guided multi-head self-attention module can enhance the long-range dependencies of the received image. The multi-scale dual-gated feedforward network can copy the received image and input it into two branches. One branch can extract the detailed information of the received image, and the other branch can extract the information of the adjacent pixels of the received image, highlighting the pixel information that is important in both branches. After feature enhancement of the images of the two branches, feature fusion is performed, and the fused image is output.

[0089] Further, please continue to refer to Figure 1 The specific steps for constructing a low-light image enhancement network model are as follows:

[0090] S402, Construct a color prior generation module;

[0091] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the color prior generation module in a low-light image enhancement network model provided in an embodiment of the present invention. Specifically, the color prior generation module includes an input layer and a color prior generation network. The input layer sorts the pixels of the input image I in descending order according to their RGB pixel values. The color prior generation network re-divides the descendingly sorted pixels of the input image I according to preset proportional coefficients α, β, and γ to generate the color prior I. p The preset scaling coefficients α, β, and γ correspond to the maximum, median, and minimum values ​​of RGB pixel values ​​sorted in descending order, respectively, with α > β > γ; the color prior I... p The combined image I is output by concatenating the input image I with other channels. R .

[0092] Furthermore, the input layer can sort the pixels of the input image in descending order according to the RGB three-channel pixel values. Specifically, H represents the height of the input image I, W represents the width of the input image I, and the number of channels of the input image I is 3. Taking an input image I with dimensions of H×W×3 as an example, in the color prior generation module, the pixels at a random position of the input image I are sorted in descending order according to the size of the RGB three-channel pixel values, and are sorted as F(x,y), S(x,y), and L(x,y) respectively, corresponding to the highest pixel value, middle pixel value, and minimum pixel value of the pixel at the random position. The calculation formulas are as follows:

[0093] F(x,y)=MAX(R(x,y),G(x,y),B(x,y)) (1)

[0094] S(x,y)=SECOND(R(x,y),G(x,y),B(x,y)) (2)

[0095] L(x,y)=MIN(R(x,y),G(x,y),B(x,y)) (3)

[0096] In formulas (1), (2), and (3), (x, y) represents the position information of the pixel at the random location in the two-dimensional image, and MAX(), SECOND(), and MIN() represent extracting the channel numbers with the highest, middle, and lowest pixel values, respectively. Taking a pixel at position (x, y) as an example, the channel numbers with the highest, middle, and lowest pixel values ​​are R, G, and B, respectively. Therefore, F(x, y) = R, S(x, y) = G, and L(x, y) = B. The same process is applied to pixels at other positions until all pixels have obtained the highest, middle, and lowest pixel values, respectively, and F(x, y), S(x, y), and L(x, y) are assigned RGB numbers.

[0097] The color prior generation network re-divides the pixels of the input image I using a preset scaling factor, generating a color prior I of size H×W×3. p .

[0098] Specifically, through experimental analysis, preset scaling factors α, β, and γ are determined for the highest, middle, and lowest pixel values ​​in the RGB channels of pixels at random locations. The pixels of the input image I are then re-divided according to these preset scaling factors α, β, and γ to generate a color prior I of size H×W×3. p Combined with the original image I, a combined image I with a size of H×W×6 is formed. R This is used to guide subsequent image enhancement. Specifically, based on preset scaling coefficients α, β, and γ, a color prior I of size H×W×3 is generated using the following formula. p :

[0099] I p (x,y,F(x,y))=αI(x,y,F(x,y)) (4)

[0100] I p (x,y,S(x,y))=βI(x,y,S(x,y)) (5)

[0101] I p (x,y,L(x,y))=γI(x,y,L(x,y)) (6)

[0102] In formulas (4), (5), and (6), I(x,y,F(x,y)), I(x,y,S(x,y)), and I(x,y,L(x,y)) represent the highest, middle, and lowest pixel values ​​in the RGB channels of a pixel at a random location in the input image I, respectively. p (x,y,F(x,y)),I p (x,y,S(x,y)),I p(x,y,L(x,y)) represents the pixel value obtained by multiplying I(x,y,F(x,y)), I(x,y,S(x,y)), and I(x,y,L(x,y)) by scaling factors α, β, and γ, respectively. p (x,y,F(x,y)),I p (x,y,S(x,y)),I p (x,y,L(x,y)) represents the pixel values ​​after re-dividing the highest, middle, and lowest pixel values ​​in the RGB channels of the pixel at the random location based on preset scaling factors α, β, and γ, respectively.

[0103] The pixel values ​​I obtained by re-dividing all pixels of the input image I p (x,y,F(x,y)),I p (x,y,S(x,y)),I p (x,y,L(x,y)) is placed into a new image space of size H×W×3 as the color prior I. p Color prior I p Combined with input image I to form composite image I R As output, specifically, it is output to the image illumination enhancement network D.

[0104] The low-light image enhancement network model provided in this embodiment sorts the pixels of the input image I in descending order according to their RGB pixel values. The color prior generation network then re-divides the sorted pixels of the input image I according to a preset ratio coefficient to generate a color prior I. p Color prior I p The combined image I is output by concatenating the input image I with other channels. R This invention addresses the overexposure issue that occurs when pixel values ​​in a low-light area are similar, resulting in the enhanced image still having similar pixel values. The color prior generation module in this embodiment can further divide the low-light area where pixel values ​​are similar, thereby highlighting the differences between pixels at different locations with similar pixel values, reducing the probability of overexposure during image enhancement, and improving the accuracy of image illumination enhancement.

[0105] S404. Construct an image illumination enhancement network D;

[0106] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the illumination enhancement network structure in a low-light image enhancement network model provided by an embodiment of the present invention. The image illumination enhancement network D includes a 1×1 convolutional layer, which combines images I... R The input image I is boosted and combined using a 1×1 convolutional layer. R Dimensions.

[0107] Furthermore, the image illumination enhancement network D also includes a 9×9 depthwise separable convolutional layer, which receives the output of a 1×1 convolutional layer to extract the combined image I. R The global features in the image are used to obtain an upgraded H×W×C feature image F. lu H represents the height of the input image I, W represents the width of the input image I, and C represents the number of channels, where C is an integer in the range [32-512].

[0108] The extracted feature image F lu Make two copies, one of which is a feature image F lu As the first output, it is fed into the denoising network; specifically, the feature image F lu It serves as input to the illumination-guided attention module included in the subsequent denoising network.

[0109] Furthermore, the image illumination enhancement network D also includes a 1×1×3 convolutional kernel, which applies another feature image F... lu As input to a 1×1×3 convolution kernel, an illumination map is obtained.

[0110] Furthermore, based on Retinex theory, the input image I is decomposed into a reflectance component R and an illuminance component L, and this illuminance is mapped... The illumination image I is generated by multiplying the input image I pixel by pixel. lu As the second output, this illuminance image I lu It is characterized as being composed of the reflection component R and all interference factors Z.

[0111] Specifically, since most noise resides in the reflection component, based on Retinex theory, the input image I is decomposed into a reflection component R and an illuminance component L. Image enhancement is essentially the process of enhancing the illuminance component L. Separating the illuminance component L from the reflection component R is beneficial for noise processing in subsequent steps. Specifically, this illuminance mapping... Multiply the input image I pixel by pixel to generate the illumination image I. lu As the second output, the specific calculation steps are as follows:

[0112] Step 1: Decompose the input image I into a reflectance component R and an illuminance component L according to Retinex theory:

[0113] I = R × L (7)

[0114] Step 2: According to formula (7), considering the interference factors in the reflection component R and illuminance component L of the input image I, we obtain the reflection component R and illuminance component L of the input image I considering the interference factors:

[0115]

[0116] Specifically, and These represent the interference factors for the reflection component R and the illuminance component L, respectively.

[0117] Step 3: Map the input image I, considering interference factors, to the illumination. Multiplying them together yields the illuminance image I. lu :

[0118]

[0119] Specifically, according to Illuminance image I lu Characterized as two parts consisting of the reflection component R and all interfering factors Z:

[0120]

[0121] The low-light image enhancement network model provided in this embodiment extracts global features from the input image I and estimates the illumination mapping. Based on Retinex theory, the input image I is decomposed into a reflectance component R and an illuminance component L. Considering the influence of interference factors, the illuminance is mapped... Image enhancement is achieved by multiplying the input image I pixel by pixel. This allows for the separate processing of the input image I that needs to be preserved and the interference factors that need further processing. A convolutional network is used to estimate the illumination map. Instead of directly estimating the illumination image I lu Compared to traditional methods that ignore the process of separating noise components and directly estimate the illumination image, this method reduces the amplification of noise.

[0122] S406. Construct a noise reduction network.

[0123] Please refer to Figure 4 , Figure 4 yes Figure 1 The provided low-light image enhancement method includes a flowchart illustrating the process of constructing a denoising network. Specifically, constructing the denoising network includes the following steps:

[0124] S4062. Construct a multi-scale dual-gated feedforward network;

[0125] Please refer to Figure 5 , Figure 5This is a schematic diagram of the structure of a multi-scale dual-gate feed-forward network in a low-light image enhancement network model provided in an embodiment of the present invention. The multi-scale dual-gate feed-forward network (MDGFN) can copy the received image and input it into two branches. One branch can extract the detailed information of the received image, and the other branch can extract the information of the adjacent pixels of the received image, highlighting the pixel information that is important in both branches. After feature enhancement of the images of the two branches, feature fusion is performed, and the fused image is output.

[0126] Specifically, the multi-scale dual-gated feedforward network includes a 1×1 high-dimensional convolutional kernel. The input of the 1×1 high-dimensional convolutional kernel serves as the input of the multi-scale dual-gated feedforward network. The image received at the input of the multi-scale dual-gated feedforward network undergoes convolution operation through the 1×1 high-dimensional convolutional kernel. For example, the input image received at the input of the multi-scale dual-gated feedforward network is input image I. in .

[0127] This multi-scale dual-gated feedforward network also includes two branches: a first branch and a second branch. The output of the 1×1 high-dimensional convolution kernel is copied and input into the two branches respectively. Further, the first branch includes a 5×5 depthwise separable convolution (DSC) and a Gaussian Error Linear Unit (GELU) activation function. This 5×5 DSC can perceive the image over a larger area, extracting more detailed information from the image received by the multi-scale dual-gated feedforward network, resulting in the first convolution result. This detailed information can be, for example, edge pixel values ​​or relationships between pixel values. Simultaneously, using the 5×5 DSC reduces the number of parameters introduced into the model, simplifying the low-light image enhancement network model. Then, the GELU activation function filters out unimportant information and some noise from the image, yielding the first filtered result.

[0128] Furthermore, the second branch includes a 3×3 depthwise separable convolution. Compared to a 5×5 depthwise separable convolution, this 3×3 depthwise separable convolution can perceive the image within a smaller region, extract and focus on information from neighboring pixels, while reducing the number of parameters introduced into the model. This second branch also includes a GELU activation function. After extracting and focusing on information from neighboring pixels in the image within a smaller region, a second convolution result is obtained. The GELU activation function filters out unimportant small amounts of information and some noise in the neighboring pixel regions, resulting in a second filtered result. The unimportant small amounts of information in these two branches mainly refer to information from regions with low pixel values, such as black noise.

[0129] like Figure 5 As shown, the first filtering result of the first branch is multiplied pixel by pixel with the second convolution result of the second branch to obtain the filtering result B1 of the first branch:

[0130] B1 = GELU(DW 5×5 (Conv 1×1 (I in )))×DW 3×3 (Conv 1×1 (I in (11)

[0131] Specifically, D.W. 3×3 This represents a depthwise separable convolution with a size of 3×3, i.e., the 3×3 depthwise separable convolution, DW 5×5 This represents a depthwise separable convolution with a size of 5×5, i.e., the 5×5 depthwise separable convolution, Conv 1×1 This represents a 1×1 convolution operation, and GELU represents the GELU activation function.

[0132] The second filtering result of the second branch is multiplied pixel by pixel with the first convolution result of the first branch to obtain the second branch filtering result B2:

[0133] B2 = GELU(DW 3×3 (Conv 1×1 (I in )))×DW 5×5 (Conv 1×1 (I in (12)

[0134] Furthermore, the multi-scale dual-gated feedforward network also includes a 1×1 low-dimensional convolutional kernel. The sum of the filtering results B1 of the first branch and B2 of the second branch is used as the input of this 1×1 low-dimensional convolutional kernel. This 1×1 low-dimensional convolutional kernel can ensure that the dimension of the image after the sum of the filtering results B1 of the first branch and B2 of the second branch is consistent with the dimension of the image received at the input of the multi-scale dual-gated feedforward network, thus obtaining the fusion result S of the two branches.

[0135] S = Conv 1×1 (B1+B2) (13)

[0136] The fusion result S is then added to the image received at the input of the multi-scale dual-gated feedforward network to obtain the denoised image I. M Denoising image I M As the output of this multi-scale dual-gated feedforward network:

[0137] I M =S+I in (14)

[0138] The multi-scale dual-gated feedforward network, by setting a first branch including a 5×5 depthwise separable convolution and a GELU activation function, can perceive the image received by this branch over a larger area, extract more image details, and perform noise filtering. Simultaneously, a second branch including a 3×3 depthwise separable convolution and a GELU activation function is set up to extract and focus on the information of adjacent pixels in the image received by the second branch within a smaller area, and perform noise filtering. The first filtering result of the first branch is multiplied pixel-by-pixel with the second convolution result of the second branch to obtain the first branch filtering result B1, and the second filtering result of the second branch is multiplied pixel-by-pixel with the first convolution result of the first branch to obtain the second branch filtering result B2. This highlights the pixel information that is important in both branches and suppresses the information that is important only in one branch, thereby suppressing and enhancing image noise.

[0139] S4064. Construct an Illumination-Guided Attention Block (IGAB) based on a multi-scale dual-gated feedforward network;

[0140] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the IGAB module in a low-light image enhancement network model provided in an embodiment of the present invention. IGAB can denoise the received image and enhance the long-range dependencies between image pixels.

[0141] The IGAB consists of a first regularization layer, an illumination-guided multi-head self-attention module (IG-MSA), a second regularization layer, and the multi-scale dual-gated feedforward network from front to back. A residual connection is established between the first and second regularization layers before the first regularization layer, and a residual connection is established between the second regularization layer and the multi-scale dual-gated feedforward network after the second regularization layer.

[0142] Specifically, the image received by the first regularization layer serves as the first input to the IGAB; for example, the first input could be a denoised image I. M The first regularization layer can normalize the first input and normalize the feature image F generated in the intermediate image illumination enhancement network D in step S404. lu As the second input of the IGAB, and directly input to the illumination-guided multi-head self-attention module, the IG-MSA module can extract the relationship between pixel values ​​from the entire image globally from the normalized first input. The IG-MSA consists of two branches, one of which reshapes the first input into a two-dimensional image. Specifically, the first input is reshaped into a first reshaped image X of size HW×C, and then the first reshaped image X is divided into k groups of heads. The calculation formulas include formula (15) and formula (16):

[0143] X = [X1, X2, ..., X i ,...,X k (15)

[0144]

[0145] Specifically, k is the number of groups into which the channel is divided, k is an integer greater than or equal to 1, i∈{1,2,3,…,k}, X i Let X represent the i-th group head of the first reconstructed image X. i The dimensions are HW×d k d k H represents the number of channels assigned to each group, W represents the height of the input image I, C represents the width of the input image I, and C is an integer in the range [32-512].

[0146] Use three unbiased fully connected layers X i Linear projection onto query vector Q i Key vector K i Value vector V i As shown in formula (17):

[0147]

[0148] Specifically, Q i Indicates based on Xi The generated query vector, K i Indicates based on X i The generated key vector, V i Indicates K i The corresponding value vector, T represents the matrix transpose.

[0149] Another branch of this IG-MSA will use the feature image F generated in the intermediate part of the image illumination enhancement network D in step S404. lu As input features, the feature image F lu Reconstructing it into a two-dimensional image, specifically, reconstructing the feature image F... lu Reconstruct a second reconstructed image Y of size HW×C, such that the feature image F lu Consistent with the shape of the first reshaped image X, and divided into k groups of heads, as shown in the following formula:

[0150] Y = [Y1, Y2, ..., Y] i ,...,Y k (18)

[0151] Specifically, k is an integer greater than or equal to 1, i∈{1,2,3…,k}, Y i Y represents the i-th head of the second reconstructed image Y. i The dimensions are also HW×d k ,Right now H represents the height of input image I, W represents the width of input image I, and C represents the number of channels in the image, where C is an integer in the range [32-512]. The self-attention of each head group is represented as:

[0152]

[0153] Specifically, α i Let i represent the learnable parameters of adaptive scaling matrix multiplication, i∈{1,2,3,…,k}. This represents block-by-block multiplication, and softmax represents the activation function. Then, the k group headers are concatenated and positional encoding is added to generate the output image X. out .

[0154] In this embodiment, k = 1, such as Figure 7 As shown, the query vector Q i This example uses query vector Q and key vector K. i Sum vector V i Examples are given as key vector K and value vector V, respectively. The first reconstructed image X is linearly projected onto query vector Q, key vector K, and value vector V, and feature image F. lu The first input of IGAB is processed by the above formula (19) to obtain the output image X. out .

[0155] This illumination-guided attention module enhances the long-range dependencies of the received image through the configured IG-MSA. For example, when two pixels are far apart in the image, it is difficult to establish a connection through convolution operations, but the IG-MSA module can enhance the connection between pixel values ​​that are far apart.

[0156] S4066. A denoising network is constructed based on an illumination-guided attention module.

[0157] Please refer to Figure 8 , Figure 8 This is a schematic diagram of the denoising network structure in a low-light image enhancement network model provided in an embodiment of the present invention. The denoising network includes three downsampling convolutional layers with kernel sizes of 3×3, 4×4, and 4×4, and strides of 1, 2, and 2, respectively. The illumination image I... lu As input to the denoising network, the input of the first downsampling convolutional layer is used as the input of the denoising network. A first IGAB module, comprising one IGAB, is connected between the first and second downsampling convolutional layers. A second IGAB module is connected between the second and third downsampling convolutional layers. A third IGAB module is connected after the third downsampling convolutional layer. The result is input to a 2×2 deconvolutional layer with a stride of 2 to obtain the first deconvolution result. The result obtained from the second IGAB module is then concatenated with the first deconvolution result to obtain the first concatenated image. The first concatenated image is then input to a convolutional layer with a kernel size of 1×1. The denoising network also includes a fourth IGAB module. The output of the 1×1 convolutional layer is used as the input of the fourth IGAB module. The output of the fourth IGAB module is then fed into a 2×2 deconvolutional layer with a stride of 2 to obtain a second deconvolution result. The result obtained from the first IGAB module is concatenated with the second deconvolution result to obtain a second stitched image. The second stitched image is then fed into a convolutional layer with a kernel size of 1×1. The denoising network also includes a fifth IGAB module. The output of the second stitched image from the 1×1 convolutional layer is used as the input of the fifth IGAB module. The output of the fifth IGAB module is fed into a 3×3 convolutional layer with a stride of 1 to form the denoising component I. re , denoise component I re Illumination image generated by image illumination enhancement network D lu The final enhanced image I is obtained by summing element by element. en This cancels out the illumination image I. luTo reduce some of the interference factors and achieve the purpose of noise reduction, specifically, the second, third, fourth, and fifth IGAB modules are each equipped with two connected light-guided attention modules.

[0158] S60. Use training sample set B to train a low-light image enhancement network model based on Retinex theory and guided by color prior.

[0159] The training sample set B is used to train a low-light image enhancement network model based on Retinex theory and color prior guidance. The loss of the original low-light image after training the low-light image enhancement network model based on Retinex theory and color prior guidance is calculated according to the mean absolute error loss function. The weights of the model are updated according to the loss until the preset training conditions are met.

[0160] Specifically, a low-light image enhancement network model based on Retinex theory and guided by color priors is trained using a training sample set, which includes the following steps:

[0161] Step 1: Set the initial number of iterations to t, the maximum number of iterations to T, and let t = 1;

[0162] In this example, T = 150000.

[0163] Step 2: Randomly select b training samples without replacement from the training sample set B as the original image input to the low-light image enhancement network model guided by the color prior based on Retinex theory to form the enhanced image I of the original image. en ;

[0164] Specifically, the color prior generation module generates color prior I. p The generated color prior I p Combined image I is generated by stitching it with the original image I. R The original image I comes from b training samples, and the illumination image I is generated by the image illumination enhancement network D. lu Then, the illuminance image I lu The denoised component I is generated by processing with a denoising network. re Finally, the denoised component I re With illumination image I lu An enhanced image I is formed by adding pixels one by one to form the original image. en .

[0165] Step 3: Select the Mean Absolute Error (MAE) function and calculate the enhanced image I based on the MAE function. en The model weights are updated using the loss MAE of the real image.

[0166] Specifically, the enhanced image I is calculated based on the mean absolute error loss function. en The loss MAE of the real image is as follows:

[0167]

[0168] Specifically, W represents the width of the real image, H represents the height of the real image, Re(x,y) represents the pixel value at position (x,y) in the real image, and I... en (x,y) represents the pixel value at position (x,y) in the enhanced image.

[0169] Step 4: Repeat steps 2-3 until all original images in sample set B have been traversed and trained.

[0170] Step 5: Determine whether t = T holds. If t = T, the trained low-light image enhancement network model based on Retinex theory and guided by color prior is obtained. Otherwise, let t = t + 1 and repeat steps 2 to 3 until t = T to obtain the low-light image enhancement network model based on Retinex theory and guided by color prior.

[0171] S80. Using a low-light image enhancement network model guided by color prior based on Retinex theory, obtain the enhancement results of the original low-light images in the test sample set A.

[0172] The test sample set A is used as input to the trained Retinex-based color prior-guided low-light image enhancement network model. Forward inference is performed to obtain the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) of the test sample set A to verify the model's performance.

[0173] The technical effects of this invention will be explained below with reference to simulation experiments:

[0174] To verify the effectiveness of the proposed method on low-light image enhancement tasks, experiments were conducted on the LOL-v2-syn, SMID, and SDSD-out public datasets using the method described in this application. The experimental results were compared with those of other algorithms on the PSNR and SSIM evaluation metrics. Table 1 shows the performance comparison between the method provided in this application and general low-light image enhancement methods:

[0175] Table 1

[0176]

[0177] Table 1 lists IPT, RetinexNet, Sparse, EnGAN, RUAS, FIDE, DRBN, KinD, Restormer, MIRNet, SNR-Net, and Retinexformer as general low-light image enhancement methods, and LOL-v2-syn, SMID, and SDSD-out as three different data sources. The intersection of columns and rows represents the PSNR and SSIM values ​​of different methods for the same low-light image from the same data source. The last row shows the PSNR and SSIM values ​​of the proposed method for the same low-light image from the same data source. The table shows that the proposed method achieves the highest PSNR and SSIM values ​​on multiple datasets, and the SSIM value is higher than that of general low-light image enhancement methods.

[0178] Please refer to Figure 9 and Figure 10 , Figure 9 (a) to Figure 9 (d) are schematic diagrams comparing the enhancement of four low-light images using the low-light image enhancement method provided by this invention with the traditional Retinexformer method. Figure 10 (a) to Figure 10 (c) These are schematic diagrams comparing three low-light images enhanced using the low-light image enhancement method provided by this invention with the traditional Retinexformer method. It is clearly shown in the figures that, compared to the traditional Retinexformer method, the images enhanced by the method of this application have colors closer to reality, clearer image details, smoother images, better noise reduction, and better preservation of background details. The figures also show that the texture characteristics of the sky and lake water are fully reflected in the enhanced images of this application.

[0179] The low-light image enhancement method based on Retinex theory and guided by color prior provided in this invention utilizes a low-light image enhancement network model guided by color prior based on Retinex theory. First, based on Retinex theory, the pixel values ​​of the R, G, and B channels of the original image are re-divided. Then, a color prior generation module generates a color prior I. p Color prior I p This image is then stitched together with the original image I to form a composite image I. R This method guides low-light image enhancement, effectively solving the problem of enhancement areas deviating from the original object colors in some algorithms. The image illumination enhancement network D enhances the pixel values ​​of the image to generate an illumination image I. lu and feature image F lu Illuminance image Ilu The input is fed into a denoising network, where it undergoes denoising through a three-layer U-shaped network consisting of multiple illumination-guided attention modules and convolutional layers. The illumination-guided attention modules introduce a multi-scale dual-gated feedforward network to denoise the image at different scales. This dual-gated feedforward network first extracts image information at different scales, filters some noise at one scale using the GELU gating function, then multiplies the filtered image pixel-by-pixel with the unfiltered image at another scale to highlight key content. Finally, the results obtained at different scales are summed element-by-element to obtain the enhanced image, improving the quality of the normal areas of the image and forming the final enhanced image. This improves the denoising effect and image quality, and solves the problems of some enhanced areas deviating from the original color in the enhanced image and the amplification of noise during the enhancement of low-light images.

[0180] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A low-illumination image enhancement method based on Retinex theory and guided by color prior, characterized in that, The method comprises the following steps: S20, acquiring a test sample set A and a training sample set B, wherein the test sample set A and the training sample set B are both provided with a plurality of original low-illumination images of different data sources; S40, constructing a low-illumination image enhancement network model based on the Retinex theory and guided by color prior, comprising: S402, construct a color prior generation module, the color prior generation module can be based on the preset proportion coefficient α, β, γ, the input image I received according to RGB pixel value is reclassified, and the color prior I is generated p , the color prior I p is spliced with the input image I to form a combined image I R ; S404, construct an image illumination enhancement network D, the image illumination enhancement network D can extract the global features of the combined image I to obtain a feature image F R lu , and obtain an illumination mapping based on the feature image F lu Based on the Retinex theory, the input image I and the illumination mapping are multiplied pixel by pixel to output an illumination image I lu to a denoising network;​​ S406, constructing a denoising network, wherein the denoising network comprises a plurality of illumination-guided attention modules (IGAB), and each IGAB is provided with a regularization layer, an illumination-guided multi-head self-attention module, a regularization layer and a multi-scale double-gated feedforward network connected in sequence, the illumination-guided multi-head self-attention module can enhance the long-range dependency of the received image, and the multi-scale double-gated feedforward network can copy the received image and input the copied image into two branches respectively, one of the two branches can extract the detail information of the received image, and the other branch can extract the information of the adjacent pixels of the received image, highlight the pixel information important in both branches, perform feature enhancement on the images of the two branches, perform feature fusion, and output the fused image; S60, training the model using the training sample set B, calculating the loss of the enhanced image obtained by training the original low-illumination image through the model and the real image according to the mean absolute error loss function, updating the weight of the model according to the loss, and stopping until a preset training condition is reached; S80, obtaining the enhancement result of the original low-illumination image in the test sample set A through the model.

2. The Retinex theory based low-light image enhancement method guided by color prior according to claim 1, characterized in that, S402, constructing a color prior generation module, comprising: The color prior generation module comprises an input layer and a color prior generation network, the input layer sorts pixel points of an input image I in descending order according to RGB pixel values; the color prior generation network re-divides the pixel points of the input image I sorted in descending order according to preset proportion coefficients alpha, beta and gamma, and generates a color prior I p ; and outputs a combined image I p after the color prior I R and the input image I are connected in series through a channel. Among them, the preset proportion coefficients alpha, beta and gamma correspond to the maximum value, the intermediate value and the minimum value of the RGB pixel values sorted in descending order respectively.

3. The Retinex theory based low-light image enhancement method guided by color prior according to claim 2, characterized in that, The color prior generation network re-divides the pixel points of the input image I in descending order according to preset proportion coefficients alpha, beta and gamma, to generate a color prior I p , comprising: The preset proportion coefficients alpha, beta and gamma of the highest pixel value, the intermediate pixel value and the minimum pixel value in the RGB channel of the pixel point at the random position are determined through experimental analysis, the pixels of the input image I are re-divided according to the preset proportion coefficients alpha, beta and gamma, and a new color prior I with the size of HxWx3 is generated p : I p (x,y,F(x,y)) = a I (x,y,F(x,y)) (4) I p (x,y,S(x,y)) = βl(x,y,S(x,y)) (5) I p (x,y,L(x,y)) = γl(x,y,L(x,y)) (6) wherein I(x, y, F(x, y)), I(x, y, S(x, y)), I(x, y, L(x, y)) represent the highest, intermediate, and minimum pixel values in the RGB channel of the pixel at the random position of the input image I, respectively, I p (x, y, F(x, y)), I p (x, y, S(x, y)), I p (x, y, L(x, y)) are pixel values re-divided based on preset proportion coefficients a, b, g on the highest, intermediate, and minimum pixel values in the RGB channel of the pixel at the random position. The pixel values of all the pixel points of the input image I after re-partitioning are put into a new image space with a size of HxWx3 as the color prior I p where H represents the height of the input image I, and W represents the width of the input image I.

4. The low-light image enhancement method guided by color prior based on Retinex theory according to claim 1, characterized in that, S404, constructing an image illumination enhancement network D, comprising: The image illumination enhancement network D includes a 1x1 convolution layer and a 9x9 depth separable convolution layer, wherein the combined image I R is input to the 1x1 convolution layer to increase the dimension of the image I R , and the 9x9 depth separable convolution receives the output of the 1x1 convolution layer to extract global features in the combined image I R to obtain a feature image F lu ; The extracted feature image F lu Copies are made of the two feature images F lu One of the feature images F lu Output to the denoising network; The image illumination enhancement network D further comprises a 1x1x3 convolution kernel, another feature image F lu As an input of the 1x1x3 convolution kernel, an illumination map is obtained Based on the Retinex theory, the input image I is decomposed into a reflection component R and an illumination component L, and the illumination is mapped Pixel by pixel multiplied with the input image I, an illumination image I lu is generated and output, the illumination image I lu is characterized by consisting of the reflection component R and all interference factors Z.

5. The Retinex theory based low-light image enhancement method guided by color prior according to claim 4, characterized in that, based on the Retinex theory, the input image I is decomposed into a reflection component R and an illumination component L, the illumination is mapped pixel by pixel with the input image I, an illumination image I is generated and output lu , comprising: Step 1, decomposing the input image I into a reflection component R and an illumination component L according to the Retinex theory: I = R x L (7) Step 2, according to formula (7), considering the interference factors of the reflection component R and the illumination component L of the input image I, obtaining the reflection component R and the illumination component L of the input image I considering the interference factors: wherein, and respectively represent the interference factor of the reflection component R and the interference factor of the illumination component L; Step 3, multiplying the input image I taking into account the interference factors with the illumination map to obtain the illumination image I lu : wherein, according to The luminance image I lu is characterized as being composed of two parts: a reflection component R and all disturbing factors Z:

6. The Retinex theory based low-light image enhancement method guided by color prior according to claim 1, characterized in that, S406, constructing a denoising network, comprising: S4062, constructing a multi-scale double-gated feedforward network; S4064, constructing an illumination-guided attention module (IGAB) based on the multi-scale double-gated feedforward network; S4066, constructing a denoising network based on the IGAB; The denoising network comprises three down-sampling convolutional layers, and the convolution kernel sizes of the three down-sampling convolutional layers are 3*3, 4*4 and 4*4 respectively, and the steps are 1, 2 and 2 respectively, and the illumination image I lu As the input of the denoising network, the input end of the first down-sampling convolutional layer receives the illumination image I as the input end of the denoising network lu A first IGAB module is connected between the first down-sampling convolutional layer and the second down-sampling convolutional layer, a second IGAB module is connected between the second down-sampling convolutional layer and the third down-sampling convolutional layer, and a third IGAB module is connected after the third down-sampling convolutional layer, and the obtained result is input into a deconvolutional layer with a size of 2*2 and a step of 2 to obtain a first deconvolutional result, the result obtained by the second IGAB module is spliced with the first deconvolutional result, and a first spliced image is obtained after splicing, the first spliced image is input into a convolutional layer with a convolution kernel size of 1*1, the denoising network further comprises a fourth IGAB module, the result output by the convolutional layer with a size of 1*1 is input into the fourth IGAB module, the output result of the fourth IGAB module is input into a deconvolutional layer with a size of 2*2 and a step of 2 to obtain a second deconvolutional result, the result obtained by the first IGAB module is spliced with the second deconvolutional result to obtain a second spliced image, the second spliced image is input into a convolutional layer with a convolution kernel size of 1*1, the denoising network further comprises a fifth IGAB module, the result output by the second spliced image through the convolutional layer with a convolution kernel size of 1*1 is input into the fifth IGAB module, and the fifth IGAB module is output to a convolutional layer with a size of 3*3 and a step of 1 to form a denoising component I re The denoising component I re is element-wise summed with the illumination image I lu generated by the image illumination enhancement network D to obtain a final enhanced image I en , wherein the first IGAB module comprises one IGAB, and the second IGAB module to the fifth IGAB module are each provided with two connected IGABs.

7. The Retinex theory based low-light image enhancement method guided by color prior according to claim 6, characterized in that, S4062, constructing a multi-scale double-gated feedforward network, comprising: The multi-scale double-gated feedforward network comprises a 1x1 high-dimensional convolution kernel, a first branch and a second branch, the first branch comprises a 5x5 depth separable convolution and a GELU activation function, the second branch comprises a 3x3 depth separable convolution and a GELU activation function, the input end of the 1x1 high-dimensional convolution kernel serves as the input end of the multi-scale double-gated feedforward network, the image received by the input end of the multi-scale double-gated feedforward network is copied and input into the first branch and the second branch after passing through the 1x1 high-dimensional convolution kernel, wherein the first convolution result is obtained through the 5x5 depth separable convolution, the first filtering result is obtained through the GELU activation function, the second convolution result is obtained through the 3x3 depth separable convolution, and the second filtering result is obtained through the GELU activation function; The first filtering result is pixel by pixel multiplied with the second convolution result to obtain a first branch filtering result B1: B1 = GELU(DW 5×5 (Conv 1×1 (I in ))) x DW 3×3 (Conv 1×1 (I in )) (11) wherein, DW 3×3 represents the 3x3 depth separable convolution, DW 5×5 represents the 5x5 depth separable convolution, Conv 1×1 represents a convolution operation with size 1x1, and GELU represents a GELU activation function. The second filtering result is pixel by pixel multiplied with the first convolution result to obtain a second branch filtering result B2: B2 = GELU(DW 3×3 (Conv 1×1 (I in ))) x DW 5×5 (Conv 1×1 (I in )) (12) The multi-scale double-gated feedforward network further comprises a 1*1 low-dimension convolution kernel, which can keep the dimension of the image processed by fusing the first branch filtering result B1 and the second branch filtering result B2 consistent with the dimension of the image received by the input end of the multi-scale double-gated feedforward network, and obtain a fusion result S of the two branches: S = Conv 1×1 (B1+B2) (13) The fusion result S obtained is added to the image received by the input end of the multi-scale double-gated feedforward network to obtain a denoised image I M and output.

8. The low-light image enhancement method guided by color prior based on Retinex theory according to claim 6, characterized in that, S4064, the illumination-guided attention module (IGAB) is constructed based on the multi-scale double-gated feedforward network, comprising: The IGAB module is composed of a first regularization layer, an illumination-guided multi-head self-attention module, a second regularization layer and a multi-scale double-gated feedforward network (MDGFN) from front to back, and a residual connection is established before the first regularization layer and before the second regularization layer, and a residual connection is established before the second regularization layer and after the multi-scale double-gated feedforward network, wherein, The received image of the first regularization layer as the first input of the IGAB, the first regularization layer can normalize the first input, and the feature image F generated by the image illumination enhancement network D lu As the second input of the IGAB, it is directly input to the light-guided multi-head self-attention module, which is composed of two branches, one of which reshapes the first input into a first reshaped image X, and divides the first reshaped image X into k groups of heads: X = [X1, X2,..., X i ,...,X k ] (15) wherein k is an integer greater than or equal to 1, i ∈ {1, 2, 3, …, k}, d k represents the number of channels distributed in each group, X i The size of is HW x d k H represents the height of the input image I, W represents the width of the input image I, C represents the number of channels, and C is an integer in [32-512]. Another branch, the feature image F lu is reshaped into a second reshaped image Y such that the feature image F lu is consistent with the shape of the first reshaped image X and is divided into k groups of heads: Y = [Y1, Y2,..., Y i ,...,Y k ] (18) where Y i The size of each group head is also HW x d k The self-attention representation of each group head is: wherein Q i represents the generated query vector based on X i represents the generated key vector based on X i represents the generated value vector based on X i represents the generated key vector based on X i represents the generated value vector based on X i represents the generated value vector based on X represents the block-wise multiplication, softmax represents an activation function, the k groups of heads are connected to generate an output image X out .

9. The Retinex theory based low-light image enhancement method guided by color prior according to any one of claims 1-8, characterized in that, α>β>γ.

Images