Low-illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction

Abstract

The invention provides a low-illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction. Comprising the following steps: constructing a training data set, and performing data enhancement on paired low-illumination / normal-illumination images to obtain an image data set for training; designing a low-illumination image enhancement network based on adaptive brightness enhancement and high-fidelity color correction; designing a loss function for training the designed network; training the designed network by using the training data set; and inputting a to-be-measured low-illumination image into the trained network to generate a normal-illumination image. According to the method, the adaptive brightness improvement strategy and the color correction under the multi-color space constraint are utilized, the image brightness can be adaptively enhanced, the image color is subjected to high-fidelity correction, and a high-quality image with proper brightness and high-fidelity color is obtained; the problems of uneven brightness, color distortion, detail loss and the like of the low-illumination image are effectively solved.

Description

Technical Field

[0001] The present invention relates to the fields of image and video processing and computer vision, and in particular to a low-illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction. Background Art

[0002] In the field of modern image processing and computer vision, low-light image enhancement technology has received increasing attention. Low-light images usually appear at night, in dark environments, or under unfavorable shooting conditions. At this time, the number of photons received by the sensor is limited, which often leads to problems such as insufficient brightness and color distortion in the acquired images, which in turn affects the visual perception and recognizability of the image. As a critical and challenging task, low-light image enhancement aims to improve the brightness and clarity of low-light images while maintaining the accuracy of content and color, so that it can be widely used in various downstream tasks and play an indispensable role in night photography, autonomous driving, remote sensing and other fields. Therefore, it has received widespread attention from researchers in related fields.

[0003] In security monitoring, low-light image enhancement methods can improve blurry or noisy images captured by surveillance cameras at night or in low-light environments, thereby improving the efficiency of public safety and crime detection. In medical imaging, low-light enhancement methods can improve the visibility and details of images acquired by medical imaging equipment under low-light conditions, and improve the accuracy of early disease detection. Low-light image enhancement methods not only improve the quality and usability of images, but also play an important role in improving security, accuracy and efficiency, and promote the development and innovation of various industries.

[0004] Early traditional low-light image enhancement methods were mainly based on histogram equalization or Retinex theory. Among them, histogram equalization uses the cumulative distribution function of the image to redistribute the grayscale values ​​of the image to enhance the contrast of the image. Retinex theory aims to explain how the human visual system perceives color and brightness under different lighting conditions. It believes that the original image can be decomposed into two parts: a reflectance map and an illumination map. The reflectance map represents the inherent color and texture of the object surface, while the illumination map represents the distribution of light intensity in the scene. Although these methods have achieved certain results in early low-light image enhancement, due to their reliance on mathematical statistics and manually designed mappings, they are prone to problems such as detail loss and over-enhancement during the enhancement process and introduce artifacts.

[0005] With the development of deep learning, methods based on deep learning have been deeply explored and studied in recent years and have been widely used. For example, some methods introduce Fourier transform into convolutional neural networks, use the correlation between frequency domain amplitude and brightness to achieve brightness enhancement, and design signal-to-noise ratio mapping to improve details. There are also some methods that combine Retinex theory with convolutional neural networks to achieve low-light image enhancement by decomposing the reflection component and illumination component of the original image, or through decomposition, reflection map restoration and illumination enhancement three sub-networks and use loss functions to constrain the reflection component and illumination component to achieve low-light image enhancement.

[0006] However, since the Retinex theory is based on the color constancy of the human visual system under different lighting conditions, it is believed that the color of objects perceived by humans remains unchanged when the lighting changes. Although this has brought new inspiration to the field of low-light image enhancement, its overly idealistic assumption ignores the existence of self-luminous objects. The decomposition formula of its reflection part and illumination part is not applicable to self-luminous objects. Therefore, the method based on the Retinex theory is difficult to handle various artificial light sources in real scenes, resulting in overexposure and color cast. In addition, for real low-light images with uneven lighting areas, although some methods introduce signal-to-noise ratio information, frequency domain information or illumination maps to achieve adaptive brightness improvement, overexposure may still occur because these methods fail to accurately identify the parts that need to increase brightness and suppress brightness. At the same time, some methods have less targeted design for color, and the constraints on color are limited to the RGB color space, resulting in color distortion in real scenes. Therefore, it is challenging and necessary to study low-light image enhancement methods for real low-light scenes that can adaptively increase brightness and accurately correct color.

[0007] The present invention aims to further solve the problems of over-enhancement, color distortion and detail loss caused by it in low-light image enhancement methods, and designs a low-light image enhancement method based on adaptive brightness enhancement and high-fidelity color correction. First, an adaptive brightness enhancement subnetwork is designed to achieve adaptive enhancement of image brightness, including a multi-stage dual attention residual module for accurately extracting image brightness priors; then a high-fidelity color correction subnetwork is designed to achieve high-fidelity correction of image color, including a progressive multi-scale feature fusion module and an adaptive multi-receptive field feature extraction module for fully extracting and fusing image color features from multiple scales, and a multi-color space loss function for effectively guiding and constraining the color correction process to obtain a color high-fidelity image; finally, an image output module is designed to convert the final enhanced features into a high-quality image with appropriate final brightness and high color fidelity. This low-light image enhancement method based on adaptive brightness enhancement and high-fidelity color correction can effectively avoid the occurrence of overexposure and color distortion during the enhancement process, and effectively improve the quality of the image Summary of the invention

[0008] The purpose of the present invention is to propose a low-illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction. The method combines spatial attention and channel attention to extract brightness prior, and can adaptively enhance image brightness to avoid overexposure. At the same time, a progressive multi-scale feature fusion module is designed to extract and fuse color features, and a loss function based on multi-color space is introduced to constrain and guide the color correction process to achieve high-fidelity color correction, so that the model can better enhance low-illumination images in real scenes.

[0009] To achieve the above object, the technical solution of the present invention is: a low illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction, comprising the following steps:

[0010] Step S1, constructing a training data set, performing data enhancement on paired low illumination / normal illumination images, and obtaining an image data set for training;

[0011] Step S2, designing a low-illumination image enhancement network based on adaptive brightness enhancement and high-fidelity color correction, wherein the network consists of an adaptive brightness enhancement subnetwork and a high-fidelity color correction subnetwork;

[0012] Step S3, designing a loss function for training the network designed in step S2;

[0013] Step S4, using the training data set obtained in step S1 to train the low-illumination image enhancement network based on adaptive brightness enhancement and high-fidelity color correction in step S2, to obtain a trained low-illumination image enhancement network based on adaptive brightness enhancement and high-fidelity color correction;

[0014] Step S5: input the low-illuminance image to be tested into a trained low-illuminance image enhancement network based on adaptive brightness enhancement and high-fidelity color correction to generate a normal-illuminance image.

[0015] Furthermore, step S1 includes the following steps:

[0016] Step S11, pairing the low illumination image with a reference image of normal illumination;

[0017] Step S12: For each pair of paired images in the training set, randomly perform one of the following data augmentation operations: horizontal flipping, vertical flipping, 90 degree rotation, 180 degree rotation, and 270 degree rotation.

[0018] Further, step S2 includes the following steps:

[0019] Step S21, designing an adaptive brightness enhancement subnetwork, including a brightness prior extraction module, a multi-stage dual attention residual module and an adaptive brightness enhancement module, for extracting the brightness prior of the low-light image and then adaptively enhancing the brightness of the low-light image to obtain an image after adaptive brightness enhancement;

[0020] Step S22, design a high-fidelity color correction subnetwork, including a color feature extraction module, a progressive multi-scale feature fusion module and a high-fidelity color image output module, which is used to extract the color features of the image after adaptive brightness enhancement and perform color correction to obtain a normal illumination image with appropriate brightness and high-fidelity color.

[0021] Further, step S21 includes the following steps:

[0022] Step S211, design a brightness prior extraction module, the brightness prior extraction module includes a 3×3 convolution layer and a Parametric ReLU activation function; for the input RGB image First, convert it to the YUV color space to obtain the images of the Y, U and V channels, where the image of the Y channel is the original brightness component of the original RGB image. Then, the designed brightness prior extraction module is used to extract the features of the original brightness component to obtain the primary brightness prior Among them, H and W are the height and width of the image respectively, and C is the number of channels of the feature;

[0023] Step S212, design a multi-stage dual attention residual module, the multi-stage dual attention residual module includes four continuously stacked dual attention residual blocks, a 3×3 convolution layer, a Parametric ReLU activation function and a residual connection operation; each dual attention residual block consists of an initial feature extraction block, a channel attention block, a spatial attention block and a feature output block; the dual attention residual block extracts and fuses local and global features at the channel and spatial levels through a dual attention mechanism, thereby realizing the perception and extraction of local and global uneven brightness; after the multi-stage dual attention residual module, a high-level brightness prior is obtained Right now:

[0024] L prior =Parametric ReLU(Conv3(DARB(DARB(DARB(DARB(L prior )))))))+L prior

[0025] Among them, Parametric ReLU(·) represents the activation function, Conv3 table represents the 3×3 convolutional layer, and DARB(·) represents the dual attention residual block;

[0026] Step S213, design an adaptive brightness enhancement module, the adaptive brightness enhancement module includes a 3×3 convolution layer, a Parametric ReLU activation function and an element-by-element division operation; first, the high-level brightness prior L' obtained in step S212 is prior Input 3×3 convolution layer and Parametric ReLU activation function to get the final brightness prior after full extraction Then the brightness component L of the original image original Divide by the final brightness prior Get the adaptively enhanced brightness component That is, the enhanced Y channel image is expressed as follows:

[0027]

[0028] in, represents element-by-element division, and ε represents a very small constant used to prevent the divisor from being zero. Finally, the U and V channel images of the original image are concatenated with the enhanced Y channel image and converted back to the RGB color space to obtain the final result image after adaptive enhancement.

[0029] Further, step S212 includes the following steps:

[0030] Step S2121, design the initial feature extraction block, for the input feature of the dual attention residual block After passing through a convolutional layer consisting of 3×3 convolution and Parametric ReLU activation function stacked twice, it is copied into two copies, represented as channel attention block input features And the spatial attention block input features

[0031] Step S2122: Design a channel attention block, and input features into the channel attention block The two channel attention weights are obtained through the global average pooling operation and the global maximum pooling operation at the channel level. and They are used to capture the global information and local information of features at the channel level respectively; then the convolution layer composed of 1×1 convolution and Parametric ReLU activation function reduces the number of channels of these two channel attention weights by half to C / 2, and then a 1×1 convolution layer is used to restore the number of channels to C; the two channel attention weights are added element by element, and then a 1×1 convolution and Sigmoid activation function are used to obtain the final channel attention weight Combine the channel attention weight with the initial features Perform element-by-element multiplication to obtain the output features of the channel attention block The specific formula is as follows:

[0032]

[0033] in, represents element-wise multiplication, Sigmoid(·) represents the activation function, Conv1 represents a 1×1 convolutional layer, GAP(·) represents the global average pooling operation, and GMP(·) represents the global maximum pooling operation;

[0034] Step S2123: Design a spatial attention block, input feature Two spatial attention weights are obtained through global average pooling and global maximum pooling at the spatial level and They are used to capture the global and local information of features at the spatial level respectively; then the two spatial attention weights are concatenated and passed through a 1×1 convolution and Sigmoid activation function to obtain the final channel attention weight Combine the channel attention weight with the initial features Perform element-by-element multiplication to obtain the output features of the spatial attention block The specific formula is as follows:

[0035]

[0036] in, represents element-wise multiplication, Sigmoid(·) represents activation function, Conv1 represents 1×1 convolutional layer, GAP(·) represents global average pooling operation, GMP(·) represents global maximum pooling operation, and [·;·] represents feature concatenation operation;

[0037] Step S2124: Design a feature output block to combine the output features of the channel attention block and the spatial attention block and After concatenation, a 3×3 convolution and a Parametric ReLU activation function are performed, and then the input features of the dual attention residual block are As the residual, add it element by element to get the output feature of the dual attention residual block The specific formula is as follows:

[0038]

[0039] Among them, Parametric ReLU(·) represents the activation function, Conv3 represents the 3×3 convolutional layer, and [·;·] represents the feature concatenation operation.

[0040] Further, step S22 includes the following steps:

[0041] Step S221, design a color feature extraction module, the color feature extraction module includes a 3×3 convolution layer and the multi-stage dual attention residual module in step S212; for the adaptively brightened image I obtained in step S21 lightened , and then input it into the color feature extraction module to obtain the initial image color feature

[0042] Step S222: Design a progressive multi-scale feature fusion module, which includes an upsampling module, a downsampling module, a long skip connection, a short skip connection, and an adaptive multi-receptive field feature fusion module; extract and fuse color features at 5 stages and 5 scales to obtain the output image color features.

[0043] Step S223, design a high-fidelity color image output module, the core of which includes a 3×3 convolution layer, a Parametric ReLU activation function, a Sigmoid activation function and a residual connection operation; the output image color features obtained in step S222 After a 3×3 convolutional layer and a Parametric ReLU activation function, the adaptively brightened image I obtained in step S21 is lightened Perform residual connection and pass a Sigmoid activation function to obtain the final color-corrected output image. That is, a high-quality image with appropriate brightness and accurate color; the specific formula is as follows:

[0044]

[0045] Among them, Sigmoid(·) and Parametric ReLU(·) represent activation functions, and Conv3 represents a 3×3 convolutional layer.

[0046] Further, step S222 includes the following steps:

[0047] Step S2221: In the first stage, a downsampling module is designed, wherein the downsampling module is composed of 3×3 convolution and Parametric ReLU activation function stacked twice; the initial color features are converted into Perform four downsampling operations in sequence to obtain downsampled features of four scales, plus the initial color features that have not been downsampled. A total of 5 features of different scales are obtained, which are represented as and Among them, μ represents the number of channels reduced or increased each time upsampling or downsampling, the first s in the feature subscript represents the scale, and the second s represents the stage;

[0048] Step S2222: In the second stage, an upsampling module is designed, wherein the upsampling module is composed of a 3×3 transposed convolution, a Parametric ReLU activation function, a 3×3 convolution, and a Parametric ReLU activation function stack; for the features F of the second to fourth scales in the first stage, s2s1 、F s3s1 and F s4s1 , and then upsample it to the previous scale using the designed upsampling module and compare it with the 1st to 3rd scale features F of the first stage respectively. s1s1 、F s2s1 and F s3s1 Perform additive fusion, i.e. short skip connection, to obtain three preliminary fused multi-scale features and The specific formula is as follows:

[0049] F s1s2 =F s1s1 +Upsample(F s2s1 ),

[0050] F s2s2 =F s2s1 +Upsample(F s3s1 ),

[0051] F s3s2 =F s3s1 +Upsample(F s4s1 )

[0052] Among them, Upsample(·) represents the designed upsampling module, and the addition of features of two adjacent stages at the same scale is a short skip connection;

[0053] Step S2223: In the second stage, for the fifth scale feature F at the deep layer s5s1First, we design an adaptive multi-receptive field feature extraction module to perform adaptive multi-receptive field feature extraction; for the input feature F s5s1 , 3×3 convolution, 5×5 convolution and 7×7 convolution are used to extract features of three different receptive fields. and After adding them together, they are fused using global average pooling to obtain the global channel weights Then, a combination of three 1×1 convolutions and Sigmoid activation functions is used to extract the feature weights for the three receptive fields. and Then, these three feature weights are multiplied by the original three different receptive field features to achieve adaptive representation of the features, and three adaptively selected different receptive field features are obtained. and Finally, the three adaptively selected different receptive field features are added together to obtain the output of the adaptive multi-receptive field feature extraction module Then, it is upsampled to the 4th scale and combined with the 4th scale feature F in the first stage s4s1 Perform addition fusion to obtain global features under multiple receptive fields The specific formula is as follows:

[0054] W channel =GAP(Conv3(F s5s1 )+Conv5(F s5s1 )+Conv7(F s5s1 )),

[0055]

[0056] F s4s2 =F s4s1 +Upsample(F′ s5s1 )

[0057] Among them, GAP(·) represents the global average pooling operation, Conv1 represents the 1×1 convolution layer, Conv3 represents the 3×3 convolution layer, Conv5 represents the 5×5 convolution layer, and Conv7 represents the 7×7 convolution layer. represents element-by-element multiplication, Sigmoid(·) represents the activation function, and Upsample(·) represents the upsampling module;

[0058] Step S2224: In the third stage, in addition to upsampling and adding the features of the previous stage, the same-scale features of the first stage are fused using long skip connections to obtain three further fused multi-scale features. and The specific formula is as follows:

[0059] F s1s3 =F s1s1 +F s1s2 +Upsample(F s2s2 ),

[0060] F s2s3 =F s2s1 +F s2s2 +Upsample(F s3s2 ),

[0061] F s3s3 =F s3s1 +F s3s2 +Upsample(F s4s2 )

[0062] Among them, Upsample(·) represents the upsampling module, and the addition of features of two non-adjacent stages at the same scale is a long skip connection;

[0063] Step S2225: In the fourth stage, the upsampling module, the short skip connection and the long skip connection are used to perform multi-scale fusion on the features of the first three stages to obtain two fully fused multi-scale features. and The specific formula is as follows:

[0064] F s1s4 =F s1s1 +F s1s2 +F s1s3 +Upsample(F s2s3 ),

[0065] F s2s4 =F s2s1 +F s2s2 +F s2s3 +Upsample(F s3s3 )

[0066] Among them, Upsample(·) represents the upsampling module;

[0067] Step S2226: In the fifth stage, the second scale feature F of the fourth stage is s2s4 After upsampling to the first scale, it is added and fused with the first scale features of the first four stages to obtain the final output feature F of the progressive multi-scale feature fusion module s1s5 ; The specific formula is as follows:

[0068] F s1s5 =F s1s1 +F s1s2 +F s1s3 +Fs1s4 +Upsample(F s2s4 )

[0069] Among them, Upsample(·) represents the upsampling module.

[0070] Further, step S3 includes the following steps:

[0071] Step S31, designing the loss function of the adaptive brightness enhancement sub-network; in order to make the adaptively enhanced brightness component L en As close as possible to the luminance component L of the reference image gt , the objective loss function of the adaptive brightness enhancement sub-network It is expressed as follows:

[0072]

[0073] in, represents L1 loss, ||·||1 represents the calculation of L1 norm, represents the SSIM loss, λ ssim is the equilibrium parameter, μ en and μ gt For L en and L gt The average brightness, and For L en and L gt The variance, σ en gt For L en and L gt The covariance of C1 and C2 is a constant used for stable calculation. By combining the two, it is possible to reduce the error at the pixel level while retaining the overall structure and details of the brightness component, thereby achieving accurate adaptive brightness enhancement.

[0074] Step S32, designing a loss function of the high-fidelity color correction sub-network; target loss function of the high-fidelity color correction sub-network The design is based on three color spaces: RGB, HSV and YUV, as shown below:

[0075]

[0076]

[0077] Among them, I out represents the output image, I gt represents the reference image, and Respectively represent the U and V channel images after the output image and the reference image are converted to the YUV color space, and Respectively represent the H and S channel images after the output image and the reference image are converted to the HSV color space, represents the perceived loss, Indicates RGB color space loss, Indicates YUV color space loss, represents the HSV color space loss, ||·|| 2 represents the calculation of mean square error, ||·||1 represents the calculation of L1 norm, Indicates calculation of L2 norm, VGG [:31] (·) indicates that the features of the first 31 layers are extracted using the VGG-16 classification model pre-trained on the ImageNet dataset for calculating the perceptual loss, λ per , rgb , uv and λ hs All are equilibrium parameters;

[0078] Step S33, the total target loss function of the low-light image enhancement network based on adaptive brightness enhancement and high-fidelity color correction in step S2 This is the target loss function of the adaptive brightness enhancement subnetwork Objective loss function with high-fidelity color correction sub-network The sum is expressed as follows:

[0079]

[0080] Further, step S4 includes the following steps:

[0081] Step S41, randomly dividing the image data set for training obtained in step S1 to obtain a plurality of batches containing N pairs of images;

[0082] Step S42: input low-light image I in After the low-light image enhancement network based on adaptive brightness enhancement and high-fidelity color correction in step S2, the adaptively enhanced brightness component L is obtained. en And the final output image I out , calculate the loss using the formula in step S3

[0083] Step S43: Calculate the gradient of the parameters in the network using the back propagation method according to the loss function, and update the network parameters using the Adam optimization method;

[0084] Step S44, repeatedly executing steps S41 to S43 in batches to obtain a low-illumination image enhancement network model based on adaptive brightness enhancement and high-fidelity color correction.

[0085] Compared with the prior art, the present invention has the following beneficial effects:

[0086] First, the present invention designs an adaptive brightness enhancement subnetwork, including a multi-stage dual attention residual module, etc. By combining channel attention and spatial attention, it can accurately and effectively extract image brightness priors and realize adaptive enhancement of image brightness. Secondly, the present invention designs a high-fidelity color correction subnetwork, including a progressive multi-scale feature fusion module, an adaptive multi-receptive field feature extraction module, etc. These modules fully extract and fuse the multi-scale features of image color through up and down sampling and short jump connections and long jump connections, and realize high-fidelity color correction effects. In addition, the present invention designs a multi-color space loss function, which can effectively guide and constrain the color correction process, and more meticulously optimize the chromaticity, hue and saturation information to achieve better high-fidelity color correction. Finally, through the combination of two subnetworks, the present invention uses the final image output module to convert the enhanced image features into an output image, and can obtain a high-quality image with appropriate brightness and high color fidelity. Different from other low-illumination image enhancement methods based on deep learning, the present invention not only adopts an adaptive brightness enhancement strategy to avoid the overexposure problem caused by directly learning the mapping from low-illumination images to normal-illumination images, so that the brightness of the enhanced results is appropriate; it also adopts a high-fidelity color correction strategy under multi-color space constraints to avoid the color distortion problem caused by constraining only in the RGB color space. Through the different representation angles of color information in different color spaces, a better color correction effect is achieved, so that the color of the enhanced results is high-fidelity. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 It is a flow chart for realizing the method of the present invention.

[0088] Figure 2 It is a structural diagram of a low-illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction in an embodiment of the present invention.

[0089] Figure 3 is a structural diagram of the adaptive brightness enhancement sub-network in an embodiment of the present invention.

[0090] Figure 4 4 is a structural diagram of a high-fidelity color correction sub-network in an embodiment of the present invention. DETAILED DESCRIPTION

[0091] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0092] The present invention proposes a low illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction, such as Figure 1 As shown, the following steps are included:

[0093] Step S1, constructing a training data set, performing data enhancement on paired low illumination / normal illumination images, and obtaining an image data set for training;

[0094] Step S2, designing a low-illumination image enhancement network based on adaptive brightness enhancement and high-fidelity color correction, wherein the network consists of an adaptive brightness enhancement subnetwork and a high-fidelity color correction subnetwork;

[0095] Step S3, designing a loss function for training the network designed in step S2;

[0096] Step S4, using the training data set obtained in step S1 to train the low-illumination image enhancement network based on adaptive brightness enhancement and high-fidelity color correction in step S2, to obtain a trained low-illumination image enhancement network based on adaptive brightness enhancement and high-fidelity color correction;

[0097] Step S5: input the low-illuminance image to be tested into a trained low-illuminance image enhancement network based on adaptive brightness enhancement and high-fidelity color correction to generate a normal-illuminance image.

[0098] Furthermore, the step S1 includes the following steps:

[0099] Step S11, pairing the low illumination image with a reference image of normal illumination;

[0100] Step S12: For each pair of paired images in the training set, randomly perform one of the following data augmentation operations: horizontal flipping, vertical flipping, 90 degree rotation, 180 degree rotation, and 270 degree rotation.

[0101] Further, refer to Figure 2 , the step S2 comprises the following steps:

[0102] Step S21, designing an adaptive brightness enhancement subnetwork, including a brightness prior extraction module, a multi-stage dual attention residual module and an adaptive brightness enhancement module, for extracting the brightness prior of the low-light image and then adaptively enhancing the brightness of the low-light image to obtain an image after adaptive brightness enhancement;

[0103] Step S22, design a high-fidelity color correction subnetwork, including a color feature extraction module, a progressive multi-scale feature fusion module and a high-fidelity color image output module, which is used to extract the color features of the image after adaptive brightness enhancement and perform color correction to obtain a normal illumination image with appropriate brightness and high-fidelity color.

[0104] Further, refer to Figure 3 , the step S21 comprises the following steps:

[0105] Step S211, design a brightness prior extraction module, the brightness prior extraction module includes a 3×3 convolution layer and a Parametric ReLU activation function; for the input RGB image First, convert it to the YUV color space to obtain the images of the Y, U and V channels, where the image of the Y channel is the original brightness component of the original RGB image. Then, the designed brightness prior extraction module is used to extract the features of the original brightness component to obtain the primary brightness prior Among them, H and W are the height and width of the image respectively, and C is the number of channels of the feature;

[0106] Step S212, design a multi-stage dual attention residual module, the multi-stage dual attention residual module includes four continuously stacked dual attention residual blocks, a 3×3 convolution layer, a Parametric ReLU activation function and a residual connection operation; each dual attention residual block consists of an initial feature extraction block, a channel attention block, a spatial attention block and a feature output block; the dual attention residual block extracts and fuses local and global features at the channel and spatial levels through a dual attention mechanism, thereby realizing the perception and extraction of local and global uneven brightness; after the multi-stage dual attention residual module, a high-level brightness prior is obtained Right now:

[0107] L prior =Parametric ReLU(Conv3(DARB(DARB(DARB(DARB(L prior )))))))+L prior

[0108] Among them, Parametric ReLU(·) represents the activation function, Conv3 table represents the 3×3 convolutional layer, and DARB(·) represents the dual attention residual block;

[0109] Step S213, design an adaptive brightness enhancement module, the adaptive brightness enhancement module includes a 3×3 convolution layer, a Parametric ReLU activation function and an element-by-element division operation; first, the high-level brightness prior L' obtained in step S212 is prior Input 3×3 convolution layer and Parametric ReLU activation function to get the final brightness prior after full extraction Then the brightness component L of the original image original Divide by the final brightness prior Get the adaptively enhanced brightness component That is, the enhanced Y channel image is expressed as follows:

[0110]

[0111] in, represents element-by-element division, and ε represents a very small constant used to prevent the divisor from being zero. Finally, the U and V channel images of the original image are concatenated with the enhanced Y channel image and converted back to the RGB color space to obtain the final result image after adaptive enhancement.

[0112] Furthermore, the step S212 includes the following steps:

[0113] Step S2121, design the initial feature extraction block, for the input feature of the dual attention residual block After passing through a convolutional layer consisting of 3×3 convolution and Parametric ReLU activation function stacked twice, it is copied into two copies, represented as channel attention block input features And the spatial attention block input features

[0114] Step S2122: Design a channel attention block, and input features into the channel attention block The two channel attention weights are obtained through the global average pooling operation and the global maximum pooling operation at the channel level. and They are used to capture the global information and local information of features at the channel level respectively; then the convolution layer composed of 1×1 convolution and Parametric ReLU activation function reduces the number of channels of these two channel attention weights by half to C / 2, and then a 1×1 convolution layer is used to restore the number of channels to C; the two channel attention weights are added element by element, and then a 1×1 convolution and Sigmoid activation function are used to obtain the final channel attention weight Combine the channel attention weight with the initial features Perform element-by-element multiplication to obtain the output features of the channel attention block The specific formula is as follows:

[0115]

[0116] in, represents element-wise multiplication, Sigmoid(·) represents the activation function, Conv1 represents a 1×1 convolutional layer, GAP(·) represents the global average pooling operation, and GMP(·) represents the global maximum pooling operation;

[0117] Step S2123: Design a spatial attention block, input feature Two spatial attention weights are obtained through global average pooling and global maximum pooling at the spatial level and They are used to capture the global and local information of features at the spatial level respectively; then the two spatial attention weights are concatenated and passed through a 1×1 convolution and Sigmoid activation function to obtain the final channel attention weight Combine the channel attention weight with the initial features Perform element-by-element multiplication to obtain the output features of the spatial attention block The specific formula is as follows:

[0118]

[0119] in, represents element-wise multiplication, Sigmoid(·) represents activation function, Conv1 represents 1×1 convolutional layer, GAP(·) represents global average pooling operation, GMP(·) represents global maximum pooling operation, and [·;·] represents feature concatenation operation;

[0120] Step S2124: Design a feature output block to combine the output features of the channel attention block and the spatial attention block and After concatenation, a 3×3 convolution and a Parametric ReLU activation function are performed, and then the input features of the dual attention residual block are As the residual, add it element by element to get the output feature of the dual attention residual block The specific formula is as follows:

[0121]

[0122] Among them, Parametric ReLU(·) represents the activation function, Conv3 represents the 3×3 convolutional layer, and [·;·] represents the feature concatenation operation.

[0123] Further, refer to Figure 4 , the step S22 comprises the following steps:

[0124] Step S221, design a color feature extraction module, the color feature extraction module includes a 3×3 convolution layer and the multi-stage dual attention residual module in step S212; for the adaptively brightened image I obtained in step S21 lightened , and then input it into the color feature extraction module to obtain the initial image color feature

[0125] Step S222: Design a progressive multi-scale feature fusion module, which includes an upsampling module, a downsampling module, a long skip connection, a short skip connection, and an adaptive multi-receptive field feature fusion module; extract and fuse color features at 5 stages and 5 scales to obtain the output image color features.

[0126] Step S223, design a high-fidelity color image output module, the core of which includes a 3×3 convolution layer, a Parametric ReLU activation function, a Sigmoid activation function and a residual connection operation; the output image color features obtained in step S222 After a 3×3 convolutional layer and a Parametric ReLU activation function, the adaptively brightened image I obtained in step S21 is lightened Perform residual connection and pass a Sigmoid activation function to obtain the final color-corrected output image. That is, a high-quality image with appropriate brightness and accurate color; the specific formula is as follows:

[0127]

[0128] Among them, Sigmoid(·) and Parametric ReLU(·) represent activation functions, and Conv3 represents a 3×3 convolutional layer.

[0129] Furthermore, the step S222 includes the following steps:

[0130] Step S2221: In the first stage, a downsampling module is designed, wherein the downsampling module is composed of 3×3 convolution and Parametric ReLU activation function stacked twice; the initial color features are converted into Perform four downsampling operations in sequence to obtain downsampled features of four scales, plus the initial color features that have not been downsampled. A total of 5 features of different scales are obtained, which are represented as and Among them, μ represents the number of channels reduced or increased each time upsampling or downsampling, the first s in the feature subscript represents the scale, and the second s represents the stage;

[0131] Step S2222: In the second stage, an upsampling module is designed, wherein the upsampling module is composed of a 3×3 transposed convolution, a Parametric ReLU activation function, a 3×3 convolution, and a Parametric ReLU activation function stack; for the features F of the second to fourth scales in the first stage, s2s1 、Fs3s1 and F s4s1 , and then upsample it to the previous scale using the designed upsampling module and compare it with the 1st to 3rd scale features F of the first stage respectively. s1s1 、F s2s1 and F s3s1 Perform additive fusion, i.e. short skip connection, to obtain three preliminary fused multi-scale features and The specific formula is as follows:

[0132] F s1s2 =F s1s1 +Upsample(F s2s1 ),

[0133] F s2s2 =F s2s1 +Upsample(F s3s1 ),

[0134] F s3s2 =F s3s1 +Upsample(F s4s1 )

[0135] Among them, Upsample(·) represents the designed upsampling module, and the addition of features of two adjacent stages at the same scale is a short skip connection;

[0136] Step S2223: In the second stage, for the fifth scale feature F at the deep layer s5s1 First, we design an adaptive multi-receptive field feature extraction module to perform adaptive multi-receptive field feature extraction; for the input feature F s5s1 , 3×3 convolution, 5×5 convolution and 7×7 convolution are used to extract features of three different receptive fields. and After adding them together, they are fused using global average pooling to obtain the global channel weights Then, a combination of three 1×1 convolutions and Sigmoid activation functions is used to extract the feature weights for the three receptive fields. and Then, these three feature weights are multiplied by the original three different receptive field features to achieve adaptive representation of the features, and three adaptively selected different receptive field features are obtained. and Finally, the three adaptively selected different receptive field features are added together to obtain the output of the adaptive multi-receptive field feature extraction module Then, it is upsampled to the 4th scale and combined with the 4th scale feature F in the first stage s4s1 Perform addition fusion to obtain global features under multiple receptive fields The specific formula is as follows:

[0137] W channel =GAP(Conv3(F s5s1 )+Conv5(F s5s1 )+Conv7(F s5s1 )),

[0138]

[0139] F s4s2 =F s4s1 +Upsample(F′ s5s1 )

[0140] Among them, GAP(·) represents the global average pooling operation, Conv1 represents the 1×1 convolution layer, Conv3 represents the 3×3 convolution layer, Conv5 represents the 5×5 convolution layer, and Conv7 represents the 7×7 convolution layer. represents element-by-element multiplication, Sigmoid(·) represents the activation function, and Upsample(·) represents the upsampling module;

[0141] Step S2224: In the third stage, in addition to upsampling and adding the features of the previous stage, the same-scale features of the first stage are fused using long skip connections to obtain three further fused multi-scale features. and The specific formula is as follows:

[0142] F s1s3 =F s1s1 +F s1s2 +Upsample(F s2s2 ),

[0143] F s2s3 =F s2s1 +F s2s2 +Upsample(F s3s2 ),

[0144] F s3s3 =F s3s1 +F s3s2 +Upsample(F s4s2 )

[0145] Among them, Upsample(·) represents the upsampling module, and the addition of features of two non-adjacent stages at the same scale is a long skip connection;

[0146] Step S2225: In the fourth stage, the upsampling module, the short skip connection and the long skip connection are used to perform multi-scale fusion on the features of the first three stages to obtain two fully fused multi-scale features. and The specific formula is as follows:

[0147] F s1s4 =F s1s1 +F s1s2 +F s1s3 +Upsample(F s2s3 ),

[0148] F s2s4 =F s2s1 +F s2s2 +F s2s3 +Upsample(F s3s3 )

[0149] Among them, Upsamole(·) represents the upsampling module;

[0150] Step S2226: In the fifth stage, the second scale feature F of the fourth stage is s2s4 After upsampling to the first scale, it is added and fused with the first scale features of the first four stages to obtain the final output feature F of the progressive multi-scale feature fusion module s1s5 ; The specific formula is as follows:

[0151] F s1s5 =F s1s1 +F s1s2 +F s1s3 +F s1s4 +Upsample(F s2s4 )

[0152] Among them, Upsample(·) represents the upsampling module.

[0153] Furthermore, step S3 includes the following steps:

[0154] Step S31, designing the loss function of the adaptive brightness enhancement sub-network; in order to make the adaptively enhanced brightness component L en As close as possible to the luminance component L of the reference image gt , the objective loss function of the adaptive brightness enhancement sub-network It is expressed as follows:

[0155]

[0156] in, represents L1 loss, ||·||1 represents the calculation of L1 norm, represents the SSIM loss, λ ssim is the equilibrium parameter, μ en and μ gt For L en and L gt The average brightness, and For L en and L gt The variance, σ en gt For L en and L gt The covariance of C1 and C2 is a constant used for stable calculation. By combining the two, it is possible to reduce the error at the pixel level while retaining the overall structure and details of the brightness component, thereby achieving accurate adaptive brightness enhancement.

[0157] Step S32, designing a loss function of the high-fidelity color correction sub-network; target loss function of the high-fidelity color correction sub-network The design is based on three color spaces: RGB, HSV and YUV, as shown below:

[0158]

[0159] Among them, I out represents the output image, I gt represents the reference image, and Respectively represent the U and V channel images after the output image and the reference image are converted to the YUV color space, and Respectively represent the H and S channel images after the output image and the reference image are converted to the HSV color space, represents the perceived loss, Indicates RGB color space loss, Indicates YUV color space loss, represents the HSV color space loss, ||·|| 2 represents the calculation of mean square error, ||·||1 represents the calculation of L1 norm, Indicates calculation of L2 norm, VGG [:31] (·) indicates that the features of the first 31 layers are extracted using the VGG-16 classification model pre-trained on the ImageNet dataset for calculating the perceptual loss, λ per , rgb , uv and λ hs All are equilibrium parameters;

[0160] Step S33, the total target loss function of the low-light image enhancement network based on adaptive brightness enhancement and high-fidelity color correction in step S2 This is the target loss function of the adaptive brightness enhancement subnetwork Objective loss function with high-fidelity color correction sub-network The sum is expressed as follows:

[0161]

[0162] Furthermore, the step S4 comprises the following steps:

[0163] Step S41, randomly dividing the image data set for training obtained in step S1 to obtain a plurality of batches containing N pairs of images;

[0164] Step S42: input low-light image I in After the low-light image enhancement network based on adaptive brightness enhancement and high-fidelity color correction in step S2, the adaptively enhanced brightness component L is obtained. en And the final output image I out , calculate the loss using the formula in step S3

[0165] Step S43: Calculate the gradient of the parameters in the network using the back propagation method according to the loss function, and update the network parameters using the Adam optimization method;

[0166] Step S44, repeatedly executing steps S41 to S43 in batches to obtain a low-illumination image enhancement network model based on adaptive brightness enhancement and high-fidelity color correction.

[0167] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.

Claims

1. A low-light image enhancement method based on adaptive brightness enhancement and high-fidelity color correction, characterized in that: The steps include: Step S1, constructing a training data set, performing data enhancement on paired low illumination / normal illumination images, and obtaining an image data set for training; Step S2, designing a low-illumination image enhancement network based on adaptive brightness enhancement and high-fidelity color correction, wherein the network consists of an adaptive brightness enhancement subnetwork and a high-fidelity color correction subnetwork; Step S3, designing a loss function for training the network designed in step S2; Step S4, using the training data set obtained in step S1 to train the low-illumination image enhancement network based on adaptive brightness enhancement and high-fidelity color correction in step S2, to obtain a trained low-illumination image enhancement network based on adaptive brightness enhancement and high-fidelity color correction; Step S5: input the low-illuminance image to be tested into a trained low-illuminance image enhancement network based on adaptive brightness enhancement and high-fidelity color correction to generate a normal-illuminance image.

2. The low illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction according to claim 1, characterized in that: The specific implementation steps of step S1 are as follows: Step S11, pairing the low illumination image with a reference image of normal illumination; Step S12: For each pair of paired images in the training set, randomly perform one of the following data augmentation operations: horizontal flipping, vertical flipping, 90 degree rotation, 180 degree rotation, and 270 degree rotation.

3. The low illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction according to claim 1, characterized in that: The specific implementation steps of step S2 are as follows: Step S21, designing an adaptive brightness enhancement subnetwork, including a brightness prior extraction module, a multi-stage dual attention residual module and an adaptive brightness enhancement module, for extracting the brightness prior of the low-light image and then adaptively enhancing the brightness of the low-light image to obtain an image after adaptive brightness enhancement; Step S22, design a high-fidelity color correction subnetwork, including a color feature extraction module, a progressive multi-scale feature fusion module and a high-fidelity color image output module, which is used to extract the color features of the image after adaptive brightness enhancement and perform color correction to obtain a normal illumination image with appropriate brightness and high-fidelity color.

4. The low illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction according to claim 3, characterized in that: The specific implementation steps of step S21 are as follows: Step S211, design a brightness prior extraction module, the brightness prior extraction module includes a 3×3 convolution layer and a Parametric ReLU activation function; for the input RGB image First, convert it to the YUV color space to obtain the images of the Y, U and V channels, where the image of the Y channel is the original brightness component of the original RGB image. Then, the designed brightness prior extraction module is used to extract the features of the original brightness component to obtain the primary brightness prior Among them, H and W are the height and width of the image respectively, and C is the number of channels of the feature; Step S212, design a multi-stage dual attention residual module, the multi-stage dual attention residual module includes four continuously stacked dual attention residual blocks, a 3×3 convolution layer, a Parametric ReLU activation function and a residual connection operation; each dual attention residual block consists of an initial feature extraction block, a channel attention block, a spatial attention block and a feature output block; the dual attention residual block extracts and fuses local and global features at the channel and spatial levels through a dual attention mechanism, thereby realizing the perception and extraction of local and global uneven brightness; after the multi-stage dual attention residual module, a high-level brightness prior is obtained Right now: The prior =Parametric ReLU(Conv3(DARB(DARB(DARB(DARB(L prior ))))))+L prior Among them, Parametric ReLU(·) represents the activation function, Conv3 table represents the 3×3 convolutional layer, and DARB(·) represents the dual attention residual block; Step S213, design an adaptive brightness enhancement module, the adaptive brightness enhancement module includes a 3×3 convolution layer, a Parametric ReLU activation function and an element-by-element division operation; first, the high-level brightness prior L' obtained in step S212 is prior Input 3×3 convolution layer and Parametric ReLU activation function to get the final brightness prior after full extraction Then the brightness component L of the original image original Divide by the final brightness prior Get the adaptively enhanced brightness component That is, the enhanced Y channel image is expressed as follows: in, represents element-by-element division, and ε represents a very small constant used to prevent the divisor from being zero. Finally, the U and V channel images of the original image are concatenated with the enhanced Y channel image and converted back to the RGB color space to obtain the final result image after adaptive enhancement.

5. The low illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction according to claim 4, characterized in that: The specific implementation steps of step S212 are as follows: Step S2121, design the initial feature extraction block, for the input feature of the dual attention residual block After passing through a convolutional layer consisting of 3×3 convolution and Parametric ReLU activation function stacked twice, it is copied into two copies, represented as channel attention block input features And the spatial attention block input features Step S2122: Design a channel attention block, and input features into the channel attention block The two channel attention weights are obtained through the global average pooling operation and the global maximum pooling operation at the channel level. and They are used to capture the global information and local information of the features at the channel level respectively; then the convolution layer composed of 1×1 convolution and ParametricReLU activation function reduces the number of channels of the two channel attention weights by half to C / 2, and then a 1×1 convolution layer is used to restore the number of channels to C; the two channel attention weights are added element by element, and then a 1×1 convolution and Sigmoid activation function are used to obtain the final channel attention weight Combine the channel attention weight with the initial features Perform element-by-element multiplication to obtain the output features of the channel attention block The specific formula is as follows: in, represents element-wise multiplication, Sigmoid(·) represents the activation function, Conv1 represents a 1×1 convolutional layer, GAP(·) represents the global average pooling operation, and GMP(·) represents the global maximum pooling operation; Step S2123: Design a spatial attention block, input feature Two spatial attention weights are obtained through global average pooling and global maximum pooling at the spatial level and They are used to capture the global and local information of features at the spatial level respectively; then the two spatial attention weights are concatenated and passed through a 1×1 convolution and Sigmoid activation function to obtain the final channel attention weight Combine the channel attention weight with the initial features Perform element-by-element multiplication to obtain the output features of the spatial attention block The specific formula is as follows: in, represents element-wise multiplication, Sigmoid(·) represents activation function, Conv1 represents 1×1 convolutional layer, GAP(·) represents global average pooling operation, GMP(·) represents global maximum pooling operation, and [·;·] represents feature concatenation operation; Step S2124: Design a feature output block to combine the output features of the channel attention block and the spatial attention block and After concatenation, a 3×3 convolution and a Parametric ReLU activation function are performed, and then the input features of the dual attention residual block are As the residual, add it element by element to get the output feature of the dual attention residual block The specific formula is as follows: Among them, Parametric ReLU(·) represents the activation function, Conv3 represents the 3×3 convolutional layer, and [·;·] represents the feature concatenation operation.

6. The low illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction according to claim 3, characterized in that: The specific implementation steps of step S22 are as follows: Step S221, design a color feature extraction module, the color feature extraction module includes a 3×3 convolution layer and the multi-stage dual attention residual module in step S212; for the adaptively brightened image I obtained in step S21 lightened , and then input it into the color feature extraction module to obtain the initial image color feature Step S222: designing a progressive multi-scale feature fusion module, wherein the progressive multi-scale feature fusion module includes an upsampling module, a downsampling module, a long skip connection, a short skip connection, and an adaptive multi-receptive field feature fusion module; The color features of the output image are obtained by extracting and fusing color features at 5 stages and 5 scales. Step S223, design a high-fidelity color image output module, the core of which includes a 3×3 convolution layer, a Parametric ReLU activation function, a Sigmoid activation function and a residual connection operation; the output image color features obtained in step S222 After a 3×3 convolutional layer and a Parametric ReLU activation function, the adaptively brightened image I obtained in step S21 is lightened Perform residual connection and pass a Sigmoid activation function to obtain the final color-corrected output image. That is, a high-quality image with appropriate brightness and accurate color; the specific formula is as follows: Among them, Sigmoid(·) and Parametric ReLU(·) represent activation functions, and Conv3 represents a 3×3 convolutional layer.

7. The low illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction according to claim 6, characterized in that: The specific implementation steps of step S222 are as follows: Step S2221: In the first stage, a downsampling module is designed, wherein the downsampling module is composed of 3×3 convolution and Parametric ReLU activation function stacked twice; the initial color features are converted into Perform four downsampling operations in sequence to obtain downsampled features of four scales, plus the initial color features that have not been downsampled. A total of 5 features of different scales are obtained, which are represented as and Among them, μ represents the number of channels reduced or increased each time upsampling or downsampling, the first s in the feature subscript represents the scale, and the second s represents the stage; Step S2222: In the second stage, an upsampling module is designed, wherein the upsampling module is composed of a 3×3 transposed convolution, a Parametric ReLU activation function, a 3×3 convolution, and a Parametric ReLU activation function stack; for the features F of the second to fourth scales in the first stage, s2s1 、F s3s1 and F s4s1 , and then upsample it to the previous scale using the designed upsampling module and compare it with the 1st to 3rd scale features F of the first stage respectively. s1s1 、F s2s1 and F s3s1 Perform additive fusion, i.e. short skip connection, to obtain three preliminary fused multi-scale features and The specific formula is as follows: F s1s2 =F s1s1 +Upsample(F s2s1 ), F s2s2 =F s2s1 +Upsample(F s3s1 ), F s3s2 =F s3s1 +Upsample(F s4s1 ) Among them, Upsample(·) represents the designed upsampling module, and the addition of features of two adjacent stages at the same scale is a short skip connection; Step S2223: In the second stage, for the fifth scale feature F at the deep layer s5s1 First, we design an adaptive multi-receptive field feature extraction module to perform adaptive multi-receptive field feature extraction; for the input feature F s5s1 , 3×3 convolution, 5×5 convolution and 7×7 convolution are used to extract features of three different receptive fields. and After adding them together, they are fused using global average pooling to obtain the global channel weights Then, a combination of three 1×1 convolutions and Sigmoid activation functions is used to extract the feature weights for the three receptive fields. and Then, these three feature weights are multiplied by the original three different receptive field features to achieve adaptive representation of the features, and three adaptively selected different receptive field features are obtained. and Finally, the three adaptively selected different receptive field features are added together to obtain the output of the adaptive multi-receptive field feature extraction module Then, it is upsampled to the 4th scale and combined with the 4th scale feature F in the first stage s4s1 Perform addition fusion to obtain global features under multiple receptive fields The specific formula is as follows: W channel =GAP(Conv3(F s5s1 )+Conv5(F s5s1 )+Conv7(F s5s1 )), F s4s2 =F s4s1 +Upsample(F′ s5s1 ) Among them, GAP(·) represents the global average pooling operation, Conv1 represents the 1×1 convolution layer, Conv3 represents the 3×3 convolution layer, Conv5 represents the 5×5 convolution layer, and Conv7 represents the 7×7 convolution layer. represents element-by-element multiplication, Sigmoid(·) represents the activation function, and Upsample(·) represents the upsampling module; Step S2224: In the third stage, in addition to upsampling and adding the features of the previous stage, the same-scale features of the first stage are fused using long skip connections to obtain three further fused multi-scale features. and The specific formula is as follows: F s1s3 =F s1s1 +F s1s2 +Upsample(F s2s2 ), F s2s3 =F s2s1 +F s2s2 +Upsample(F s3s2 ), F s3s3 =F s3s1 +F s3s2 +Upsample(F s4s2 ) Among them, Upsample(·) represents the upsampling module, and the addition of features of two non-adjacent stages at the same scale is a long skip connection; Step S2225: In the fourth stage, the upsampling module, the short skip connection and the long skip connection are used to perform multi-scale fusion on the features of the first three stages to obtain two fully fused multi-scale features. and The specific formula is as follows: F s1s4 =F s1s1 +F s1s2 +F s1s3 +Upsample(F s2s3 ), F s2s4 =F s2s1 +F s2s2 +F s2s3 +Upsample(F s3s3 ) Among them, Upsmaple(·) represents the upsampling module; Step S2226: In the fifth stage, the second scale feature F of the fourth stage is s2s4 After upsampling to the first scale, it is added and fused with the first scale features of the first four stages to obtain the final output feature F of the progressive multi-scale feature fusion module s1s5 ; The specific formula is as follows: F s1s5 =F s1s1 +F s1s2 +F s1s3 +F s1s4 +Upsample(F s2s4 ) Among them, Upsample(·) represents the upsampling module.

8. The low illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction according to claim 1, characterized in that: The specific implementation steps of step S3 are as follows: Step S31, designing a loss function of the brightness enhancement subnetwork that is adaptively close to the reference image; in order to make the adaptively enhanced brightness component L en As much as possible, L gt , the objective loss function l of the adaptive brightness enhancement sub-network L It is expressed as follows: Among them, l L1 represents L1 loss, ||·||1 represents the calculation of L1 norm, l ssim represents the SSIM loss, λ ssim is the equilibrium parameter, μ en and μ gt For L en and L gt The average brightness, and For L en and L gt The variance, σ engt For L en and L gt The covariance of C1 and C2 is a constant used for stable calculation. By combining the two, it is possible to reduce the error at the pixel level while retaining the overall structure and details of the brightness component, thereby achieving accurate adaptive brightness enhancement. Step S32, designing a loss function of the high-fidelity color correction sub-network; the target loss function of the high-fidelity color correction sub-network is l C The design is based on three color spaces: RGB, HSV and YUV, as shown below: Among them, I out represents the output image, I gt represents the reference image, and Respectively represent the U and V channel images after the output image and the reference image are converted to the YUV color space, and Respectively represent the H and S channel images after the output image and the reference image are converted to the HSV color space, l per represents the perceptual loss, l rgb Indicates the RGB color space loss, l uv Indicates the loss of YUV color space, l hs represents the HSV color space loss, ||·|| 2 represents the calculation of mean square error, ||·||1 represents the calculation of L1 norm, Indicates calculation of L2 norm, VGG [:31] (·) indicates that the features of the first 31 layers are extracted using the VGG-16 classification model pre-trained on the ImageNet dataset for calculating the perceptual loss, λ per , rgb , uv and λ hs All are equilibrium parameters; The total target loss function l of the low illumination image enhancement network based on adaptive brightness enhancement and high-fidelity color correction in step S33 and step S2 is total That is the target loss function l of the adaptive brightness enhancement sub-network L The objective loss function l of the high-fidelity color correction sub-network C The sum is expressed as follows: l total ＝l L +l C 。 9. The low illumination image enhancement method based on adaptive brightness enhancement and high-fidelity color correction according to claim 1, characterized in that: The specific implementation steps of step S4 are as follows: Step S41, randomly dividing the image data set for training obtained in step S1 to obtain a plurality of batches containing N pairs of images; Step S42: input low-light image I in After the low-light image enhancement network based on adaptive brightness enhancement and high-fidelity color correction in step S2, the adaptively enhanced brightness component L is obtained. en And the final output image I out , use the formula in step S3 to calculate the loss l total ; Step S43: Calculate the gradient of the parameters in the network using the back propagation method according to the loss function, and update the network parameters using the Adam optimization method; Step S44, repeatedly executing steps S41 to S43 in batches to obtain a low-illumination image enhancement network model based on adaptive brightness enhancement and high-fidelity color correction.

Images