A low-light enhancement method, system and storage medium based on deep learning

Abstract

This invention discloses a low-light enhancement method, system, and storage medium based on deep learning. The method includes: acquiring low-light image information; performing data preprocessing on the low-light image information to obtain image brightness information V and color information; analyzing the low-light image information according to a variant of Retinex theory to obtain image composition information; processing the brightness information V according to a decoupled subnetwork to obtain first illumination map information, reflectance information, and noise map information of the image brightness information V; performing gamma correction on the first illumination map information to obtain second illumination map information after illumination adjustment; processing the second illumination map information and the reflectance information according to Retinex theory to obtain enhanced image brightness V'; and combining the enhanced image brightness V' with the image color information to obtain low-light enhanced image information. This invention, based on deep learning, achieves a better low-light enhancement effect.

Description

Technical Field

[0001] This application relates to the fields of data processing and data transmission, and more specifically, to a deep learning-based method, system, and storage medium for low-light enhancement. Background Technology

[0002] Insufficient lighting during image capture significantly reduces image visibility. Lost details and low contrast not only lead to unpleasant subjective experiences but also impair the performance of many computer vision systems designed for normal-light images. Insufficient light can be caused by various factors, such as dimly lit environments, limited photographic equipment performance, and inappropriate equipment configuration. To make hidden details visible and improve the subjective experience and usability of current computer vision systems, low-light image enhancement is needed. This low-light enhancement method can solve the problem of high-quality shooting in dim conditions, enable road condition monitoring to clearly capture usable vehicle information in low-light environments, and achieve pleasing visual effects in photos acquired under any lighting conditions. Over the past few decades, many researchers have dedicated themselves to low-light image enhancement. The earliest methods achieved enhancement by adjusting the image contrast. Later, histogram equalization (HE) methods expanded the dynamic range of images, but their results can present amplified strong noise. Methods based on Retinex theory decompose an image into two components—reflectance and illuminance—process these components separately, and then recombine them to obtain the final enhanced result. Many subsequent works added priors to the decomposed illuminance and reflectance, or extended them to robust models with well-defined noise terms. With the widespread use of CNNs in the image processing field, the first Retinex-Net model based on Retinex theory combined with CNNs was published in 2018. Most subsequent models based on this theory were inspired by Retinex-Net. Recent methods include HWM, which uses wavelet domain feature knowledge to enhance low-light features; LLflow, which currently has the best performance for low-light enhancement using video illumination flow; the Self-calibrated Illumination (SCI) learning framework, used to quickly, flexibly, and robustly brighten images in real-world low-light scenes; EnlightenGAN, which uses generative adversarial loss to achieve unsupervised targeted low-light enhancement using PatchGAN; and Zero, which uses only four loss functions to achieve pixel-level speckle prediction with SoTa-like performance.

[0003] However, current low-light enhancement models based on Retinex theory face challenges in eliminating noise and color distortion previously hidden in the darkness after addressing bright and dark areas. Furthermore, they struggle with training models using only unpaired data or single images to achieve better generalization in low-light image enhancement networks. Due to the limitations of image decoupling, the reflectance learned by the network often contains noise, lacks clear details in dark areas, and suffers from color degradation. These challenges and objectives remain to be addressed in current unsupervised learning-based low-light enhancement methods. Summary of the Invention

[0004] Decomposition models based on Retinex theory tend to generate images with defects such as poor detail, noise, and unsatisfactory colors. The purpose of this invention is to provide a low-light enhancement method, system, and storage medium based on deep learning.

[0005] The first aspect of this invention provides a low-light enhancement method based on deep learning, comprising:

[0006] Acquire low-light image information, perform data preprocessing on the low-light image information to obtain the image's brightness information V and color information;

[0007] The low-light image information is analyzed according to a variant of Retinex theory to obtain image composition information. The brightness information V is processed according to a decoupled subnetwork to obtain the first illumination map information, reflectance information and noise map information of the image brightness information V.

[0008] Gamma correction is applied to the first illumination map information to obtain the second illumination map information after illumination adjustment. The second illumination map information and the reflectance information are then processed according to Retinex theory to obtain the enhanced image brightness V'.

[0009] The enhanced image brightness V' is combined with the color information of the image to obtain low-light enhanced image information;

[0010] The image composition information consists of reflectance, illumination map, and noise map.

[0011] In this scheme, the low-light image information is preprocessed to obtain the image's brightness information V and color information, including:

[0012] The pixel values ​​of the low-light image are mapped to the [0,1] range, and then the RGB to HSV conversion algorithm is used to obtain the brightness information V and color information of the image.

[0013] In this scheme, the variant of the Retinex theory is specifically as follows:

[0014] V = (R·I) + N

[0015] Where V represents the brightness information of the image, R represents the reflectance information, I represents the illumination map information, and N represents the noise map information.

[0016] In this scheme, the brightness information V is processed according to the decoupled subnetwork, including:

[0017]

[0018]

[0019]

[0020] L n =||W n *N|| F

[0021] The total loss function of the network model is expressed as:

[0022] L total =λ R L R +L r +λ I L I +λ n L n

[0023] Among them, L R For reflectivity loss, L IC Illuminance Figure 1 Loss of sexual function, L IS For the smoothing loss of the illuminance map, L n For noise estimation loss, L r To rebuild the losses, G R For reflectivity decoupling subnetwork, G I For lighting decoupling subnetwork, G N This is a noise decoupling subnetwork, where Z1, Z2, and Z3 are all random noise. For the first derivative operator, it includes level and Vertical, ||X|| F ||X||1 and ||X||2 are the Frobenius norm, L1 norm and L2 norm of matrix X, respectively, and W I W n For the weight matrix, λ R , λ I , λ n These represent the balance parameters for different loss functions, where V represents the brightness information of the image and N represents the noise map information.

[0024] In this scheme, gamma correction is performed on the first illumination map information to obtain the second illumination map information after illumination adjustment. The second illumination map information and the reflectance information are then processed according to Retinex theory to obtain the enhanced image brightness V', specifically:

[0025] according to The brightness of the illuminance map is adjusted, and the adjusted illuminance map is obtained. Multiply by the reflectivity R at the pixel level;

[0026] The enhanced image brightness V' is obtained;

[0027] in, This represents the adjusted illuminance diagram, where y represents the correction factor.

[0028] In this scheme, the enhanced image brightness V' is combined with the color information of the image to obtain low-light enhanced image information, including:

[0029] The enhanced image brightness V' is combined with the color information of the image, and then interval mapping and HSV to RGB conversion algorithm are used.

[0030] Obtain low-light enhanced image information;

[0031] The color information of the image includes color H and saturation S.

[0032] A second aspect of the present invention provides a low-light enhancement system based on deep learning, comprising a memory and a processor. The memory includes a low-light enhancement method program based on deep learning, which, when executed by the processor, performs the following steps:

[0033] Acquire low-light image information, perform data preprocessing on the low-light image information to obtain the image's brightness information V and color information;

[0034] The low-light image information is analyzed according to a variant of Retinex theory to obtain image composition information. The brightness information V is processed according to a decoupled subnetwork to obtain the first illumination map information, reflectance information and noise map information of the image brightness information V.

[0035] Gamma correction is applied to the first illumination map information to obtain the second illumination map information after illumination adjustment. The second illumination map information and the reflectance information are then processed according to Retinex theory to obtain the enhanced image brightness V'.

[0036] The enhanced image brightness V' is combined with the color information of the image to obtain low-light enhanced image information;

[0037] The image composition information consists of reflectance, illumination map, and noise map.

[0038] In this scheme, the low-light image information is preprocessed to obtain the image's brightness information V and color information, including:

[0039] The pixel values ​​of the low-light image are mapped to the [0,1] range, and then the RGB to HSV conversion algorithm is used to obtain the brightness information V and color information of the image.

[0040] In this scheme, the variant of the Retinex theory is specifically as follows:

[0041] V = (R·I) + N

[0042] Where V represents the brightness information of the image, R represents the reflectance information, I represents the illumination map information, and N represents the noise map information.

[0043] In this scheme, the brightness information V is processed according to the decoupled subnetwork, including:

[0044]

[0045]

[0046]

[0047] L n =||W n *N|| F

[0048] The total loss function of the network model is expressed as:

[0049] L total =λ R L R +L r +λ I I I +λ n L n

[0050] Among them, L R For reflectivity loss, L IC Illuminance Figure 1 Loss of sexual function, L IS For the smoothing loss of the illuminance map, L n For noise estimation loss, L r To rebuild the losses, G R For reflectivity decoupling subnetwork, G I For lighting decoupling subnetwork, G N This is a noise decoupling subnetwork, where Z1, Z2, and Z3 are all random noise. For the first derivative operator, it includes level and Vertical, ||X|| F ||X||1 and ||X||2 are the Frobenius norm, L1 norm and L2 norm of matrix X, respectively, and W I W n For the weight matrix, λ R , λ I , λ n These represent the balance parameters for different loss functions, where V represents the brightness information of the image and N represents the noise map information.

[0051] In this scheme, gamma correction is performed on the first illumination map information to obtain the second illumination map information after illumination adjustment. The second illumination map information and the reflectance information are then processed according to Retinex theory to obtain the enhanced image brightness V', specifically:

[0052] according to The brightness of the illuminance map is adjusted, and the adjusted illuminance map is obtained. Multiply by the reflectivity R at the pixel level;

[0053] The enhanced image brightness V' is obtained;

[0054] in, This represents the adjusted illuminance diagram, where y represents the correction factor.

[0055] In this scheme, the enhanced image brightness V' is combined with the color information of the image to obtain low-light enhanced image information, including:

[0056] The enhanced image brightness V' is combined with the color information of the image, and then interval mapping and HSV to RGB conversion algorithm are used.

[0057] Obtain low-light enhanced image information;

[0058] The color information of the image includes color H and saturation S.

[0059] A third aspect of the present invention provides a computer-readable storage medium comprising a deep learning-based low-light enhancement method program, wherein when the deep learning-based low-light enhancement method program is executed by a processor, it implements the steps of the deep learning-based low-light enhancement method as described in any of the preceding claims.

[0060] This invention discloses a low-light enhancement method, system, and storage medium based on deep learning. The method includes: acquiring low-light image information; performing data preprocessing on the low-light image information to obtain image brightness information V and color information; analyzing the low-light image information according to a variant of Retinex theory to obtain image composition information; processing the brightness information V according to a decoupled subnetwork to obtain first illumination map information, reflectance information, and noise map information of the image brightness information V; performing gamma correction on the first illumination map information to obtain second illumination map information after illumination adjustment; processing the second illumination map information and the reflectance information according to Retinex theory to obtain enhanced image brightness V'; and combining the enhanced image brightness V' with the image color information to obtain low-light enhanced image information. This invention, based on deep learning, achieves a better low-light enhancement effect. Attached Figure Description

[0061] Figure 1 A flowchart of a low-light enhancement method based on deep learning according to the present invention is shown;

[0062] Figure 2 A schematic diagram of an overall model framework of the present invention is shown;

[0063] Figure 3 A schematic diagram of a decoupled subnetwork GR model framework for learning reflectivity according to the present invention is shown;

[0064] Figure 4 This diagram illustrates a framework for a decoupled subnetwork GI model for learning illumination maps according to the present invention.

[0065] Figure 5 A schematic diagram of a decoupled subnetwork GN model framework for learning noise maps according to the present invention is shown.

[0066] Figure 6 A block diagram of a low-light enhancement system based on deep learning according to the present invention is shown. Detailed Implementation

[0067] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0068] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0069] Figure 1 A flowchart of a low-light enhancement method based on deep learning according to the present invention is shown.

[0070] like Figure 1 As shown, this invention discloses a low-light enhancement method based on deep learning, comprising:

[0071] S102, acquire low-light image information, perform data preprocessing on the low-light image information to obtain the image's brightness information V and color information;

[0072] S104, Analyze the low-light image information according to a variant of Retinex theory to obtain image composition information, and process the brightness information V according to the decoupled sub-network to obtain the first illumination map information, reflectance information and noise map information of the brightness information V of the image;

[0073] S106, Perform gamma correction on the first illumination map information to obtain the second illumination map information after illumination adjustment, and process the second illumination map information and the reflectance information according to Retinex theory to obtain the enhanced image brightness V';

[0074] S108, combine the enhanced image brightness V' with the color information of the image to obtain low-light enhanced image information.

[0075] According to an embodiment of the present invention,

[0076] S1. Perform data preprocessing on multiple low-light images to obtain the brightness and color of the images in the HSV color space;

[0077] S2. Three decoupled sub-networks, GR, GI, and GN, are built using U-Net networks of different depths. According to a variant of Retinex theory, the observed image consists of reflectance, illumination map, and noise map. This invention utilizes the decoupled sub-networks to iteratively learn the illumination map, reflectance, and noise map of image brightness (V) from random noise.

[0078] S3. Adjust the illumination map and obtain enhanced brightness; the specific steps include: performing gamma correction on the illumination map obtained in S2 of the claims to obtain an illumination map with adjusted illumination. Then, according to Retinex theory, multiply the adjusted illumination map by the reflectance obtained in S2 of the claims at the pixel level to obtain the enhanced image (i.e., the enhanced image brightness (V'));

[0079] S4. Reconstruct the original colors and enhanced brightness to generate an enhanced image. Specific steps include: combining the enhanced brightness obtained in claim S3 with the color description obtained in claim S1, and mapping the pixel range back to [0, 255] to obtain the final low-light enhancement result of the model;

[0080] Due to the lack of supervised learning based on fundamental facts, CNNs cannot learn the color mapping from low-light images to their corresponding normal-light images. Furthermore, in the HSV color space, the H, S, and V channels are orthogonal, thus decoupling brightness and color. Based on this characteristic, this invention proposes an unsupervised low-light enhancement method built on a Depth Image Prior (DIP) in the HSV color space. Utilizing a deep learning network with strong repair learning capabilities, combined with the DIP, the method applies the relevant prior of the low-light image brightness (V) obtained in step S1 to the random noise data to decouple and learn its reflectance, illumination map, and noise map. The learned illumination map is then gamma-corrected and multiplied by the reflectance to obtain the enhanced brightness. Finally, this enhanced brightness is combined with the original color obtained in step S1 and mapped back to the RGB space to obtain the final enhancement result. Over hundreds of iterations, the model, learning with the DIP, achieves better quality results than directly building a decoupled network to decompose the image.

[0081] According to an embodiment of the present invention, the low-light image information is preprocessed to obtain the image's brightness information V and color information, including:

[0082] The pixel values ​​of the low-light image are mapped to the [0,1] range, and then the RGB to HSV conversion algorithm is used to obtain the brightness information V and color information of the image.

[0083] It should be noted that, further, the image preprocessing process described in step S1 is as follows: the image pixel values ​​are mapped to the [0,1] interval, and then converted into image color and image brightness in the HSV color space through the RGB to HSV algorithm to obtain the hue (H), saturation (S) and brightness (V) of the low light image.

[0084] According to an embodiment of the present invention, the variant of the Retinex theory is specifically as follows:

[0085] V = (R·I) + N

[0086] Where V represents the brightness information of the image, R represents the reflectance information, I represents the illumination map information, and N represents the noise map information.

[0087] It should be noted that the process of building decoupled subnetworks using U-Net networks of different depths to obtain reflectance, illumination map and noise map in step S2 is as follows: According to a variant of Retinex theory: V = (R·I) + N. The observed image V (the brightness V of the image to be decoupled) consists of reflectance R, illumination map I and noise map N.

[0088] According to an embodiment of the present invention, processing the brightness information V based on the decoupled subnetwork includes:

[0089]

[0090]

[0091]

[0092] L n =||W n *N|| F

[0093] The total loss function of the network model is expressed as:

[0094] L total =λ R L R +L r +λ I L I +λ n L n

[0095] Among them, L R For reflectivity loss, L IC Illuminance Figure 1 Loss of sexual function, L IS For the smoothing loss of the illuminance map, L n For noise estimation loss, L r To rebuild the losses, G R For reflectivity decoupling subnetwork, G I For lighting decoupling subnetwork, G N This is a noise decoupling subnetwork, where Z1, Z2, and Z3 are all random noise. For the first derivative operator, it includes level and Vertical, ||X|| F ||X||1 and ||X||2 are the Frobenius norm, L1 norm and L2 norm of matrix X, respectively, and W I Wn is the weight matrix, λ R , λ I , λ n These represent the balance parameters for different loss functions, where V represents the brightness information of the image and N represents the noise map information.

[0096] It should be noted that three decoupling subnetworks, GR, GI, and GN, are designed. The reflectivity decoupling subnetwork GR is constructed using a typical seven-layer U-Net combined with network residuals, while the other two subnetworks GI and GN are constructed using five-layer U-Nets with skip connections. The decoupling subnetworks use a simple total variation (TV) LR constraint to learn reflectivity and illuminance. Figure 1The learning of illumination maps is constrained by consistency loss LIC and smoothness loss LIS. The learning of noise maps is constrained by illumination-guided noise estimation loss Ln. The reconstruction loss Ir ensures the effectiveness of acquiring multiple components.

[0097] Furthermore, the loss mentioned in step S2 includes reflectance loss LR and illuminance loss. Figure 1 The consistency loss LIC, illuminance map smoothing loss LIS, noise estimation loss Ln, and reconstruction loss Lr are:

[0098]

[0099]

[0100]

[0101] L n =||W n *N|| F

[0102] The total loss function of the network model is expressed as:

[0103] L total =λ R L R +L r +λ I L I +λ n L n

[0104] Among them, L R For reflectivity loss, L IC Illuminance Figure 1 Loss of sexual function, L IS For the smoothing loss of the illuminance map, L n For noise estimation loss, L r To rebuild the losses, G R For reflectivity decoupling subnetwork, G I For lighting decoupling subnetwork, G N This is a noise decoupling subnetwork, where Z1, Z2, and Z3 are all random noise. For the first derivative operator, it includes level and Vertical, ||X|| F ||X||1 and ||X||2 are the Frobenius norm, L1 norm and L2 norm of matrix X, respectively, and W I Wn is the weight matrix, λ R , λ I , λ n These represent the balance parameters for different loss functions, where V represents the brightness information of the image and N represents the noise map information.

[0105] Furthermore, the process of obtaining the weight matrices of the two decoupled sub-networks described in step S2 is as follows: The ideal illumination map should exhibit staged smoothness and contain only the general outline of the image rather than details; therefore, the design of the WI is particularly important, and is expressed as follows:

[0106]

[0107]

[0108]

[0109] Among them, T x,p Inspired by the total variation RTV, the function D(p, q) calculates the spatial Euclidean distance Ω between pixels p and q. p ε represents a 15×15 square window centered at pixel p, and ε guarantees the validity of the data.

[0110] According to an embodiment of the present invention, gamma correction is performed on the first illumination map information to obtain the second illumination map information after illumination adjustment. The second illumination map information and the reflectance information are then processed according to Retinex theory to obtain the enhanced image brightness V', specifically:

[0111] according to The brightness of the illuminance map is adjusted, and the adjusted illuminance map is obtained. Multiply by the reflectivity R at the pixel level;

[0112] The enhanced image brightness V' is obtained;

[0113] in, This represents the adjusted illuminance diagram, where y represents the correction factor.

[0114] It should be noted that the process of adjusting the illumination map and enhancing its brightness described in step S3 is as follows: based on the illuminance map and reflectance learned from the image brightness in step S2, the gamma correction formula is first used. Adjust the brightness according to the brightness chart, among which... The adjusted illuminance map is represented by y, which represents the correction factor. Then, the adjusted illuminance map is multiplied by the reflectance at the pixel level to obtain the enhanced image brightness V'.

[0115] According to an embodiment of the present invention, the enhanced image brightness V' is combined with the color information of the image to obtain low-light enhanced image information, including:

[0116] The enhanced image brightness V' is combined with the color information of the image, and then interval mapping and HSV to RGB conversion algorithm are used.

[0117] Obtain low-light enhanced image information;

[0118] The color information of the image includes color H and saturation S.

[0119] It should be noted that the process of recombining the original color and enhanced brightness to generate the final enhanced result in step S4 is as follows: the enhanced image brightness V' obtained in step S3 is combined with the original color (color and saturation) of the input image obtained in step S1 to form the description of the image in the HSV color space, and then the final enhanced output of the model is obtained through interval mapping and HSV to RGB conversion algorithm.

[0120] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0121] Due to the lack of supervised learning based on fundamental facts, CNNs cannot learn the color mapping from low-light images to their corresponding normal-light images. Furthermore, in the HSV color space, the H, S, and V channels are orthogonal, thus decoupling brightness and color. Based on this characteristic, this invention proposes an unsupervised low-light enhancement method built on a Depth Image Prior (DIP) in the HSV color space. Utilizing a deep learning network with strong repair learning capabilities, combined with the DIP, the method applies the relevant prior of the low-light image brightness (V) obtained in step S1 to the random noise data to decouple and learn its reflectance, illumination map, and noise map. The learned illumination map is then gamma-corrected and multiplied by the reflectance to obtain the enhanced brightness. Finally, this enhanced brightness is combined with the original color obtained in step S1 and mapped back to the RGB space to obtain the final enhancement result. Over hundreds of iterations, the model, learning with the DIP, achieves better quality results than directly building a decoupled network to decompose the image.

[0122] Figure 6 A block diagram of a low-light enhancement system based on deep learning according to the present invention is shown.

[0123] like Figure 6 As shown, a second aspect of the present invention provides a low-light enhancement system 6 based on deep learning, including a memory 61 and a processor 62. The memory includes a low-light enhancement method program based on deep learning. When the processor executes the low-light enhancement method program based on deep learning, it performs the following steps:

[0124] Acquire low-light image information, perform data preprocessing on the low-light image information to obtain the image's brightness information V and color information;

[0125] The low-light image information is analyzed according to a variant of Retinex theory to obtain image composition information. The brightness information V is processed according to a decoupled subnetwork to obtain the first illumination map information, reflectance information and noise map information of the image brightness information V.

[0126] Gamma correction is applied to the first illumination map information to obtain the second illumination map information after illumination adjustment. The second illumination map information and the reflectance information are then processed according to Retinex theory to obtain the enhanced image brightness V'.

[0127] The enhanced image brightness V' is combined with the color information of the image to obtain low-light enhanced image information.

[0128] According to an embodiment of the present invention,

[0129] S1. Perform data preprocessing on multiple low-light images to obtain the brightness and color of the images in the HSV color space;

[0130] S2. Three decoupled sub-networks, GR, GI, and GN, are built using U-Net networks of different depths. According to a variant of Retinex theory, the observed image consists of reflectance, illumination map, and noise map. This invention utilizes the decoupled sub-networks to iteratively learn the illumination map, reflectance, and noise map of image brightness (V) from random noise.

[0131] S3. Adjust the illumination map and obtain enhanced brightness; the specific steps include: performing gamma correction on the illumination map obtained in S2 of the claims to obtain an illumination map with adjusted illumination. Then, according to Retinex theory, multiply the adjusted illumination map by the reflectance obtained in S2 of the claims at the pixel level to obtain the enhanced image (i.e., the enhanced image brightness (V'));

[0132] S4. Reconstruct the original colors and enhanced brightness to generate an enhanced image. Specific steps include: combining the enhanced brightness obtained in claim S3 with the color description obtained in claim S1, and mapping the pixel range back to [0, 255] to obtain the final low-light enhancement result of the model;

[0133] Due to the lack of supervised learning based on fundamental facts, CNNs cannot learn the color mapping from low-light images to their corresponding normal-light images. Furthermore, in the HSV color space, the H, S, and V channels are orthogonal, thus decoupling brightness and color. Based on this characteristic, this invention proposes an unsupervised low-light enhancement method built on a Depth Image Prior (DIP) in the HSV color space. Utilizing a deep learning network with strong repair learning capabilities, combined with the DIP, the method applies the relevant prior of the low-light image brightness (V) obtained in step S1 to the random noise data to decouple and learn its reflectance, illumination map, and noise map. The learned illumination map is then gamma-corrected and multiplied by the reflectance to obtain the enhanced brightness. Finally, this enhanced brightness is combined with the original color obtained in step S1 and mapped back to the RGB space to obtain the final enhancement result. Over hundreds of iterations, the model, learning with the DIP, achieves better quality results than directly building a decoupled network to decompose the image.

[0134] According to an embodiment of the present invention, the low-light image information is preprocessed to obtain the image's brightness information V and color information, including:

[0135] The pixel values ​​of the low-light image are mapped to the [0,1] range, and then the RGB to HSV conversion algorithm is used to obtain the brightness information V and color information of the image.

[0136] It should be noted that, further, the image preprocessing process described in step S1 is as follows: the image pixel values ​​are mapped to the [0,1] interval, and then converted into image color and image brightness in the HSV color space through the RGB to HSV algorithm to obtain the hue (H), saturation (S) and brightness (V) of the low light image.

[0137] According to an embodiment of the present invention, the variant of the Retinex theory is specifically as follows:

[0138] V = (R·I) + N

[0139] Where V represents the brightness information of the image, R represents the reflectance information, I represents the illumination map information, and N represents the noise map information.

[0140] It should be noted that the process of building decoupled subnetworks using U-Net networks of different depths to obtain reflectance, illumination map and noise map in step S2 is as follows: According to a variant of Retinex theory: V = (R·I) + N. The observed image V (the brightness V of the image to be decoupled) consists of reflectance R, illumination map I and noise map N.

[0141] According to an embodiment of the present invention, processing the brightness information V based on the decoupled subnetwork includes:

[0142]

[0143]

[0144]

[0145] L n =||W n *N|| F

[0146] The total loss function of the network model is expressed as:

[0147] L total =λ R L R +L r +λ I L I +λ n L n

[0148] Among them, L R For reflectivity loss, L IC Illuminance Figure 1 Loss of sexual function, L IS For the smoothing loss of the illuminance map, L n For noise estimation loss, L r To rebuild the losses, G R For reflectivity decoupling subnetwork, G I For lighting decoupling subnetwork, G N This is a noise decoupling subnetwork, where Z1, Z2, and Z3 are all random noise. For the first derivative operator, it includes level and Vertical, ||X|| F ||X||1 and ||X||2 are the Frobenius norm, L1 norm and L2 norm of matrix X, respectively, and W I Wn is the weight matrix, λ R , λ I , λ n These represent the balance parameters for different loss functions, where V represents the brightness information of the image and N represents the noise map information.

[0149] It should be noted that three decoupling subnetworks, GR, GI, and GN, are designed. The reflectivity decoupling subnetwork GR is constructed using a typical seven-layer U-Net combined with network residuals, while the other two subnetworks GI and GN are constructed using five-layer U-Nets with skip connections. The decoupling subnetworks use a simple total variation (TV) LR constraint to learn reflectivity and illuminance. Figure 1The learning of illumination maps is constrained by consistency loss LIC and smoothness loss LIS. The learning of noise maps is constrained by illumination-guided noise estimation loss Ln. The reconstruction loss Ir ensures the effectiveness of acquiring multiple components.

[0150] Furthermore, the loss mentioned in step S2 includes reflectance loss LR and illuminance loss. Figure 1 The consistency loss LIC, illuminance map smoothing loss LIS, noise estimation loss Ln, and reconstruction loss Lr are:

[0151]

[0152]

[0153]

[0154] L n =||W n *N|| F

[0155] The total loss function of the network model is expressed as:

[0156] L total =λ R L R +L r +λ I L I +λ n L n

[0157] Among them, L R For reflectivity loss, L IC Illuminance Figure 1 Loss of sexual function, L IS For the smoothing loss of the illuminance map, L n For noise estimation loss, L r To rebuild the losses, G R For reflectivity decoupling subnetwork, G I For lighting decoupling subnetwork, G N This is a noise decoupling subnetwork, where Z1, Z2, and Z3 are all random noise. For the first derivative operator, it includes level and Vertical, ||X|| F ||X||1 and ||X||2 are the Frobenius norm, L1 norm and L2 norm of matrix X, respectively, and W I Wn is the weight matrix, λ R , λ I , λ n These represent the balance parameters for different loss functions, where V represents the brightness information of the image and N represents the noise map information.

[0158] Furthermore, the process of obtaining the weight matrices of the two decoupled sub-networks described in step S2 is as follows: The ideal illumination map should exhibit staged smoothness and contain only the general outline of the image rather than details; therefore, the design of the WI is particularly important, and is expressed as follows:

[0159]

[0160]

[0161]

[0162] Among them, T x，p Inspired by the total variation RTV, the function D(p, q) calculates the spatial Euclidean distance Ω between pixels p and q. p ε represents a 15×15 square window centered at pixel p, and ε guarantees the validity of the data.

[0163] According to an embodiment of the present invention, gamma correction is performed on the first illumination map information to obtain the second illumination map information after illumination adjustment. The second illumination map information and the reflectance information are then processed according to Retinex theory to obtain the enhanced image brightness V', specifically:

[0164] according to The brightness of the illuminance map is adjusted, and the adjusted illuminance map is obtained. Multiply by the reflectivity R at the pixel level;

[0165] The enhanced image brightness V' is obtained;

[0166] in, This represents the adjusted illuminance diagram, where y represents the correction factor.

[0167] It should be noted that the process of adjusting the illumination map and enhancing its brightness described in step S3 is as follows: based on the illuminance map and reflectance learned from the image brightness in step S2, the gamma correction formula is first used. Adjust the brightness according to the brightness chart, among which... The adjusted illuminance map is represented by y, which represents the correction factor. Then, the adjusted illuminance map is multiplied by the reflectance at the pixel level to obtain the enhanced image brightness V'.

[0168] According to an embodiment of the present invention, the enhanced image brightness V' is combined with the color information of the image to obtain low-light enhanced image information, including:

[0169] The enhanced image brightness V' is combined with the color information of the image, and then interval mapping and HSV to RGB conversion algorithm are used.

[0170] Obtain low-light enhanced image information;

[0171] The color information of the image includes color H and saturation S.

[0172] It should be noted that the process of recombining the original color and enhanced brightness to generate the final enhanced result in step S4 is as follows: the enhanced image brightness V' obtained in step S3 is combined with the original color (color and saturation) of the input image obtained in step S1 to form the description of the image in the HSV color space, and then the final enhanced output of the model is obtained through interval mapping and HSV to RGB conversion algorithm.

[0173] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0174] Due to the lack of supervised learning based on fundamental facts, CNNs cannot learn the color mapping from low-light images to their corresponding normal-light images. Furthermore, in the HSV color space, the H, S, and V channels are orthogonal, thus decoupling brightness and color. Based on this characteristic, this invention proposes an unsupervised low-light enhancement method built on a Depth Image Prior (DIP) in the HSV color space. Utilizing a deep learning network with strong repair learning capabilities, combined with the DIP, the method applies the relevant prior of the low-light image brightness (V) obtained in step S1 to the random noise data to decouple and learn its reflectance, illumination map, and noise map. The learned illumination map is then gamma-corrected and multiplied by the reflectance to obtain the enhanced brightness. Finally, this enhanced brightness is combined with the original color obtained in step S1 and mapped back to the RGB space to obtain the final enhancement result. Over hundreds of iterations, the model, learning with the DIP, achieves better quality results than directly building a decoupled network to decompose the image.

[0175] A third aspect of the present invention provides a computer-readable storage medium comprising a deep learning-based low-light enhancement method program, wherein when the deep learning-based low-light enhancement method program is executed by a processor, it implements the steps of the deep learning-based low-light enhancement method as described in any of the preceding claims.

[0176] This invention discloses a low-light enhancement method, system, and storage medium based on deep learning. The method includes: acquiring low-light image information; performing data preprocessing on the low-light image information to obtain image brightness information V and color information; analyzing the low-light image information according to a variant of Retinex theory to obtain image composition information; processing the brightness information V according to a decoupled subnetwork to obtain first illumination map information, reflectance information, and noise map information of the image brightness information V; performing gamma correction on the first illumination map information to obtain second illumination map information after illumination adjustment; processing the second illumination map information and the reflectance information according to Retinex theory to obtain enhanced image brightness V'; and combining the enhanced image brightness V' with the image color information to obtain low-light enhanced image information. This invention, based on deep learning, achieves a better low-light enhancement effect.

[0177] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0178] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0179] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0180] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0181] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A low-light enhancement method based on deep learning, characterized in that, include: Acquire low-light image information, perform data preprocessing on the low-light image information to obtain the image's brightness information V and color information; The low-light image information is analyzed according to a variant of Retinex theory to obtain image composition information. The brightness information V is processed according to a decoupled subnetwork to obtain the first illumination map information, reflectance information and noise map information of the image brightness information V. Gamma correction is applied to the first illumination map information to obtain the second illumination map information after illumination adjustment. The second illumination map information and the reflectance information are then processed according to Retinex theory to obtain the enhanced image brightness V'. The enhanced image brightness V' is combined with the color information of the image to obtain low-light enhanced image information; The image composition information consists of reflectance, illumination map, and noise map; The brightness information V is processed according to the decoupled subnetwork, including: ； ； ； ； The total loss function of the network model is expressed as: ； in, For reflectivity loss, For the consistency loss of illuminance map, For the smoothing loss of the illuminance map, For noise loss estimation, To rebuild the losses, For reflectivity decoupling subnetwork, For lighting decoupling subnetwork, For noise decoupling subnetwork, , , All are random noise, and ∇ is the first derivative operator containing ∇h horizontal and ∇v vertical. , and These represent the Frobenius norm, L1 norm, and L2 norm of matrix X, respectively. , For the weight matrix, , , These represent the balance parameters for different loss functions, where V represents the brightness information of the image and N represents the noise map information. Gamma correction is applied to the first illumination map information to obtain the second illumination map information after illumination adjustment. The second illumination map information and the reflectance information are then processed according to Retinex theory to obtain the enhanced image brightness V', specifically: according to The brightness of the illuminance map is adjusted, and the adjusted illuminance map is obtained. Multiply by the reflectivity R at the pixel level; The enhanced image brightness V' is obtained; in, This represents the adjusted illuminance diagram, where y represents the correction factor.

2. The low-light enhancement method based on deep learning according to claim 1, characterized in that, The low-light image information is preprocessed to obtain the image's brightness information V and color information, including: The pixel values ​​of the low-light image are mapped to the [0,1] range, and then the RGB to HSV conversion algorithm is used to obtain the brightness information V and color information of the image.

3. The low-light enhancement method based on deep learning according to claim 1, characterized in that, The variant of the Retinex theory is as follows: V = (R·I) + N; Where V represents the brightness information of the image, R represents the reflectance information, I represents the illumination map information, and N represents the noise map information.

4. The low-light enhancement method based on deep learning according to claim 1, characterized in that, The enhanced image brightness V' is combined with the image color information to obtain low-light enhanced image information, including: The enhanced image brightness V' is combined with the color information of the image, and then interval mapping and HSV to RGB conversion algorithm are used. Obtain low-light enhanced image information; The color information of the image includes color H and saturation S.

5. A low-light enhancement system based on deep learning, characterized in that, The system includes a memory and a processor. The memory contains a deep learning-based low-light enhancement method program, which, when executed by the processor, performs the following steps: Acquire low-light image information, perform data preprocessing on the low-light image information to obtain the image's brightness information V and color information; The low-light image information is analyzed according to a variant of Retinex theory to obtain image composition information. The brightness information V is processed according to a decoupled subnetwork to obtain the first illumination map information, reflectance information and noise map information of the image brightness information V. Gamma correction is applied to the first illumination map information to obtain the second illumination map information after illumination adjustment. The second illumination map information and the reflectance information are then processed according to Retinex theory to obtain the enhanced image brightness V'. The enhanced image brightness V' is combined with the color information of the image to obtain low-light enhanced image information; The brightness information V is processed according to the decoupled subnetwork, including: ； ； ； ； The total loss function of the network model is expressed as: ； in, For reflectivity loss, For the consistency loss of illuminance map, For the smoothing loss of the illuminance map, For noise loss estimation, To rebuild the losses, For reflectivity decoupling subnetwork, For lighting decoupling subnetwork, For noise decoupling subnetwork, , , All are random noise, and ∇ is the first derivative operator containing ∇h horizontal and ∇v vertical. , and These represent the Frobenius norm, L1 norm, and L2 norm of matrix X, respectively. , For the weight matrix, , , These represent the balance parameters for different loss functions, where V represents the brightness information of the image and N represents the noise map information. Gamma correction is applied to the first illumination map information to obtain the second illumination map information after illumination adjustment. The second illumination map information and the reflectance information are then processed according to Retinex theory to obtain the enhanced image brightness V', specifically: according to The brightness of the illuminance map is adjusted, and the adjusted illuminance map is obtained. Multiply by the reflectivity R at the pixel level; The enhanced image brightness V' is obtained; in, This represents the adjusted illuminance diagram, where y represents the correction factor.

6. A low-light enhancement system based on deep learning according to claim 5, characterized in that, The low-light image information is preprocessed to obtain the image's brightness information V and color information, including: The pixel values ​​of the low-light image are mapped to the [0,1] range, and then the RGB to HSV conversion algorithm is used to obtain the brightness information V and color information of the image.

7. A low-light enhancement system based on deep learning according to claim 5, characterized in that, Gamma correction is applied to the first illumination map information to obtain the second illumination map information after illumination adjustment. The second illumination map information and the reflectance information are then processed according to Retinex theory to obtain the enhanced image brightness V', specifically: according to The brightness of the illuminance map is adjusted, and the adjusted illuminance map is obtained. Multiply by the reflectivity R at the pixel level; The enhanced image brightness V' is obtained; in, This represents the adjusted illuminance diagram, where y represents the correction factor.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a deep learning-based low-light enhancement method program, which, when executed by a processor, implements the steps of the deep learning-based low-light enhancement method as described in any one of claims 1 to 4.

Images