A low-light image enhancement method, system, device and storage medium

Abstract

A low-light image enhancement method, system, device and storage medium, which relate to the field of computer vision technology, solve the problems of uneven illumination distribution and difficulty in denoising in the image restoration process in the existing illumination mapping technology. Based on the low-light image, a low-frequency image of the low-light image and a high-frequency image of the low-light image are obtained respectively; based on the low-frequency image of the low-light image, an enhanced low-frequency image of the low-light image is obtained; based on the high-frequency image of the low-light image, an enhanced high-frequency image of the low-light image is obtained; based on the enhanced low-frequency image of the low-light image and the high-frequency image of the low-light image, an enhanced image is obtained; based on the enhanced image, a loss function is designed to train a frequency decomposition network model, a low-frequency path illumination adjustment network model and a high-frequency path illumination adjustment network model; the above operations are performed on the low-light image to obtain the final enhanced image.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a low-light image enhancement method, system, device and storage medium. Background Art

[0002] The quality of images taken in low-light environments usually degrades significantly, showing dim images, blurred details, and significantly increased noise. This quality degradation seriously impairs the visual experience and the accuracy of information transmission, which in turn affects downstream computer vision tasks such as object detection, semantic segmentation, and autonomous driving.

[0003] Although existing image enhancement methods such as increasing the ISO (sensitivity) value or using long exposure can improve the brightness of the image to a certain extent, these methods are often accompanied by side effects such as increased noise and color distortion, which cannot effectively solve the problem of image degradation in low-light conditions.

[0004] In addition, although existing low-light image enhancement methods aim to improve image quality, their actual effects are often limited by challenges such as insufficient noise suppression and color distortion. In recent years, deep learning technology has made significant progress in the field of low-light image enhancement. Many advanced algorithms use complex network structures and training mechanisms to improve image quality and mitigate the side effects of traditional technologies. This method can be divided into two main categories. The first method learns the distribution of normal light images through convolution, relying on a series of mappings and generative adversarial networks for training, but this method ignores the differences in illumination distribution between data sets and human perception of color and illumination, so the theoretical basis is still imperfect. The second method combines the Retinex (a combination of the retina and the cerebral cortex) theory, uses convolution to decompose the image, adjusts the reflection and illumination, and aims to fundamentally improve the quality of low-light images. However, this method has difficulty in effectively handling the noise introduced in the decomposition step.

[0005] In summary, in order to achieve a naturally perceived illumination mapping effect and effectively solve the noise problem introduced by existing decomposition methods during the decomposition process, a stable and highly adaptable dark-light image enhancement method is urgently needed to overcome image degradation problems such as color distortion and noise. Summary of the invention

[0006] The present invention solves the problems of uneven illumination distribution and difficulty in denoising in the image restoration process in the existing illumination mapping technology.

[0007] A low-light image enhancement method according to the present invention comprises the following steps:

[0008] Step S1, obtaining a training set and a test set respectively, where both the training set and the test set are low-light images and their corresponding normal-light images;

[0009] Step S2, for the low-light images in the training set, the frequency-based decomposition network model of the low-light images respectively obtains a low-frequency image of the low-light images and a high-frequency image of the low-light images;

[0010] Step S3, enhancing the low-frequency image of the low-light image based on the low-frequency pathway illumination adjustment network model to obtain an enhanced low-frequency image of the low-light image;

[0011] Step S4, the high-frequency image of the low-light image is enhanced based on the high-frequency pathway illumination adjustment network model to obtain an enhanced high-frequency image of the low-light image;

[0012] Step S5, adding the low-frequency image of the enhanced low-light image and the high-frequency image of the low-light image bit by bit to obtain an enhanced image;

[0013] Step S6, designing a loss function based on the enhanced image and the normal illumination image corresponding to the low illumination image in the training set, and using the loss function to train the frequency decomposition network model, the low-frequency pathway illumination adjustment network model, and the high-frequency pathway illumination adjustment network model respectively;

[0014] Step S7: The low-light image in the test set and the corresponding normal-light image perform the operations of steps S2 to S5 to obtain a final enhanced image.

[0015] Furthermore, in one embodiment of the present invention, the frequency decomposition network model includes five Convs connected in series, a PReLU activation function is connected in series after each of the first four Convs, and a Sigmoid activation function is connected in series after the fifth Conv.

[0016] Furthermore, in one embodiment of the present invention, the low-frequency path illumination adjustment network model is:

[0017] First, the low-frequency image of the low-light image passes through DCE-Net to obtain the parameters required by the NR equation, and then it iteratively passes through the NR equation for illumination adjustment. All the obtained illumination information is concat. At the same time, the low-frequency image of the low-light image obtains scene color information through CEM, which is input into FusionNet together with all the concat illumination information to obtain the enhanced low-frequency image of the low-light image.

[0018] Further, in one embodiment of the present invention, the NR equation is:

[0019] ;

[0020] Among them, for the lighting adjustment factor , defined as:

[0021] ;

[0022] In the formula, is the current iteration number, To control the slope adaptation factor of the NR equation, , , , and are all learnable parameters. For images, It is the adaptive factor for controlling the average value of the NR equation curve.

[0023] Furthermore, in one embodiment of the present invention, the high-frequency path illumination adjustment network model is:

[0024] First, the high-frequency image of the low-light image is passed through PD to obtain the high-frequency image of the low-light image after denoising, and then passed through Conv to obtain the high-frequency image of the preprocessed low-light image. The high-frequency image of the preprocessed low-light image is successively passed through AEM and ESA to obtain the high-frequency image of the low-light image after spatial attention enhancement. After the high-frequency image of the low-light image after spatial attention enhancement passes through Conv, it is bitwise added with the high-frequency image of the low-light image after denoising to obtain the high-frequency image of the enhanced low-light image.

[0025] Further, in one embodiment of the present invention, the high-frequency image of the preprocessed low-light image is sequentially processed by AEM and ESA to obtain the high-frequency image of the low-light image after spatial attention enhancement, specifically:

[0026] The high-frequency image of the preprocessed low-light image generates four different enhanced features through AEM, among which the first three enhanced features are concat with the high-frequency image of the preprocessed low-light image to obtain the high-frequency image of the low-light image after pre-attention enhancement, and the high-frequency image of the low-light image after pre-attention enhancement and the last enhanced feature are all entered into ESA to obtain the high-frequency image of the low-light image after spatial attention enhancement.

[0027] Furthermore, in one embodiment of the present invention, in step S6, the loss function is:

[0028] ;

[0029] In the formula is the loss function, is the current iteration number, is the normal light image corresponding to the low light image, For the enhanced image, Feature maps extracted from different layers of the VGG16 network, The structural similarity loss.

[0030] A low-light image enhancement system according to the present invention comprises the following modules:

[0031] Module S1, obtaining a training set and a test set, both of which are low-light images and their corresponding normal-light images;

[0032] Module S2, for the low-light images in the training set, the frequency-based decomposition network of the low-light images respectively models the low-frequency images and the high-frequency images of the low-light images;

[0033] Module S3, enhancing the low-frequency image of the low-light image based on the low-frequency pathway illumination adjustment network model to obtain an enhanced low-frequency image of the low-light image;

[0034] Module S4, enhancing the high-frequency image of the low-light image based on the high-frequency pathway illumination adjustment network model to obtain an enhanced high-frequency image of the low-light image;

[0035] Module S5, adding the low-frequency image of the enhanced low-light image and the high-frequency image of the low-light image bit by bit to obtain an enhanced image;

[0036] Module S6, designs a loss function based on the enhanced image and the normal illumination image corresponding to the low illumination image in the training set, and uses the loss function to train the frequency decomposition network model, the low-frequency pathway illumination adjustment network model, and the high-frequency pathway illumination adjustment network model respectively;

[0037] Module S7: The low-light image in the test set and the corresponding normal-light image perform the operations of modules S2 to S5 to obtain a final enhanced image.

[0038] An electronic device described in the present invention comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0039] Memory, used to store computer programs;

[0040] The processor is used to implement any of the method steps described in the above method when executing the program stored in the memory.

[0041] The computer-readable storage medium described in the present invention stores a computer program, and when the computer program is executed by a processor, the method steps of any of the above methods are implemented.

[0042] The present invention solves the problems of uneven illumination distribution and difficulty in denoising in the image restoration process in the existing illumination mapping technology. Specific beneficial effects include:

[0043] 1. A low-light image enhancement method described in the present invention. In the prior art, there are problems of uneven illumination distribution and difficulty in denoising in the image restoration process in the illumination mapping technology. In order to solve the above technical problems, the present invention designs a low-frequency pathway illumination adjustment network model and a high-frequency pathway illumination adjustment network model. The low-frequency pathway illumination adjustment network model can dynamically adjust the brightness and color of the image, effectively avoiding uneven illumination distribution and color distortion. The high-frequency pathway illumination adjustment network model is a combination of PD, AEM and ESA, which can accurately enhance image details and effectively suppress noise, thereby optimizing detail retention and computational efficiency. The images that pass through the low-frequency pathway illumination adjustment network model and the high-frequency pathway illumination adjustment network model are bitwise added to obtain an enhanced image, which effectively solves the problems of uneven illumination distribution and difficulty in denoising in the image restoration process in the existing illumination mapping technology;

[0044] The low-light image enhancement method described in the present invention performs experiments on an image dataset under low-light conditions and compares and analyzes it with the prior art. The experimental data confirms that the present invention can effectively reduce the noise level and maintain the clarity of image details while improving image brightness. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0046] Figure 1 It is a framework diagram of the low-light image enhancement method described in Implementation Method 1. DETAILED DESCRIPTION

[0047] The following will clearly and completely describe various embodiments of the present invention in conjunction with the accompanying drawings. The embodiments described with reference to the accompanying drawings are exemplary and intended to be used to explain the present invention, but should not be understood as limiting the present invention.

[0048] Embodiment 1: A low-light image enhancement method described in this embodiment includes the following steps:

[0049] Step S1, obtaining a training set and a test set respectively, where both the training set and the test set are low-light images and their corresponding normal-light images;

[0050] Step S2, for the low-light images in the training set, the frequency-based decomposition network model of the low-light images respectively obtains a low-frequency image of the low-light images and a high-frequency image of the low-light images;

[0051] Step S3, enhancing the low-frequency image of the low-light image based on the low-frequency pathway illumination adjustment network model to obtain an enhanced low-frequency image of the low-light image;

[0052] Step S4, the high-frequency image of the low-light image is enhanced based on the high-frequency pathway illumination adjustment network model to obtain an enhanced high-frequency image of the low-light image;

[0053] Step S5, adding the low-frequency image of the enhanced low-light image and the high-frequency image of the low-light image bit by bit to obtain an enhanced image;

[0054] Step S6, designing a loss function based on the enhanced image and the normal illumination image corresponding to the low illumination image in the training set, and using the loss function to train the frequency decomposition network model, the low-frequency pathway illumination adjustment network model, and the high-frequency pathway illumination adjustment network model respectively;

[0055] Step S7: The low-light image in the test set and the corresponding normal-light image perform the operations of steps S2 to S5 to obtain a final enhanced image.

[0056] In this implementation, the frequency decomposition network model includes five Convs connected in series, a PReLU activation function is connected in series after each of the first four Convs, and a Sigmoid activation function is connected in series after the fifth Conv.

[0057] In this implementation, the low-frequency path illumination adjustment network model is:

[0058] First, the low-frequency image of the low-light image passes through DCE-Net to obtain the parameters required by the NR equation, and then it iteratively passes through the NR equation for illumination adjustment. All the obtained illumination information is concat. At the same time, the low-frequency image of the low-light image obtains scene color information through CEM, which is input into FusionNet together with all the concat illumination information to obtain the enhanced low-frequency image of the low-light image.

[0059] In this implementation, the NR equation is:

[0060] ;

[0061] Among them, for the lighting adjustment factor , defined as:

[0062] ;

[0063] In the formula, is the current iteration number, To control the slope adaptation factor of the NR equation, , , , and are all learnable parameters. For images, It is the adaptive factor for controlling the average value of the NR equation curve.

[0064] In this implementation, the high-frequency path illumination adjustment network model is:

[0065] First, the high-frequency image of the low-light image is passed through PD to obtain the high-frequency image of the low-light image after denoising, and then passed through Conv to obtain the high-frequency image of the preprocessed low-light image. The high-frequency image of the preprocessed low-light image is successively passed through AEM and ESA to obtain the high-frequency image of the low-light image after spatial attention enhancement. After the high-frequency image of the low-light image after spatial attention enhancement passes through Conv, it is bitwise added with the high-frequency image of the low-light image after denoising to obtain the high-frequency image of the enhanced low-light image.

[0066] In this embodiment, the high-frequency image of the preprocessed low-light image is sequentially processed by AEM and ESA to obtain the high-frequency image of the low-light image after spatial attention enhancement, specifically:

[0067] The high-frequency image of the preprocessed low-light image generates four different enhanced features through AEM, among which the first three enhanced features are concat with the high-frequency image of the preprocessed low-light image to obtain the high-frequency image of the low-light image after pre-attention enhancement, and the high-frequency image of the low-light image after pre-attention enhancement and the last enhanced feature are all entered into ESA to obtain the high-frequency image of the low-light image after spatial attention enhancement.

[0068] In this implementation, in step S6, the loss function is:

[0069] ;

[0070] In the formula, is the loss function, is the current iteration number, is the normal light image corresponding to the low light image, For the enhanced image, Feature maps extracted from different layers of the VGG16 network, The structural similarity loss.

[0071] In the prior art, the existing illumination mapping technology learns the distribution of normal light images through convolution or combines with Retinex theory to decompose the image by convolution to restore the image. However, there are problems of uneven illumination distribution and difficulty in denoising during the image restoration process.

[0072] To solve the above technical problems, this embodiment designs a low-light image enhancement method based on a biological vision model, comprising the following steps:

[0073] Step S1, obtain a data set, the data set includes a training set and a test set, the training set and the test set are low-light images and its corresponding normal illumination image , the test set is low-light images and its corresponding normal illumination image .

[0074] Step S2: for low-light images in the training set , low light images The low-frequency images of low-light images are obtained through the frequency decomposition network model. and high-frequency images of low-light images , the normal light image corresponding to the low light image The low-frequency images of the normal illumination images are obtained through the frequency decomposition network model. and high-frequency images of normal illumination images ;

[0075] The frequency decomposition network model includes five concatenated Convs (convolutional layers), each of the first four Convs is connected in series with a PReLU activation function (parameterized rectified linear unit activation function), and the fifth Conv is connected in series with a Sigmoid activation function (S-shaped activation function);

[0076] Each Conv layer has a size of 3×3, the first four Convs have 64 convolution channels, and the fifth Conv has 6 convolution channels;

[0077] Low-light images and its corresponding normal illumination image are input into the frequency decomposition network model to obtain low-frequency images of low-light images. , high-frequency images of low-light images , low-frequency image of normal illumination image and high-frequency images of normal illumination images , low-frequency images based on low-light images , high-frequency images of low-light images , low-frequency image of normal illumination image and high-frequency images of normal illumination images Calculate the decomposition loss of the frequency decomposition network model , defined as:

[0078] ;

[0079] Calculate the decomposition loss of the frequency decomposition network model This is to ensure the effectiveness of the frequency decomposition network model.

[0080] Step S3, construct a low-frequency path illumination adjustment network model, and transform the low-frequency image of the low-light image Input into the low-frequency pathway illumination adjustment network model for illumination enhancement to obtain the enhanced low-frequency image of the low-light image ;

[0081] like Figure 1 As shown, first, the low-frequency image of the low-light image is The required NR equation (Navier-Stokes equation) is obtained through DCE-Net (Deep Curve Estimation Network) , and Parameters, then iterate The NR equation is used to adjust the illumination for the second time, and all the illumination information is concat (stitching). At the same time, in order to avoid the artifact problem caused by illumination adjustment, the low-frequency image of the low-light image The scene color information is also obtained through CEM (convolution enhancement module). The restored scene color information and all the lighting information enter the FusionNet (fusion network) together to finally obtain the low-frequency image of the enhanced low-light image. ;

[0082] The NR equation described in this embodiment is an improved NR equation based on the inspiration of the biological vision model. The NR equation gradually adjusts the low-frequency illumination information in an iterative manner, and its expression is:

[0083] ;

[0084] In the formula, is the current iteration number, To control the slope adaptation factor of the NR equation, is the illumination adjustment factor, which controls the average value of the NR equation and provides flexibility during training to adapt to different lighting conditions and image characteristics. For images, It is the adaptive factor for controlling the average value of the NR equation curve.

[0085] Specifically, for the lighting adjustment factor , defined as:

[0086] ;

[0087] Each time with image and five learnable parameters , , , and Using a logarithmic function as input enables the NR equation to simulate the nonlinear sensitivity of biological vision, generating enhanced images that maintain a natural appearance and improve contrast and details.

[0088] In order to achieve a better effect of light adjustment, this implementation method iterates The illumination is adjusted by the NR equation for the first time. Specifically, at the beginning, the low-frequency image of the low-light image As input, the image after the first illumination adjustment is obtained through the NR equation, and the image is used as the input of the second NR equation. The image is iterated sequentially to obtain A group of images after illumination adjustment, After the group images are concat, they enter a FusionNet for feature fusion;

[0089] The NR equations simulate the nonlinear response of retinal neurons to light intensity, allowing the model to mimic the way human perception adapts to different brightness levels.

[0090] The CEM described in this embodiment consists of a layer of 3×3 Conv, a 7×7 DW-Conv (depthwise separable convolution), and finally three consecutive 1×1 Convs;

[0091] Low-frequency image based on enhanced low-light image and low-frequency images of normal illumination images Calculate the low frequency loss , defined as:

[0092] .

[0093] Step S4, constructing a high-frequency path illumination adjustment network model to convert the high-frequency image of the low-light image Input to the high-frequency path illumination adjustment network model for detail enhancement and denoising to obtain the enhanced high-frequency image of the low-light image ;

[0094] The high-frequency pathway illumination regulation network model described in this embodiment includes a PD (preliminary denoiser), an AEM (adaptive enhancement module) and an ESA (efficient spatial attention module);

[0095] The purpose of PD is to reduce noise amplification before detail enhancement. It starts with a 5×5 Conv+ReLU (rectified linear unit) block, followed by three parallel 3×3 Conv+ReLU blocks, and finally with the high-frequency image of the low-light image. Perform bitwise addition to obtain the result after preliminary denoising;

[0096] The purpose of AEM is to refine the edge and texture details of the image. The input is first passed to two 3×3 Conv blocks, and then a logarithmic transformation is performed to manage the dynamic range of the image. Subsequently, two 1×1 Conv blocks are passed through the convolution blocks to obtain the result of detail enhancement;

[0097] ESA achieves effective channel feature recalibration without significantly increasing the number of parameters. It consists of three Sep-Conv (separable convolution) and PReLU, and is finally output through the Sigmoid function.

[0098] like Figure 1 As shown, the high-frequency image of the low-light image First enter PD to obtain the high-frequency image of the low-light image after noise reduction , and then the high-frequency image of the preprocessed low-light image is obtained through 1×1 Conv , high-frequency image of the preprocessed low-light image The details and texture features are obtained through AEM, and then enter ESA to obtain the high-frequency image of the low-light image after spatial attention enhancement. , and finally through a residual connection, the high-frequency image of the enhanced low-light image is obtained The high-frequency pathway illumination conditioning network model is designed to preserve and enhance image details while effectively suppressing noise.

[0099] Specifically, high-frequency images of low-light images First, it passes through PD, which aims to preserve and enhance image details while effectively suppressing noise. PD can also capture the deep features of the image while retaining important texture information, providing a solid foundation for subsequent detail enhancement.

[0100] High-frequency image of preprocessed low-light image After AEM, four different enhanced features are generated, of which the first three features are related to the high-frequency image of the input preprocessed low-light image. Concat is performed to form a new texture feature to obtain the high-frequency image of the low-light image after pre-attention enhancement. , the last enhanced feature is used as the detail feature, that is, the high-frequency image of the low-light image after detail enhancement It's all in ESA. AEM adaptively enhances textures and fine details, ensuring that the inherent character of the image is maintained even after processing.

[0101] ESA achieves effective channel feature recalibration without significantly increasing the number of parameters, thereby improving the efficiency of the model while maintaining high performance. The introduced Sep-Conv reduces the computational complexity of the high-frequency pathway illumination regulation network model while retaining its ability to effectively extract features.

[0102] The high-frequency image of the enhanced low-light image is obtained by passing it through the high-frequency path illumination adjustment network model High frequency image with normal illumination image pass The loss function (absolute error loss function) calculates the high-frequency loss , defined as:

[0103] .

[0104] Step S5: The low-frequency image of the enhanced low-light image and high-frequency images of low-light images Perform bitwise addition operation to obtain the enhanced image ;

[0105] Get the enhanced image , defined as:

[0106] .

[0107] Step S6, based on the enhanced image and the normal-light images corresponding to the low-light images in the training set The loss function is designed based on the features extracted from the image, and the parameters of the frequency decomposition network model, the low-frequency pathway illumination adjustment network model, and the high-frequency pathway illumination adjustment network model are updated respectively by using the loss function and the back propagation algorithm;

[0108] To ensure pixel-level accuracy, use The loss function (mean squared error loss function) and structural similarity loss are used to maintain the structural consistency between the enhanced image and the reference image. In addition, a perceptual loss based on the VGG16 (Visual Geometry Group of the University of Oxford) feature representation is introduced to ensure high-level semantic details and visual fidelity, which is defined as:

[0109] ;

[0110] In the formula, is the loss function, is the current iteration number, is the normal light image corresponding to the low light image, For the enhanced image, Feature maps extracted from different layers of the VGG16 network, The structural similarity loss.

[0111] Therefore, the above loss function can be summarized as:

[0112] ;

[0113] in, , , and are weights, set to 100, 10, 10, and 5 respectively.

[0114] Step S7, implement a judgment mechanism to determine whether a predetermined number of training iterations have been completed. When all training iterations are completed, the frequency decomposition network model, the low-frequency path illumination adjustment network model and the high-frequency path illumination adjustment network model optimized by training will be obtained.

[0115] Step S8: The low-light images in the test set and its corresponding normal illumination image The decomposition network model, the low-frequency path illumination adjustment network model and the high-frequency path illumination adjustment network model trained in step S7 are sequentially inputted. The decomposition network model, the low-frequency path illumination adjustment network model and the high-frequency path illumination adjustment network model will adjust the input low-light image. and its corresponding normal illumination image Processing is performed to output the final enhanced image.

[0116] In summary, this embodiment dynamically adjusts the brightness and color of the image through the low-frequency path illumination adjustment network model, effectively avoiding uneven illumination distribution and color distortion. And through the high-frequency path illumination adjustment network model, the model accurately enhances image details and effectively suppresses noise through the combination of PD, AEM and ESA, thereby optimizing detail retention and computational efficiency. This solves the problem of uneven illumination distribution and difficulty in denoising in the image restoration process in the existing illumination mapping technology.

[0117] Embodiment 2: A low-light image enhancement system described in this embodiment includes the following modules:

[0118] Module S1, obtaining a training set and a test set, both of which are low-light images and their corresponding normal-light images;

[0119] Module S2, for the low-light images in the training set, the frequency-based decomposition network of the low-light images respectively models the low-frequency images and the high-frequency images of the low-light images;

[0120] Module S3, enhancing the low-frequency image of the low-light image based on the low-frequency pathway illumination adjustment network model to obtain an enhanced low-frequency image of the low-light image;

[0121] Module S4, enhancing the high-frequency image of the low-light image based on the high-frequency pathway illumination adjustment network model to obtain an enhanced high-frequency image of the low-light image;

[0122] Module S5, adding the low-frequency image of the enhanced low-light image and the high-frequency image of the low-light image bit by bit to obtain an enhanced image;

[0123] Module S6, designs a loss function based on the enhanced image and the normal illumination image corresponding to the low illumination image in the training set, and uses the loss function to train the frequency decomposition network model, the low-frequency pathway illumination adjustment network model, and the high-frequency pathway illumination adjustment network model respectively;

[0124] Module S7: The low-light image in the test set and the corresponding normal-light image perform the operations of modules S2 to S5 to obtain a final enhanced image.

[0125] Embodiment 3: An electronic device described in this embodiment includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0126] Memory, used to store computer programs;

[0127] The processor is used to implement the method steps described in the first embodiment when executing the program stored in the memory.

[0128] Embodiment 4: A computer-readable storage medium described in this embodiment stores a computer program, and when the computer program is executed by a processor, the method steps described in embodiment 1 are implemented.

[0129] The low-light image enhancement method, system, device and storage medium proposed in the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A low-light image enhancement method, characterized in that: The following steps are involved: Step S1, obtaining a training set and a test set respectively, where both the training set and the test set are low-light images and their corresponding normal-light images; Step S2, for the low-light images in the training set, the frequency-based decomposition network model of the low-light images respectively obtains a low-frequency image of the low-light images and a high-frequency image of the low-light images; Step S3, enhancing the low-frequency image of the low-light image based on the low-frequency pathway illumination adjustment network model to obtain an enhanced low-frequency image of the low-light image; Step S4, the high-frequency image of the low-light image is enhanced based on the high-frequency pathway illumination adjustment network model to obtain an enhanced high-frequency image of the low-light image; Step S5, adding the low-frequency image of the enhanced low-light image and the high-frequency image of the low-light image bit by bit to obtain an enhanced image; Step S6, designing a loss function based on the enhanced image and the normal illumination image corresponding to the low illumination image in the training set, and using the loss function to train the frequency decomposition network model, the low-frequency pathway illumination adjustment network model, and the high-frequency pathway illumination adjustment network model respectively; Step S7, performing the operations of steps S2 to S5 on the low-light image in the test set and the normal-light image corresponding thereto to obtain a final enhanced image; The low-frequency path illumination adjustment network model is: First, the low-frequency image of the low-light image is passed through the deep curve estimation network to obtain the parameters required by the Navier-Stokes equation, and then iteratively passes through the Navier-Stokes equation to adjust the illumination. All the obtained illumination information is concat. At the same time, the low-frequency image of the low-light image is passed through the convolution enhancement module to obtain the scene color information, which is input into FusionNet together with all the concat illumination information to obtain the enhanced low-frequency image of the low-light image. The high-frequency path illumination adjustment network model is: First, the high-frequency image of the low-light image passes through a preliminary denoiser to obtain a high-frequency image of the low-light image after denoising, and then passes through Conv to obtain the high-frequency image of the preprocessed low-light image. The high-frequency image of the preprocessed low-light image passes through an adaptive enhancement module and an efficient spatial attention module in turn to obtain a high-frequency image of the low-light image after spatial attention enhancement. After the high-frequency image of the low-light image after spatial attention enhancement passes through Conv, it is bitwise added with the high-frequency image of the low-light image after denoising to obtain the high-frequency image of the enhanced low-light image.

2. The low-light image enhancement method according to claim 1, characterized in that: The frequency decomposition network model includes five Convs connected in series, each of the first four Convs is connected in series with a PReLU activation function, and the fifth Conv is connected in series with a Sigmoid activation function.

3. The low-light image enhancement method according to claim 1, characterized in that: The Navier-Stokes equations are: Among them, the lighting adjustment factor r is defined as: r(I)=r0+r1log(1+αI)+r2log(1+βI); Where k is the current iteration number, n is the slope adaptive factor controlling the Navier-Stokes equation, r0, r1, r2, α and β are all learnable parameters, I is the image, and σ is the average adaptive factor controlling the Navier-Stokes equation curve.

4. The low-light image enhancement method according to claim 1, characterized in that: The high-frequency image of the preprocessed low-light image is sequentially processed through AEM and ESA to obtain the high-frequency image of the low-light image after spatial attention enhancement, specifically: The high-frequency image of the preprocessed low-light image generates four different enhanced features through AEM, among which the first three enhanced features are concat with the high-frequency image of the preprocessed low-light image to obtain the high-frequency image of the low-light image after pre-attention enhancement, and the high-frequency image of the low-light image after pre-attention enhancement and the last enhanced feature are all entered into ESA to obtain the high-frequency image of the low-light image after spatial attention enhancement.

5. The low-light image enhancement method according to claim 1, characterized in that: In the step S6, the loss function is: Where 1 com is the loss function, k is the current iteration number, T is the normal light image corresponding to the low light image, I out is the enhanced image, φ k Feature maps extracted from different layers of the VGG16 network, 1 ssim The structural similarity loss.

6. A low-light image enhancement system, characterized in that: Includes the following modules: Module S1, obtaining a training set and a test set, both of which are low-light images and their corresponding normal-light images; Module S2, for the low-light images in the training set, the frequency-based decomposition network of the low-light images respectively models the low-frequency images and the high-frequency images of the low-light images; Module S3, enhancing the low-frequency image of the low-light image based on the low-frequency pathway illumination adjustment network model to obtain an enhanced low-frequency image of the low-light image; Module S4, enhancing the high-frequency image of the low-light image based on the high-frequency pathway illumination adjustment network model to obtain an enhanced high-frequency image of the low-light image; Module S5, adding the low-frequency image of the enhanced low-light image and the high-frequency image of the low-light image bit by bit to obtain an enhanced image; Module S6, designs a loss function based on the enhanced image and the normal illumination image corresponding to the low illumination image in the training set, and uses the loss function to train the frequency decomposition network model, the low-frequency pathway illumination adjustment network model, and the high-frequency pathway illumination adjustment network model respectively; Module S7, the low-light image in the test set and the corresponding normal-light image perform the operations of modules S2 to S5 to obtain the final enhanced image; The low-frequency path illumination adjustment network model is: First, the low-frequency image of the low-light image is passed through the deep curve estimation network to obtain the parameters required by the Navier-Stokes equation, and then iteratively passes through the Navier-Stokes equation to adjust the illumination. All the obtained illumination information is concat. At the same time, the low-frequency image of the low-light image is passed through the convolution enhancement module to obtain the scene color information, which is input into FusionNet together with all the concat illumination information to obtain the enhanced low-frequency image of the low-light image. The high-frequency path illumination adjustment network model is: First, the high-frequency image of the low-light image passes through a preliminary denoiser to obtain a high-frequency image of the low-light image after denoising, and then passes through Conv to obtain the high-frequency image of the preprocessed low-light image. The high-frequency image of the preprocessed low-light image passes through an adaptive enhancement module and an efficient spatial attention module in turn to obtain a high-frequency image of the low-light image after spatial attention enhancement. After the high-frequency image of the low-light image after spatial attention enhancement passes through Conv, it is bitwise added with the high-frequency image of the low-light image after denoising to obtain the high-frequency image of the enhanced low-light image.

7. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 5 when executing a program stored in a memory.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 5 are implemented.

Images