A low-light image enhancement method and system based on customized unfolding enhancer

By combining the Retinex processing module and the U-Net architecture image enhancement network, the problem of limited effects of low-light image enhancement technology is solved, and efficient, interpretable and real-time image enhancement is achieved, suitable for complex scenarios and practical applications.

CN118365548BActive Publication Date: 2024-11-01JIANGNAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410792810.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-19
Publication Date
2024-11-01
Estimated Expiration
2044-06-19

AI Technical Summary

Technical Problem

The existing low-light image enhancement technology has limited effects when processing complex scenes, deep learning methods lack interpretability, rely on a large amount of labeled data, and have high computational complexity, making it difficult to meet the needs of real-time and efficient.

Method used

Using a low-light image enhancement method based on a custom expansion enhancer, by constructing an image enhancement network combining Retinex processing module and U-Net architecture, introducing structural similarity index and total variation, defining noise suppression loss terms, optimizing loss functions, and realizing images efficient enhancement.

Benefits of technology

It significantly improves the brightness and contrast of low-light images, reduces noise, maintains image structure information, improves the generalization ability and practical value of the model, and meets the needs of real-time and efficientness.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118365548B_ABST
    Figure CN118365548B_ABST
Patent Text Reader

Abstract

The present invention provides a low-light image enhancement method and system based on a customized unfolding enhancer, which relates to the field of image processing technology. The method includes obtaining a data set, the data set includes low-light and normal-light images, dividing the data set into a training set and a test set, and preprocessing the images in the data set; constructing an image enhancement network model, training the image enhancement network model using the training set, optimizing the loss function by introducing a structural similarity index and a total variation, and defining a noise suppression loss term until a preset number of iterations is reached, and obtaining a trained image enhancement network model; testing the trained image enhancement network model using a test set, and evaluating the model performance. The image enhancement network model proposed by the present invention achieves the effect of efficiently enhancing low-light images, reduces noise and maintains structural information by optimizing the loss function, and exhibits wide applicability and practical value.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a low-light image enhancement method and system based on a customized expansion enhancer. Background Art

[0002] In the field of image processing, low-light image enhancement technology has always been a topic of great concern. With the continuous development of science and technology, especially in the fields of security monitoring, night photography, aerial imaging and medical imaging, higher requirements are placed on the image quality under low-light conditions. Low-light image enhancement technology aims to improve the quality of images captured under low-light conditions, making them more clear, realistic and visually comfortable.

[0003] Traditional low-light image enhancement techniques mainly include histogram equalization (HE), gamma correction (GC) and methods based on Retinex theory. These methods improve the overall visual effect of the image by adjusting the contrast, brightness and color balance of the image. However, these methods often have limitations when processing images in complex scenes and low-light conditions. For example, the histogram equalization method may over-enhance high-contrast areas, resulting in loss of details; the gamma correction method has inconsistent processing effects for different brightness levels; and the method based on Retinex theory may face the challenge of insufficient generalization ability when estimating the reflection component and the illumination component at the same time.

[0004] In recent years, with the rapid development of deep learning technology, deep learning-based low-light image enhancement methods have made significant progress. These methods build an end-to-end network architecture and use a large amount of image data to learn the mapping relationship from low illumination to normal illumination, thereby realizing intelligent enhancement of low-light images. Compared with traditional image enhancement methods, deep learning-based methods have stronger generalization ability and higher accuracy, and can handle more diverse and challenging low-light conditions.

[0005] However, deep learning-based methods also face some challenges. First, these methods are usually built in a black box manner and lack sufficient interpretability, making it difficult for users to understand and trust the output of the model. Second, these methods require a large amount of labeled data during the training process, while there are relatively few datasets of labeled low-light images, which limits the generalization ability of the model. In addition, deep learning-based methods are also relatively high in computational complexity and resource consumption, making it difficult to meet the requirements of real-time and high efficiency. Summary of the invention

[0006] To this end, an embodiment of the present invention provides a low-light image enhancement method and system based on a customized expansion enhancer, which is used to solve the problems that traditional methods in the prior art have limited effects when processing complex scenes and low-light images, and may cause loss of details; although deep learning methods have made significant progress, they lack interpretability, rely on a large amount of labeled data, and have high computational complexity, making it difficult to meet real-time and high efficiency requirements.

[0007] In order to solve the above problem, an embodiment of the present invention provides a low-light image enhancement method based on a customized unfolding enhancer, the method comprising:

[0008] Acquire a data set, the data set including low-light images and normal-light images, divide the data set into a training set and a test set, and preprocess the images in the data set;

[0009] Constructing an image enhancement network model, the image enhancement network includes an input layer, a Retinex processing module, a mask autoencoder module, and an output layer, the input layer is used to input an image, the Retinex processing module is used to decompose the input image into an illumination component and a reflection component, the mask autoencoder module is based on a U-Net architecture, is used to adjust the illumination component and restore the reflection component, and multiply the adjusted illumination component and the restored reflection component, and the output layer is used to output an enhanced image;

[0010] The image enhancement network model is trained using the training set, and the loss function is optimized by introducing the structural similarity index and the total variation and defining the noise suppression loss term until a preset number of iterations is reached, thereby obtaining a trained image enhancement network model;

[0011] The trained image enhancement network model is tested using the test set to evaluate the model performance.

[0012] Preferably, the method of preprocessing the images in the data set includes image resizing, image data normalization, and data enhancement techniques, including random cropping, rotation, and flipping.

[0013] Preferably, the Retinex processing module is used to decompose the input image into an illumination component and a reflection component, specifically comprising:

[0014]

[0015] In the formula, I L represents the input image; ° represents the element-by-element product; L L Represents the illumination component; R L Represents the reflection component; N L Represents the noise component.

[0016] Preferably, the masked autoencoder module is based on a U-Net architecture, and is used to adjust the illumination component and restore the reflection component, and multiply the adjusted illumination component and the restored reflection component, specifically comprising:

[0017] According to the received light component, using formula (2), the adjusted illumination component L is obtained en :

[0018] L en =U-Net(L L ; Θ U-Net ) (2)

[0019] In the formula, Θ U-Net Represents the parameter set of the U-Net network; L L represents the illumination component; U-Net(; ) represents the U-Net architecture;

[0020] According to the received reflection component, the restored reflection component R is obtained by reducing the color deviation and noise pollution in the image. en ;

[0021] Multiply the adjusted illumination component and the restored reflection component to obtain the enhanced image I en :

[0022]

[0023] In the formula, Represents element-wise product.

[0024] Preferably, the method of optimizing the loss function by introducing the structural similarity index and the total variation and defining a noise suppression loss term specifically includes:

[0025] By introducing the structural similarity index and total variation, the structural information and smoothness of the image are enhanced respectively. The formula is as follows:

[0026]

[0027] in

[0028] In the formula, Represents the new loss function, SSIM(I en ,I gt ) represents the image I after calculation enhancement en and the target image I gt The structural similarity index between en ) represents the enhanced image I en Total variation loss; L D is the original loss function; The reconstruction loss aims to reduce the pixel-level difference between the enhanced image and the real image; is the reflection similarity loss, which is used to keep the reflectance of the enhanced image similar to that of the original image; It is the mutual edge similarity loss, which emphasizes suppressing noise while maintaining edge information; Smooth the loss of light and ensure a natural transition of light changes;

[0029] Define the noise suppression loss term LN, combine the noise suppression loss term LN and the new loss function Get the final loss function:

[0030]

[0031] Where, L total Represents the final loss function.

[0032] Preferably, the noise suppression loss term LN is:

[0033]

[0034] In the formula, Enhanced image gradient representation for mask autoencoder; represents the normal illumination image gradient representation of the mask autoencoder; ||·||1 represents the L1 norm.

[0035] Preferably, the method for evaluating model performance is:

[0036] Peak signal-to-noise ratio and structural similarity index were used as evaluation indicators to evaluate the model performance.

[0037] The embodiment of the present invention further provides a low-light image enhancement system based on a customized unfolding enhancer, which is used to implement the above-mentioned low-light image enhancement method based on a customized unfolding enhancer, and specifically includes:

[0038] A data preparation module is used to obtain a data set, wherein the data set includes low-light and normal-light images, divide the data set into a training set and a test set, and preprocess the images in the data set;

[0039] An image enhancement network construction module is used to construct an image enhancement network model, wherein the image enhancement network model includes an input layer, a Retinex processing module, a mask autoencoder module, and an output layer. The input layer is used to input an image. The Retinex processing module is used to decompose the input image into an illumination component and a reflection component. The mask autoencoder module is based on a U-Net architecture and is used to adjust the illumination component and restore the reflection component, and multiply the adjusted illumination component and the restored reflection component. The output layer is used to output an enhanced image.

[0040] A model training module is used to train the image enhancement network model using a training set, by introducing a structural similarity index and a total variation, and defining a noise suppression loss term to optimize the loss function until a preset number of iterations is reached, thereby obtaining a trained image enhancement network model;

[0041] The model testing and evaluation module is used to test the trained image enhancement network model using the test set and evaluate the model performance.

[0042] An embodiment of the present invention also provides an electronic device, which includes a processor, a memory and a bus system, wherein the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the low-light image enhancement method based on the customized expansion enhancer described above.

[0043] An embodiment of the present invention further provides a computer storage medium storing a computer software product. The computer software product includes several instructions for enabling a computer device to execute the above-mentioned low-light image enhancement method based on a customized expansion enhancer.

[0044] It can be seen from the above technical solutions that the present invention has the following beneficial effects:

[0045] (1) Innovation and efficiency of model design: The technical solution of the present invention designs an efficient and innovative image enhancement network model by combining the Retinex processing module and the masked autoencoder module based on the U-Net architecture. This design enables the model to accurately decompose the image into illumination components and reflection components, and perform targeted processing on each component, thereby significantly improving the brightness and contrast of low-light images while maintaining the image structure information.

[0046] (2) Optimization of loss function and performance improvement: During the training process, the structural similarity index (SSIM) and total variation (TV) are introduced as regularization terms, and a noise suppression loss term is defined to optimize the loss function. This optimization strategy ensures that the model can reduce noise and maintain the structural information of the image while enhancing image quality. By adjusting the weights of the loss terms, the model can achieve balanced optimization in multiple aspects, thereby achieving performance that is superior to existing technologies in multiple evaluation indicators.

[0047] (3) Wide applicability and practicality: The technical solution of the present invention not only achieved remarkable results in laboratory environments, but also showed good generalization ability and practical value when processing images in actual low-light environments. This means that the technology is not only valuable for theoretical research, but also has broad application prospects in practical applications, such as security monitoring, night driving assistance and other fields. At the same time, through rigorous testing and evaluation, the reliability and stability of the technology have been fully verified, providing strong support for practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the implementation cases of the present invention or the technical solutions in the prior art, the following is a brief description of the drawings required for use in the embodiments. By referring to the drawings, the features and advantages of the present invention will be more clearly understood. The drawings are schematic and should not be understood as limiting the present invention in any way. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0049] Figure 1 A flowchart of a low-light image enhancement method based on a customized unfolding enhancer provided in an embodiment;

[0050] Figure 2 It is an architecture diagram of the image enhancement network model in the embodiment;

[0051] Figure 3 The comparison results of the method of the present invention and the traditional method on the LOL dataset in the embodiment are shown in FIG., wherein (a) is the input image, (b) is the output result of the traditional method, and (c) is the output result of the method of the present invention;

[0052] Figure 4 A block diagram of a low-light image enhancement system based on a customized expansion enhancer provided in an embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] Embodiment 1

[0055] In order to solve the problems that traditional methods in the existing technology have limited effects in processing complex scenes and low-light images, which may lead to loss of details; deep learning methods have made significant progress, but lack interpretability, rely on a large amount of labeled data, and have high computational complexity, making it difficult to meet the real-time and high-efficiency requirements. Figure 1 As shown, an embodiment of the present invention provides a low-light image enhancement method based on a customized unfolding enhancer, the method comprising:

[0056] Step S1: obtaining a data set, the data set including low-light and normal-light images, dividing the data set into a training set and a test set, and preprocessing the images in the data set;

[0057] Step S2: constructing an image enhancement network model, the image enhancement network model comprising an input layer, a Retinex processing module, a mask autoencoder module, and an output layer, the input layer is used to input an image, the Retinex processing module is used to decompose the input image into an illumination component and a reflection component, the mask autoencoder module is based on a U-Net architecture, and is used to adjust the illumination component and restore the reflection component, and multiply the adjusted illumination component and the restored reflection component, and the output layer is used to output an enhanced image;

[0058] Step S3: training the image enhancement network model using the training set, optimizing the loss function by introducing the structural similarity index and the total variation and defining the noise suppression loss term, until a preset number of iterations is reached, thereby obtaining a trained image enhancement network model;

[0059] Step S4: Use the test set to test the trained image enhancement network model and evaluate the model performance.

[0060] It can be seen from the above technical solution that the present invention provides a low-light image enhancement method based on a customized unfolding enhancer. First, by obtaining and preprocessing a data set containing low-light and normal-light images, the basic data quality of model training is ensured. Subsequently, an image enhancement network that integrates a Retinex processing module and a masked autoencoder module based on a U-Net architecture is constructed. This design can accurately decompose and process the illumination and reflection components, thereby significantly improving the image quality. During the training process, the loss function is optimized by introducing a structural similarity index, total variation, and noise suppression loss terms, which not only ensures the brightness, contrast, and detail information of the enhanced image, but also reduces noise interference. Finally, the model performance is verified by a test set, proving that the technical solution has excellent generalization ability and practical value when processing low-light images. This comprehensive and sophisticated technical process enables the solution to show significant advantages in the field of low-light image enhancement.

[0061] In this embodiment, in step S1, a data set is obtained, the data set includes low-light and normal-light images, the data set is divided into a training set and a test set, and the images in the data set are preprocessed. The preprocessing method includes image resizing to unify the input image size; normalizing the image data so that the model can better process the input data; in order to increase the generalization ability of the model, data enhancement techniques can be used, including random cropping, rotation, and flipping.

[0062] In this embodiment, in step S2, an image enhancement network model is constructed. Figure 2 As shown, the image enhancement network includes an input layer, a Retinex processing module, a mask autoencoder module, and an output layer, wherein the input layer is used to input an image; the Retinex processing module is used to decompose the input image into an illumination component and a reflection component; the mask autoencoder module is based on a U-Net architecture, and is used to adjust the illumination component and restore the reflection component, and multiply the adjusted illumination component and the restored reflection component; the output layer is used to output an enhanced image.

[0063] The custom unwrapped enhancer is a deep learning method designed for low-light image enhancement problems. It is based on the improved Retinex theory and combines the Masked Auto Encoder (MAE) to create an end-to-end enhancement network, aiming to enhance image brightness, reduce noise, and improve color fidelity.

[0064] Specifically, the constructed image enhancement network model is used to decompose the image into two components: illumination and reflection. The illumination prior and noise model are learned through the deep network, so that the model can not only enhance the brightness of the image, but also reduce the influence of noise during the enhancement process.

[0065] Furthermore, the input image is decomposed into illumination component and reflection component, specifically including:

[0066]

[0067] In the formula, I L represents the input image; represents element-wise product; I L Represents the illumination component; R L Represents the reflection component; N L Represents the noise component.

[0068] Further,

[0069]

[0070]

[0071] Where, L is the illumination image; LN is the reflection image; N is the noise image; represents the gradient; β, γ, ω, μ, ∈, η are all parameters; argmin represents the parameter value that makes the function reach the minimum value; ||·||1 represents the L1 norm; ||·|| 2 represents the L2 norm.

[0072] According to the received light component, using formula (2), the adjusted illumination component L is obtained en :

[0073] L en =U-Net(L L ; Θ U-Net ) (2)

[0074] In the formula, Θ U-Net Represents the parameter set of the U-Net network; L L represents the illumination component; U-Net(; ) represents the U-Net architecture;

[0075] According to the received reflection component, the restored reflection component R is obtained by reducing the color deviation and noise pollution in the image. en ;

[0076] Multiply the adjusted illumination component and the restored reflection component to obtain the enhanced image I en :

[0077]

[0078] In this embodiment, in step S3, the image enhancement network model is trained using the training set, and the loss function is optimized by introducing the structural similarity index and the total variation and defining the noise suppression loss term until a preset number of iterations is reached to obtain a trained image enhancement network model, which specifically includes:

[0079] The loss function is a key factor in guiding model training in deep learning, which defines the gap between the model output and the target. In the traditional custom expansion enhancer, the original loss function L D It consists of the following four main parts:

[0080]

[0081] Where, L D is the original loss function; The reconstruction loss aims to reduce the pixel-level difference between the enhanced image and the real image; is the reflection similarity loss, which is used to keep the reflectance of the enhanced image similar to that of the original image; It is the mutual edge similarity loss, which emphasizes suppressing noise while maintaining edge information; Smooths out the loss of lighting and ensures a natural transition of lighting changes.

[0082] On this basis, the present invention introduces the structural similarity index (SSIM) and total variation (TV) as supplementary items to enhance the structural information and smoothness of the image respectively. The specific formulas are as follows:

[0083]

[0084] In the formula, Represents the new loss function, SSIM(I en ,I gt ) represents the image I after calculation enhancement en and the target image I gt The structural similarity index between en ) represents the enhanced image I en The total variation loss.

[0085] SSIM considers the similarity of the brightness, contrast and structure of the image, which makes the model pay more attention to maintaining the structural integrity of the image during training. TV is an edge-preserving smoothing term that helps the model reduce noise while avoiding excessive smoothing and loss of details.

[0086] In addition to the improvement of the loss function, this paper also focuses on how to optimize image quality by customizing noise prior. Noise prior plays a key role in low-light image enhancement because low-light images are often accompanied by increased noise. To this end, we use a MAE-based method to customize noise prior, which can learn from the noise distribution of the image and use it to guide noise removal.

[0087] In the method of the present invention, MAE is pre-trained and has a strong ability to capture image gradient information. This ability enables the present invention to constrain the noise in the enhanced image through the consistency of the gradient representation. Specifically, the present invention defines the following noise suppression loss term LN:

[0088]

[0089] In the formula, Enhanced image gradient representation for mask autoencoder; represents the normal illumination image gradient representation of the mask autoencoder; ||·||1 represents the L1 norm.

[0090] This step ensures that the enhanced image is consistent with the normal illumination image at the gradient level, thereby effectively suppressing noise. Overall, our optimization process is trained end-to-end with a comprehensive loss function, thereby achieving the goal of reducing noise and improving image quality while maintaining image details and structure. Specifically, by combining the noise suppression loss term LN and the new loss function Get the final loss function:

[0091]

[0092] Where, L total Represents the final loss function.

[0093] In this embodiment, in step S4, the trained image enhancement network model is tested using a test set and the model performance is evaluated. Specifically, the peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM) are used as evaluation indicators. PSNR measures the maximum possible error between the enhanced image and the reference image, while SSIM evaluates the similarity of the image in brightness, contrast and structure.

[0094] In addition, based on the optimization of the loss function, we took further measures to optimize the model. These measures include adjusting the model architecture, optimizing the training process, and fine-tuning the hyperparameters, aiming to improve the overall enhancement effect of low-light images.

[0095] 1. Adjustment of model architecture

[0096] Increase model depth: For the encoder part of U-Net, the depth can be increased by stacking more convolutional layers, pooling layers, or residual blocks. For example, add more convolutional layers in each residual block, or increase the number of stacked residual blocks. For the decoder part, the depth can also be increased by increasing the number of upsampling layers, convolutional layers, or residual blocks. This helps to gradually restore the details of the image during the decoding process.

[0097] Increase the number of heads of the attention mechanism: Increase the number of heads of the attention mechanism in specific parts of the model (such as the multi-head self-attention module in the Transformer structure). This can be achieved by adjusting the parameters in the multi-head attention module. Similarly, a multi-head attention mechanism can be introduced in the decoder part and the number of heads can be increased by adjusting the parameters. The multi-head attention mechanism can capture different feature representations in the input data, thereby improving the performance of the model.

[0098] 2. Training process optimization

[0099] Increase the number of iterations: In the training configuration, increase the number of iterations of model training to extend the training time of the model. More iterations means that the model has more opportunities to learn the features in the data, thereby improving its performance.

[0100] Optimizer and learning rate adjustment: Choose an optimizer suitable for the task, such as Adam, RMSprop, or SGD. These optimizers use different update strategies to update the parameters of the model during training.

[0101] Adjust the learning rate strategy, such as using learning rate decay or learning rate scheduler. Learning rate is one of the key factors affecting model training speed and performance. By adjusting the learning rate, you can control the update step size of the model during training, thereby preventing overfitting or underfitting.

[0102] 3. Hyperparameter and strategy adjustment

[0103] Learning rate scheduling strategy: Use learning rate schedulers (such as StepLR, ExponentialLR, CosineAnnealingLR, etc.) to dynamically adjust the learning rate. These schedulers can adjust the learning rate based on performance indicators during training or fixed training steps. By specifying the type and parameters of the scheduler in the training configuration, dynamic adjustment of the learning rate can be easily achieved.

[0104] Further refinement of the loss function: Add additional regularization terms such as L1 regularization, L2 regularization, or weight decay to the original loss function. These regularization terms can help prevent the model from overfitting and improve its generalization ability on unseen data. Rebalance the weights of each loss term. In some cases, some loss terms may have a greater impact on the performance of the model. By adjusting the weights of these loss terms, you can ensure that the model is optimized in a balanced manner in different aspects.

[0105] In order to further verify the superiority of the method of the present invention, it is described below in conjunction with specific experiments.

[0106] 1. Experimental Setup

[0107] To evaluate the performance of image enhancement algorithms, we used the LOL dataset, which was jointly released by Huawei and Harbin Institute of Technology. The LOL dataset contains 500 pairs of low-light and normal-light images. These image pairs provide diverse scene changes, prompting the algorithm to learn how to improve image quality under different lighting conditions. The images in the dataset have been carefully selected and processed. The low-light images reveal the common challenges when shooting in dark environments, such as noise and contrast loss, while the normal-light images provide a clear reference standard. We used peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) as evaluation metrics. PSNR measures the maximum possible error between the enhanced image and the reference image, while SSIM evaluates the similarity of the image in brightness, contrast, and structure. We used 485 pairs for training and 15 pairs for testing.

[0108] 2. Experimental Details

[0109] Due to hardware and time constraints, we originally needed to train 300,000 iterations, but we changed it to 3,000 times. However, this still took nearly 30 minutes. In addition, we selected the optimal parameters for control experiments.

[0110] 3. Experimental Results

[0111] The traditional model and the model designed by the present invention are trained respectively, and finally tested. The enhanced low-light image effects are compared. Figure 3 The evaluation index results are shown in Table 1 below:

[0112] Table 1 Index comparison

[0113]

[0114] Embodiment 2

[0115] like Figure 4 As shown, the present invention provides a low-light image enhancement system based on a customized unfolding enhancer, which is used to implement the low-light image enhancement method based on a customized unfolding enhancer of the above-mentioned embodiment 1, and specifically includes:

[0116] The data preparation module 100 is used to obtain a data set, the data set includes low-light and normal-light images, divide the data set into a training set and a test set, and preprocess the images in the data set;

[0117] An image enhancement network construction module 200 is used to construct an image enhancement network model, wherein the image enhancement network model includes an input layer, a Retinex processing module, a mask autoencoder module, and an output layer, wherein the input layer is used to input an image, the Retinex processing module is used to decompose the input image into an illumination component and a reflection component, the mask autoencoder module is based on a U-Net architecture, and is used to adjust the illumination component and restore the reflection component, and multiply the adjusted illumination component and the restored reflection component, and the output layer is used to output an enhanced image;

[0118] A model training module 300 is used to train the image enhancement network model using a training set, by introducing a structural similarity index and a total variation, and defining a noise suppression loss term to optimize a loss function, until a preset number of iterations is reached, thereby obtaining a trained image enhancement network model;

[0119] The model testing and evaluation module 400 is used to test the trained image enhancement network model using a test set and evaluate the model performance.

[0120] A low-light image enhancement system based on a customized expansion enhancer in this embodiment is used to implement the aforementioned low-light image enhancement method based on a customized expansion enhancer. Therefore, the specific implementation method of the low-light image enhancement system based on the customized expansion enhancer can be seen in the embodiment part of the low-light image enhancement method based on the customized expansion enhancer in the previous text. For example, the data preparation module 100, the image enhancement network construction module 200, the model training module 300, and the model testing and evaluation module 400 are respectively used to implement steps S1, S2, S3, and S4 in the above-mentioned low-light image enhancement method based on the customized expansion enhancer. Therefore, its specific implementation method can refer to the description of the corresponding various parts of the embodiment. In order to avoid redundancy, it will not be repeated here.

[0121] Embodiment 3

[0122] An embodiment of the present invention also provides an electronic device, which includes a processor, a memory and a bus system, wherein the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the low-light image enhancement method based on the customized expansion enhancer described above.

[0123] Embodiment 4

[0124] An embodiment of the present invention further provides a computer storage medium storing a computer software product. The computer software product includes several instructions for enabling a computer device to execute the above-mentioned low-light image enhancement method based on a customized expansion enhancer.

[0125] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0126] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0128] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A low-light image enhancement method based on a customized unfolding enhancer, characterized in that: include: Acquire a data set, the data set including low-light images and normal-light images, divide the data set into a training set and a test set, and preprocess the images in the data set; Constructing an image enhancement network model, the image enhancement network model includes an input layer, a Retinex processing module, a mask autoencoder module, and an output layer, the input layer is used to input an image, the Retinex processing module is used to decompose the input image into an illumination component and a reflection component, the mask autoencoder module is based on a U-Net architecture, is used to adjust the illumination component and restore the reflection component, and multiply the adjusted illumination component and the restored reflection component, and the output layer is used to output an enhanced image; The image enhancement network model is trained using the training set, and the loss function is optimized by introducing the structural similarity index and the total variation and defining the noise suppression loss term until a preset number of iterations is reached, thereby obtaining a trained image enhancement network model; The method of optimizing the loss function by introducing the structural similarity index and the total variation and defining the noise suppression loss term specifically includes: By introducing the structural similarity index and total variation, the structural information and smoothness of the image are enhanced respectively. The formula is as follows: in In the formula, Represents the new loss function, SSIM(I en ,I gt ) represents the image I after calculation enhancement en and the target image I gt The structural similarity index between en ) represents the enhanced image I en Total variation loss; L D is the original loss function; The reconstruction loss aims to reduce the pixel-level difference between the enhanced image and the real image; is the reflection similarity loss, which is used to keep the reflectance of the enhanced image similar to that of the original image; It is the mutual edge similarity loss, which emphasizes suppressing noise while maintaining edge information; Smooth the loss of light and ensure a natural transition of light changes; Define the noise suppression loss term LN, combine the noise suppression loss term LN and the new loss function Get the final loss function: Where, L total Represents the final loss function; The trained image enhancement network model is tested using the test set to evaluate the model performance.

2. The low-light image enhancement method based on the customized unfolding enhancer according to claim 1, characterized in that: The method for preprocessing the images in the data set includes image resizing, image data normalization, and data enhancement techniques, including random cropping, rotation, and flipping.

3. The low-light image enhancement method based on the customized unfolding enhancer according to claim 1, characterized in that: The Retinex processing module is used to decompose the input image into an illumination component and a reflection component, specifically including: In the formula, I L represents the input image; represents element-wise product; L L Represents the illumination component; R L Represents the reflection component; N L Represents the noise component.

4. The low-light image enhancement method based on a customized unfolding enhancer according to claim 1, characterized in that: The masked autoencoder module is based on the U-Net architecture and is used to adjust the illumination component and restore the reflection component, and multiply the adjusted illumination component and the restored reflection component, specifically including: According to the received light component, using formula (2), the adjusted illumination component L is obtained en : L en =U-Net(L L ;Θ U-Net ) (2) In the formula, Θ U-Net Represents the parameter set of the U-Net network; L L represents the illumination component; U-Net(; ) represents the U-Net architecture; According to the received reflection component, the restored reflection component R is obtained by reducing the color deviation and noise pollution in the image. en ; Multiply the adjusted illumination component and the restored reflection component to obtain the enhanced image I en : In the formula, Represents element-wise product.

5. The low-light image enhancement method based on a customized unfolding enhancer according to claim 1, characterized in that: The noise suppression loss term LN is: In the formula, Enhanced image gradient representation for mask autoencoder; represents the normal illumination image gradient representation of the mask autoencoder; ||·||1 represents the L1 norm.

6. The low-light image enhancement method based on a customized unfolding enhancer according to claim 1, characterized in that: The method for evaluating model performance is: Peak signal-to-noise ratio and structural similarity index were used as evaluation indicators to evaluate the model performance.

7. A low-light image enhancement system based on a customized unfolding enhancer, characterized in that: The system is used to implement the low-light image enhancement method based on a customized unfolding enhancer according to any one of claims 1 to 6, specifically comprising: A data preparation module is used to obtain a data set, wherein the data set includes low-light and normal-light images, divide the data set into a training set and a test set, and preprocess the images in the data set; An image enhancement network construction module is used to construct an image enhancement network model, wherein the image enhancement network model includes an input layer, a Retinex processing module, a mask autoencoder module, and an output layer. The input layer is used to input an image. The Retinex processing module is used to decompose the input image into an illumination component and a reflection component. The mask autoencoder module is based on a U-Net architecture and is used to adjust the illumination component and restore the reflection component, and multiply the adjusted illumination component and the restored reflection component. The output layer is used to output an enhanced image. A model training module is used to train the image enhancement network model using a training set, by introducing a structural similarity index and a total variation, and defining a noise suppression loss term to optimize the loss function until a preset number of iterations is reached, thereby obtaining a trained image enhancement network model; The model testing and evaluation module is used to test the trained image enhancement network model using the test set and evaluate the model performance.

8. An electronic device, characterized in that: The electronic device includes a processor, a memory and a bus system, the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the low-light image enhancement method based on the customized expansion enhancer as described in any one of claims 1 to 6.

9. A computer storage medium, characterized in that The computer storage medium stores a computer software product, and the computer software product includes a number of instructions for enabling a computer device to execute the low-light image enhancement method based on a customized expansion enhancer as described in any one of claims 1 to 6.