An unsupervised low-light enhancement method based on histogram equalization prior

Abstract

The application discloses an unsupervised low-light enhancement method based on histogram equalization prior, and comprises the following steps: obtaining a low-light image to be enhanced; inputting the low-light image to be enhanced into a pre-constructed and trained unsupervised low-light enhancement model; based on a light enhancement module of the unsupervised low-light enhancement model, performing frequency domain feature extraction and cascade feature learning on the low-light image to be enhanced to obtain a reflection image; and based on a noise removal module of the unsupervised low-light enhancement model, performing content coding and noise removal processing on the reflection image to obtain an enhanced low-light image. The application can effectively decompose the low-light image, separate noise and content, restore more fine details, and significantly improve the robustness in a complex low-light environment.

Description

Technical Field

[0001] This invention relates to an unsupervised low-light enhancement method based on histogram equalization prior, belonging to the field of image processing technology. Background Technology

[0002] Images captured in low light conditions not only have poor aesthetic quality and significantly reduce the enjoyable sensory experience, but also greatly diminish the information they convey. Furthermore, they make it difficult to achieve high performance in other advanced computer vision tasks, such as object detection, object segmentation, and object tracking, because most solutions for these tasks are designed for images with good exposure conditions. Therefore, there is a need for a method that can effectively improve the quality of low-light images.

[0003] Traditional low-light image enhancement methods include histogram-based, dehazing-based, and retina-based methods. Histogram-based methods redistribute the histogram to a uniform distribution and adjust the Gamma curve exponent. Dehazing-based methods utilize the similarity between low-light enhancement and dehazing in reverse image processing. Retina-based methods typically decompose low-light images into illumination and reflectance components, reconstructing better enhancement results. However, most retina-based methods assume that the reflectance component remains unchanged during enhancement, regardless of color distortion and lost details. In recent years, deep learning-based low-light enhancement has emerged; however, due to the difficulty of data acquisition, most existing datasets are created by artificially lowering the exposure of a normal-light image, resulting in poor performance of supervised deep learning methods in complex environments. Unsupervised methods have been proposed to eliminate the dependence on paired training data; however, these methods are often unstable, lack robustness to different complex environments, and the generated images are susceptible to significant noise, making it difficult to guarantee image quality.

[0004] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an unsupervised low-light enhancement method based on histogram equalization prior. Based on the illumination enhancement module and noise removal module of the pre-built and trained unsupervised low-light enhancement model, it can effectively decompose low-light images and separate noise and content, restore finer details, and significantly improve robustness in complex low-light environments.

[0006] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0007] This invention discloses an unsupervised low-light enhancement method based on histogram equalization prior, comprising the following steps:

[0008] Acquire the low-light image to be enhanced;

[0009] The low-light image to be enhanced is input into a pre-built and trained unsupervised low-light enhancement model;

[0010] The illumination enhancement module based on the unsupervised low-light enhancement model performs frequency domain feature extraction and cascaded feature learning on the low-light image to be enhanced to obtain a reflection image.

[0011] Based on the noise removal module of the unsupervised low-light enhancement model, the reflected image is subjected to content encoding and denoising to obtain the enhanced low-light image.

[0012] Furthermore, the training method for the unsupervised low-light enhancement model includes the following steps:

[0013] Obtain the training set and the pre-built initial model;

[0014] The training set is input into the pre-built initial model for training to obtain a trained unsupervised low-light enhancement model;

[0015] The initial model includes an initial illumination enhancement module and an initial noise removal module. The initial illumination enhancement module includes a shared weighted cascaded feature learning network composed of an initial frequency domain feature extraction unit and multiple initial cascaded feature learning units, as well as a Vgg16 network. The initial noise removal module includes an initial noise generator, an initial first content encoder, an initial second content encoder, an initial noisy image generator, and an initial denoised image generator.

[0016] The unsupervised low-light enhancement model includes an illumination enhancement module and a noise removal module. The illumination enhancement module includes a frequency domain feature extraction unit and a cascaded feature learning unit. The noise removal module includes a first content encoder and a denoised image generator.

[0017] Furthermore, based on the illumination enhancement module of the unsupervised low-light enhancement model, frequency domain feature extraction and cascaded feature learning are performed on the low-light image to be enhanced to obtain a reflection image, including the following steps;

[0018] The low-light image to be enhanced is input into the frequency domain feature extraction unit for frequency domain feature extraction, and the frequency domain features are output.

[0019] The low-light image to be enhanced is input into a cascaded feature learning unit, and the frequency domain features are fused to perform cascaded feature learning to obtain a reflection image.

[0020] Furthermore, based on the noise removal module of the unsupervised low-light enhancement model, the reflected image is subjected to content encoding and denoising to obtain the enhanced low-light image, including the following steps:

[0021] The reflection image is input into a first content encoder for content encoding, and the reflection features are output.

[0022] The reflection features are input into a denoising image generator for denoising processing, and an enhanced low-light image is output.

[0023] Furthermore, the training set includes training images to be enhanced and sample images;

[0024] The training set is input into the pre-built initial model for training, including the following steps:

[0025] Perform histogram equalization on the training image to be enhanced to obtain the first image;

[0026] The training image to be enhanced is input into a shared-weighted cascaded feature learning network, the unsupervised loss is calculated, and the second image is output.

[0027] The first image and the second image are respectively input into the Vgg16 network to obtain the histogram prior loss;

[0028] Based on the sample image and the second image, noise generation and denoising are performed using the initial noise removal module, and KL loss and adversarial loss are calculated.

[0029] The cycle consistency loss is calculated by repeating the noise generation and denoising steps.

[0030] Specifically, based on the unsupervised loss and histogram prior loss, the parameters of the initial frequency domain feature extraction unit and the initial cascaded feature learning unit are optimized to obtain the frequency domain feature extraction unit and cascaded feature learning unit of the unsupervised low-light enhancement model.

[0031] Based on the KL loss, adversarial loss, and cycle consistency loss, the parameters of the initial first content encoder and the initial denoised image generator are optimized to obtain the first content encoder and denoised image generator of the unsupervised low-light enhancement model.

[0032] Furthermore, the unsupervised loss is calculated and a second image is output, including:

[0033] The training image to be enhanced is input into the initial frequency domain feature extraction unit, and the training frequency domain features are output.

[0034] Based on multiple initial cascaded feature learning units, shared cascaded learning and cyclic decomposition are performed on the training image to be enhanced and the training frequency domain features, unsupervised loss is calculated and a second image is output; wherein, the output of the current initial cascaded feature learning unit is used as the input of the next initial cascaded feature learning unit;

[0035] The expression for the output of the initial cascaded feature learning unit is as follows:

[0036] y t =cycconv(u),if t=1,u=x,else u=y t-1

[0037] Among them, y t y represents the output of the t-th initial cascaded feature learning unit; t-1 represents the output of the (t-1)th initial cascaded feature learning unit; x represents the training image to be enhanced; u represents the variable; cycconv(·) represents the shared-weight cascaded feature learning network function;

[0038] The expression for the unsupervised loss is:

[0039] R = y t [;;0:3];

[0040] L = y t [;;3];

[0041]

[0042]

[0043]

[0044] Where R represents the reflection map; L represents the illumination map; Indicates the losses incurred during reconstruction; Indicates the loss of illumination smoothness; This represents the total loss of the initial lighting module; ε represents the gradient; ε represents the smallest positive constant.

[0045] Furthermore, the training image to be enhanced is input into the initial frequency domain feature extraction unit, and the training frequency domain features are output, including the following steps:

[0046] Based on the fourth convolutional layer within the initial frequency domain feature extraction unit, feature processing is performed on the training image to be enhanced to obtain spatial domain features;

[0047] The spatial domain features are transformed into frequency domain information through Fourier transform and FResNet function;

[0048] The frequency domain information is transformed into training frequency domain features in the spatial domain through inverse Fourier transform; the training frequency domain features are then output.

[0049] The four convolutional layers include 3×3 convolutional kernels and LeakyReLU functions.

[0050] Furthermore, the initial cascaded feature learning unit includes a first convolutional layer, a fusion sub-unit for fusing training frequency domain features, a second convolutional layer, and a third convolutional layer connected in sequence.

[0051] The first convolutional layer consists of eight 3×3 convolutional kernels and one ReLU activation function;

[0052] The second convolutional layer consists of eight 3×3 convolutional kernels and one ReLU activation function;

[0053] The third convolutional layer consists of three 3×3 convolutional kernels and a ReLU activation function.

[0054] Furthermore, the KL loss and adversarial loss are calculated, including the following steps:

[0055] The second image is input into the initial noise generator, the KL loss is calculated and the noise information is output.

[0056] The sample image is input into the initial second content encoder, which outputs sample features; the sample features and noise information are input into the initial noise image generator, which outputs a third image.

[0057] The second image is input into the initial first content encoder, which outputs the second feature; the second feature is input into the initial denoised image generator, which outputs the fourth image.

[0058] Calculate the adversarial loss based on the third and fourth images;

[0059] The expression for the KL loss is as follows:

[0060]

[0061] in, d represents the KL loss; d represents the dimension of the noise feature. The standard deviation of noise; The average value of the noise is represented by ; log represents the logarithmic function.

[0062] The expression for the adversarial loss is as follows:

[0063]

[0064] in, Indicates resistance to loss; Represents the true sample distribution; denoted by ; D(·) represents the discriminator; x represents the real sample; G(z) represents the generated sample; b represents the real sample label; a represents the generated sample label.

[0065] Furthermore, the expression for the cycle consistency loss is as follows:

[0066]

[0067] in, Indicates the loss of cycle consistency; I rec represents the output sample; I represents the input sample; ||·||1 represents the norm modulo operation.

[0068] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0069] The present invention provides an effective low-light image enhancement method based on histogram equalization prior knowledge. It provides a pre-built and trained unsupervised low-light enhancement model. Based on the illumination enhancement module and noise removal module in the model, it can effectively decompose low-light images and separate noise and content. On the one hand, it can restore finer details and suppress noise in real-world low-light scenes. On the other hand, it can significantly improve robustness in complex low-light environments. Attached Figure Description

[0070] Figure 1 This is a flowchart of an unsupervised low-light enhancement model;

[0071] Figure 2 This is a flowchart of the initial model training process;

[0072] Figure 3 It is a detailed structural diagram of the initial frequency domain feature extraction unit or the frequency domain feature extraction unit;

[0073] Figure 4 It is a detailed structural diagram of the initial cascaded feature learning unit or the cascaded feature learning unit;

[0074] Figure 5 These are example diagrams showing the results provided in the embodiments. Detailed Implementation

[0075] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0076] Example

[0077] This embodiment provides an unsupervised low-light enhancement method based on histogram equalization prior, including the following steps:

[0078] Acquire the low-light image to be enhanced;

[0079] The low-light image to be enhanced is input into a pre-built and trained unsupervised low-light enhancement model;

[0080] The illumination enhancement module based on the unsupervised low-light enhancement model performs frequency domain feature extraction and cascaded feature learning on the low-light image to be enhanced, and obtains the reflection image.

[0081] A noise removal module based on an unsupervised low-light enhancement model performs content encoding and denoising on the reflective image to obtain an enhanced low-light image.

[0082] like Figure 1 As shown, the unsupervised low-light enhancement model includes an illumination enhancement module and a noise removal module. The illumination enhancement module includes a frequency domain feature extraction unit and a cascaded feature learning unit, and the noise removal module includes a first content encoder. and denoising image generator G Y .

[0083] The workflow of the unsupervised low-light enhancement model is as follows:

[0084] (1) The low-light image to be enhanced X 低 The input is fed into the Frequency Domain Feature Extraction Unit (FTM) for frequency domain feature extraction, and the output is the frequency domain feature.

[0085] X the low-light image to be enhanced 低 The input is fed into the cascaded feature learning unit (CFM), where frequency domain features are fused for cascaded feature learning to obtain the reflection image X. 反 .

[0086] (2) The reflected image X 反 Input to the first content encoder Perform content encoding and output reflection features;

[0087] The reflection features are input into the denoising image generator G. Y Denoising is performed, and the enhanced low-light image X is output. 强 .

[0088] like Figure 2 As shown, the training method for the unsupervised low-light enhancement model includes the following steps:

[0089] 1. Obtain the training set and the pre-built initial model; the training set includes the training image P to be enhanced and the sample image T; the initial model includes an initial illumination enhancement module and an initial noise removal module. The initial illumination enhancement module includes a shared-weighted cascaded feature learning network composed of an initial frequency domain feature extraction unit and multiple initial cascaded feature learning units, as well as a Vgg16 network; the initial noise removal module includes an initial noise generator E. N (Initial) First Content Encoder (Initial), Initial Second Content Encoder (Initial) Initial Noise Image Generator G X (Initial) and initial denoised image generator G Y (early).

[0090] 2. Input the training set into the pre-built initial model for training to obtain a trained unsupervised low-light enhancement model, including the following steps:

[0091] Perform histogram equalization on the training image P to be enhanced to obtain the first image P1;

[0092] The training image P to be enhanced is input into a shared-weighted cascaded feature learning network, the unsupervised loss is calculated and the second image P2 is output.

[0093] The first image P1 and the second image P2 are respectively input into the Vgg16 network to obtain the histogram prior loss;

[0094] Based on the sample image T and the second image P2, noise generation and denoising are performed using the initial noise removal module, and KL loss and adversarial loss are calculated.

[0095] The steps of generating and denoising cyclic noise, and the calculation of cycle consistency loss;

[0096] Specifically, based on the unsupervised loss and histogram prior loss, the parameters of the initial frequency domain feature extraction unit and the initial cascaded feature learning unit are optimized to obtain the frequency domain feature extraction unit and cascaded feature learning unit of the unsupervised low-light enhancement model.

[0097] Based on KL loss, adversarial loss, and cycle consistency loss, an initial first content encoder is constructed. (Initial) and initial denoised image generator G Y (Initial) parameter optimization yields the first content encoder of the unsupervised low-light enhancement model. and denoising image generator G Y .

[0098] The specific steps are as follows:

[0099] Step A: Input the training image P to be enhanced into the initial frequency domain feature extraction unit, and output the training frequency domain features, such as... Figure 3 As shown, it includes the following steps:

[0100] Based on the fourth convolutional layer within the initial frequency domain feature extraction unit, feature processing is performed on the training image P to be enhanced to obtain spatial domain features.

[0101] Spatial domain features are transformed into frequency domain information through Fourier transform and FResNet function;

[0102] The frequency domain information is transformed into training frequency domain features in the spatial domain through inverse Fourier transform.

[0103] Output training frequency domain features;

[0104] The four convolutional layers consist of a 3×3 convolutional kernel and a LeakyReLU function.

[0105] The specific principle is as follows:

[0106] The goal of Fourier transform is to obtain the signal distribution in the frequency domain. Digital images are also signals; Fourier transform can convert an image from the spatial domain to the frequency domain, and inverse Fourier transform can convert it from the frequency domain back to the spatial domain. Therefore, we can obtain the spectral information of the image obtained through Fourier transform. High-frequency signals represent content that changes drastically in the image, i.e., details or noise, while low-frequency signals represent content that changes smoothly and does not change frequently, i.e., the background. We can recover more details and enhance high-frequency signals in an image, thereby restoring a clear image from a degraded image. This invention uses a fast Fourier transform method to obtain a matrix corresponding to the original image. The points in the matrix h = a + jb describe the domain information of the frequency domain image. The frequency domain convolution filter matrix is ​​W = A + jB. After convolution, we can obtain:

[0107] W*h=(A*aB*b)+j(B*a+A*b)

[0108] The frequency domain ReLU also consists of two parts: a real part and an imaginary part.

[0109] cRelu(W) = Relu(A) + jRelu(B)

[0110] This invention first processes features in the spatial domain using a first reconstruction, then converts the output function to the frequency domain using a Fast Fourier Transform, and finally extracts features from the frequency domain information using frequency domain convolution.

[0111] Specifically, we used a 3×3 convolution kernel in the spatial domain and the LeakyReLU function in the spatial domain. Then, we used Fast Fourier Transform to convert the features into frequency domain information, and used frequency domain convolution and frequency domain ReLU function to implement the ResNet module in the frequency domain. Finally, we used Inverse Fourier Transform to convert the frequency domain information back to the spatial domain. This can maximize the expansion of information in the spatial and frequency domains and fuse the feature information with the features of the shared weight cascaded feature learning network.

[0112] Step B: Simultaneously with Step A, the training image P to be enhanced is input into the initial cascaded feature learning unit. The training frequency domain features obtained from the initial frequency domain feature extraction unit are fused with the features extracted by multiple initial cascaded feature learning units to construct a fusion module. Rich low-level features are extracted through shared cascaded learning, and the low-light image is iteratively optimized and decomposed using an unsupervised loss function to construct the basic low-light image enhancement framework.

[0113] like Figure 4 As shown, the initial cascaded feature learning unit in this embodiment includes a first convolutional layer, a fusion sub-unit for fusing training frequency domain features, a second convolutional layer, and a third convolutional layer connected in sequence.

[0114] The first convolutional layer consists of eight 3×3 convolutional kernels and one ReLU activation function;

[0115] The second convolutional layer consists of eight 3×3 convolutional kernels and one ReLU activation function;

[0116] The third convolutional layer consists of three 3×3 convolutional kernels and a ReLU activation function.

[0117] It should be noted that the structure of the cascaded feature learning unit is consistent with the structure of the initial cascaded feature learning unit.

[0118] Based on multiple initial cascaded feature learning units, shared cascaded learning and cyclic decomposition are performed on the training image P to be enhanced and the training frequency domain features. Unsupervised loss is calculated and the illumination map and reflection map of the image decomposition are output. The reflection map is the second image P2. The output of the current initial cascaded feature learning unit is used as the input of the next initial cascaded feature learning unit.

[0119] The expression for the output of the initial cascaded feature learning unit is as follows:

[0120] y t =cycconv(u),if t=1,u=x,else u=y t-1

[0121] Among them, y ty represents the output of the t-th initial cascaded feature learning unit; t-1 represents the output of the (t-1)th initial cascaded feature learning unit; x represents the training image P to be enhanced; u represents the variable; cycconv(·) represents the shared-weight cascaded feature learning network function;

[0122] The expression for unsupervised loss is:

[0123] R = y t [;;0:3];

[0124] L = y t [;;3];

[0125]

[0126]

[0127]

[0128] Where R represents the reflection map; L represents the illumination map; Indicates the losses incurred during reconstruction; Indicates the loss of illumination smoothness; This represents the total loss of the initial lighting module; ε represents the gradient; ε represents the smallest positive constant.

[0129] Step C: Perform histogram equalization on the training image P to be enhanced, achieving the traditional low-light enhancement operation, to obtain the first image P1. Simultaneously, obtain the illumination map and reflection map of the image decomposition through a shared-weighted cascaded feature learning network; the reflection map is the second image P2.

[0130] Step D: Input the first image P1 and the second image P2 into the pre-trained Vgg16 network. Vgg16 extracts the low-level feature details from the two corresponding images to obtain the histogram prior loss, thus realizing the application of perceptual loss.

[0131] The first image P1 and the second image P2 are fed into the pre-trained network Vgg16. Deep features of the images are extracted through the Vgg16 convolutional network layers (relu1_2, relu2_2, relu3_2, relu4_2, relu5_2). The features obtained from the first image P1 and the second image P2 are compared to make the high-level information more similar. The formula is as follows:

[0132]

[0133] in, Indicates perceived loss; y represents the output image; y represents the input image; j represents the j-th layer of the network; C j Indicates the number of channels; H j Indicates feature height; W j Indicates the feature width; This indicates the features of the output image after processing by the VGG network; Represents the VGG network features of the input image;

[0134] Step E: Input the second image P2 into the initial noise generator E N (Initially) The noise characteristic distribution is constrained by KL distance, forcing the noise to be closer to a normal distribution. KL loss is calculated and noise information is output.

[0135] Because of the unpaired configuration, separating noise and content information in an image is not easy. To constrain the noise encoder to encode only noise information, this invention uses KL distance to limit the distribution of noise features, forcing the noise function to more closely approximate a standard normal distribution. The KL distance formula is as follows:

[0136]

[0137] Where q(z) n ) represents the distribution of noise characteristics, and p(z) represents the distribution of the standard normal distribution N(0,1).

[0138] KL difference loss will suppress noise features. n The information contained herein, minimizing KL differences to the greatest extent is equivalent to minimizing the following loss function.

[0139] The expression for KL loss is as follows:

[0140]

[0141] in, d represents the KL loss; d represents the dimension of the noise feature. The standard deviation of noise; represents the average value of the noise; log represents the logarithmic function.

[0142] Step F: Calculate the adversarial loss.

[0143] Input the sample image T into the initial second content encoder. (Initially), output sample features; input the sample features and noise information into the initial noise image generator G. X (Initial) Output the third image P3;

[0144] Input the second image P2 into the initial first content encoder. (Initially), output the second feature; input the second feature into the initial denoised image generator G. Y (Initial) Output the fourth image P4;

[0145] Calculate the adversarial loss based on the third image P3 and the fourth image P4.

[0146] The specific steps are as follows:

[0147] The second image P2, along with the unpaired training real image, i.e., the sample image T, is fed into the initial first content encoder with shared weights. (Initial) and Initial Second Content Encoder (junior high school.

[0148] Sample image T is passed through the initial second content encoder The features extracted in the initial step are fused with noise information, and then processed by the initial noise image generator G. X (Initial) Obtain the third image P3.

[0149] The second image P2 is passed through the initial first content encoder. The extracted features are processed by the initial denoising image generator G. Y (Initial) The fourth image P4 is obtained.

[0150] To make the enhanced image resemble a realistic, normally lit image, we employ adversarial loss to minimize the distance between the real image and the output distribution. This invention modifies the original least-squares GAN (LSGAN) loss of the discriminator. The expression for the adversarial loss is as follows:

[0151]

[0152] in, Indicates resistance to loss; Represents the true sample distribution; denoted by ; D(·) represents the discriminator; x represents the real sample; G(z) represents the generated sample; b represents the real sample label; a represents the generated sample label.

[0153] Step G: The steps for generating and denoising cyclic noise, and calculating the cyclic consistency loss.

[0154] Since denoised images may lose some content information without paired supervision, we introduce cycle consistency loss to ensure the accuracy of the generated content information. Steps E and F are repeated to reconvert the third image P3 back into a clean domain, while the denoised fourth image P4 is converted back into the original noisy sample. In the absence of paired training samples, the generator often loses some content information. To ensure that the generated noisy image can be converted back into a clean domain image without losing content information, this invention uses cycle consistency loss, as follows:

[0155]

[0156] in, Indicates the loss of cycle consistency; I rec represents the output sample; I represents the input sample; ||·||1 represents the norm modulo operation.

[0157] In addition to the cycle consistency loss, this invention also uses self-reconstruction loss to promote better quality generated images.

[0158] The formula for the loss function is as follows:

[0159]

[0160] Among them, I rec It is a sample generated by the generator only once, φ l This is for feature extraction operations of the pre-trained network Vgg19.

[0161] Figure 5 The following are examples of the results of processing using the method of the present invention. It can be clearly seen from the figure that the present invention restores the low-light image to a normal image, maintains the normal color, has no obvious noise, and restores the detail information of the image well.

[0162] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0163] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0164] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0165] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0166] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An unsupervised low-light enhancement method based on histogram equalization prior, characterized in that, Includes the following steps: Acquire the low-light image to be enhanced; The low-light image to be enhanced is input into a pre-constructed and trained unsupervised low-light enhancement model; wherein, the unsupervised low-light enhancement model includes an illumination enhancement module and a noise removal module, the illumination enhancement module includes a frequency domain feature extraction unit and a cascaded feature learning unit, and the noise removal module includes a first content encoder and a denoised image generator; The illumination enhancement module based on the unsupervised low-light enhancement model performs frequency domain feature extraction and cascaded feature learning on the low-light image to be enhanced to obtain a reflection image, including the following steps; The low-light image to be enhanced is input into the frequency domain feature extraction unit for frequency domain feature extraction, and the frequency domain features are output. The low-light image to be enhanced is input into the cascaded feature learning unit, and the frequency domain features are fused to perform cascaded feature learning to obtain the reflection image; The noise removal module based on the unsupervised low-light enhancement model performs content encoding and denoising on the reflected image to obtain the enhanced low-light image, including the following steps: The reflection image is input into a first content encoder for content encoding, and the reflection features are output. The reflection features are input into a denoising image generator for denoising processing, and an enhanced low-light image is output. The training method for the unsupervised low-light enhancement model includes the following steps: Obtain the training set and the pre-built initial model; The training set includes training images to be enhanced and sample images; The training set is input into the pre-built initial model for training, including the following steps: Perform histogram equalization on the training image to be enhanced to obtain the first image; The training image to be enhanced is input into a shared-weighted cascaded feature learning network, the unsupervised loss is calculated, and the second image is output. The first image and the second image are respectively input into the Vgg16 network to obtain the histogram prior loss; Based on the sample image and the second image, noise generation and denoising are performed using the initial noise removal module, and KL loss and adversarial loss are calculated. The cycle consistency loss is calculated by repeating the noise generation and denoising steps. Specifically, based on the unsupervised loss and histogram prior loss, the parameters of the initial frequency domain feature extraction unit and the initial cascaded feature learning unit are optimized to obtain the frequency domain feature extraction unit and cascaded feature learning unit of the unsupervised low-light enhancement model. Based on the KL loss, adversarial loss and cycle consistency loss, the parameters of the initial first content encoder and the initial denoised image generator are optimized to obtain the first content encoder and denoised image generator of the unsupervised low-light enhancement model. Based on multiple initial cascaded feature learning units, shared cascaded learning and cyclic decomposition are performed on the training image to be enhanced and the training frequency domain features, unsupervised loss is calculated and a second image is output; wherein, the output of the current initial cascaded feature learning unit is used as the input of the next initial cascaded feature learning unit; The expression for the output of the initial cascaded feature learning unit is as follows: ； in, Indicates the first The output of the initial cascaded feature learning units; Indicates the first The output of the initial cascaded feature learning units; This represents the training image to be enhanced; Represents variables; This represents a shared-weighted cascaded feature learning network function; The initial cascaded feature learning unit includes a first convolutional layer, a fusion sub-unit for fusing training frequency domain features, a second convolutional layer, and a third convolutional layer connected in sequence.

2. The unsupervised low-light enhancement method based on histogram equalization prior as described in claim 1, characterized in that, The training method for the unsupervised low-light enhancement model includes the following steps: The training set is input into the pre-built initial model for training to obtain a trained unsupervised low-light enhancement model; The initial model includes an initial illumination enhancement module and an initial noise removal module. The initial illumination enhancement module includes a shared weighted cascaded feature learning network composed of an initial frequency domain feature extraction unit and multiple initial cascaded feature learning units, as well as a Vgg16 network. The initial noise removal module includes an initial noise generator, an initial first content encoder, an initial second content encoder, an initial noisy image generator, and an initial denoised image generator.

3. The unsupervised low-light enhancement method based on histogram equalization prior as described in claim 1, characterized in that, Calculate the unsupervised loss and output the second image, including: The training image to be enhanced is input into the initial frequency domain feature extraction unit, and the training frequency domain features are output. The expression for the unsupervised loss is: ； in, Represents a reflection image; Represents a lighting diagram; Indicates the losses incurred during reconstruction; Indicates the loss of illumination smoothness; This represents the total loss of the initial lighting module; Represents the gradient; This represents the smallest positive constant.

4. The unsupervised low-light enhancement method based on histogram equalization prior as described in claim 3, characterized in that, The training image to be enhanced is input into the initial frequency domain feature extraction unit, and the training frequency domain features are output, including the following steps: Based on the fourth convolutional layer within the initial frequency domain feature extraction unit, feature processing is performed on the training image to be enhanced to obtain spatial domain features; The spatial domain features are transformed into frequency domain information through Fourier transform and FResNet function; The frequency domain information is transformed into training frequency domain features in the spatial domain through inverse Fourier transform. Output the trained frequency domain features; The four convolutional layers include The convolution kernel and LeakyReLU function.

5. The unsupervised low-light enhancement method based on histogram equalization prior as described in claim 3, characterized in that, The first convolutional layer includes eight A convolutional kernel and a ReLU activation function; The second convolutional layer includes eight A convolutional kernel and a ReLU activation function; The third convolutional layer includes three... The convolution kernel and a ReLU activation function.

6. The unsupervised low-light enhancement method based on histogram equalization prior as described in claim 1, characterized in that, Calculating the KL loss and adversarial loss involves the following steps: The second image is input into the initial noise generator, the KL loss is calculated and the noise information is output. The sample image is input into the initial second content encoder, which outputs sample features; the sample features and noise information are input into the initial noise image generator, which outputs a third image. The second image is input into the initial first content encoder, and the second feature is output. The second feature is input into the initial denoised image generator, and the fourth image is output. Calculate the adversarial loss based on the third and fourth images; The expression for the KL loss is as follows: ； in, Indicates KL's loss; Dimensions representing noise characteristics; The standard deviation of noise; This represents the average value of the noise. Represents a logarithmic function; The expression for the adversarial loss is as follows: ； in, Indicates resistance to loss; Represents the true sample distribution; Indicates the distribution of generated samples; Indicates the discriminator; Represents a real sample; Indicates the generation of samples; Indicates the labels of real samples; This indicates the label of the generated sample.

7. The unsupervised low-light enhancement method based on histogram equalization prior as described in claim 1, characterized in that, The expression for the cycle consistency loss is as follows: ； in, This represents the loss of cycle consistency. Indicates the output sample; Indicates the input sample; Modulo a norm.

Images