Unsupervised low-light image enhancement method based on compressed auto-encoding
By combining compressed autoencoders and lightweight convolutional neural networks, the problems of insufficient reference data and high computational resource consumption in low-light image enhancement are solved, achieving efficient and accurate image enhancement results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUILIN UNIV OF ELECTRONIC TECH
- Filing Date
- 2023-05-05
- Publication Date
- 2026-06-02
AI Technical Summary
Existing low-light image enhancement algorithms require reference data to train the model, and the images generated by Generative Adversarial Networks (GANs) are distorted and noisy, making the training process time-consuming and computationally resource-intensive.
An unsupervised low-light image enhancement method based on compressed autoencoders is adopted. Image features are learned by generative adversarial networks (GANs), and lightweight convolutional neural networks are combined to improve training efficiency and generalization ability, while reducing image distortion and noise.
It improves the effect of low-light image enhancement, reduces training time and computational resource requirements, and generates images with high similarity to the original images, demonstrating strong generalization ability.
Smart Images
Figure CN116823636B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of computer vision and machine learning, and specifically relates to an unsupervised low-light image enhancement method based on compressed autoencoders. Background Technology
[0002] Digital image processing is a method of processing images using mathematics and technology. Early digital image processing typically employed techniques such as image transformation, image coding, image enhancement, and image restoration to improve the visual quality of images, making them more consistent with human vision. Since the 1960s, digital image processing technology has experienced unprecedented growth, leading to its widespread application in scientific research, medical diagnosis, security monitoring, and communications engineering. Simultaneously, low-light image enhancement algorithms have gradually become a research focus in the field of image processing. Low-light image enhancement algorithms primarily work by transforming poorly lit images into brighter, more contrasting images to recover valuable information hidden in dark areas.
[0003] Low-light images, typically taken at night or under the cover of buildings, suffer from problems such as blurriness, severe noise, and loss of detail. While existing algorithms address these issues to some extent, they still have limitations. Most deep learning-based low-light enhancement algorithms usually require training the model on reference data to help the neural network learn image features, improving its enhancement capabilities and generalization. However, collecting reference data in low-light environments is often difficult and time-consuming. Meanwhile, Generative Adversarial Networks (GANs) consist of a generator and a discriminator. Since the generator can only generate data from random noise, the generated images or data are distorted and noisy. This noise often leads to significant differences between the generator's data and the real data. Furthermore, existing GANs are typically complex models, requiring substantial time and computational resources for training. Summary of the Invention
[0004] This invention provides an unsupervised low-light image enhancement method based on compressed autoencoders, aiming to at least address one of the technical problems existing in the prior art. The method enhances low-light images based on image similarity measurement and learning. First, the low-light image is preprocessed, including denoising and image enhancement operations. Then, a gradient-based similarity measurement method is used to compare the similarity between the enhanced image and the original image, thereby enhancing the low-light image. While unsupervised low-light enhancement algorithms can overcome the problem of insufficient reference data to some extent, their enhancement effect is still limited by the quality of the original image. To improve the performance of unsupervised low-light enhancement algorithms, Generative Adversarial Networks (GANs) are used to learn image features. A GAN is a deep learning model consisting of a generator and a discriminator. The generator attempts to generate realistic images, and the discriminator attempts to distinguish between real and generated images. By training the GAN, features of real images can be learned and used to enhance low-light images.
[0005] The technical solution of this invention relates to an unsupervised low-light image enhancement method and apparatus based on compression autoencoder. The image enhancement method is used to input a low-light image into a generative adversarial network (GAN) for image enhancement. The GAN includes a generator and a discriminator connected in sequence. The method includes the following steps:
[0006] S100. Initialize the network parameters of the generator and the discriminator;
[0007] S200: The generator acquires a low-light image, extracts a noise vector from the low-light image, and generates a fake image.
[0008] S300, the discriminator acquires the first false image and the normal illumination image, determines whether the first false image is a real data image or a false data image, and outputs a judgment result. The normal illumination image is a normal illumination image that is unrelated to the low illumination image and the first false image.
[0009] S400, The generator obtains the first judgment result, adjusts the generator's generation strategy, and obtains a new strategy, the new strategy being to make it more difficult for the discriminator to distinguish between the first fake image and the normal illumination image;
[0010] S500, Repeat steps S200 to S400 until the discriminator can hardly distinguish a fake image generated by the generator;
[0011] S600, The generator obtains the first fake image it generated and generates a second fake image;
[0012] S700, The discriminator acquires the secondary false image and the normal illumination image, determines whether the secondary false image is a real data image or a false data image, and outputs the secondary judgment result;
[0013] S800, The generator obtains the secondary judgment result, adjusts the generator's generation strategy, and obtains a new secondary strategy, which makes it more difficult for the discriminator to distinguish between the secondary false image and the normal illumination image.
[0014] S900, Repeat steps S600 to S800 until the discriminator can hardly distinguish the secondary fake images generated by the generator.
[0015] Furthermore, the generator network includes an encoder, a hidden layer, and a decoder connected in sequence, wherein,
[0016] The encoder maps the input low-light image to the hidden layer, as follows:
[0017] h = f(x) = N f (Wx+b h ),
[0018] Where, N f W is a non-linear activation function, which can be either the Sigmoid function or the ReLU function, where W is a dimensionless activation function. h *d x The weight matrix, It is the bias vector;
[0019] The decoder reconstructs a primary or secondary fake image from the output of the hidden layer, as follows:
[0020] y = g(h) = N g (W'h+b y ),
[0021] Where, N g W' is a non-linear activation function, which can be either the Sigmoid function or the ReLU function, where W' is a dimensionless activation function. x *d h The weight matrix, This is the bias vector.
[0022] Furthermore, the generator network is a compressed autoencoder network, and the process of the compressed autoencoder network is as follows:
[0023] ,
[0024] Among them, Γ CAE (θ) is the compression autoencoder, x is the input data, and D nLet x be the training set, g(f(x)) be the output graph data, and L be the loss function, which is used to calculate the minimum value between x and f(x). J is the regularization term, λ is the regularization parameter, and J is the regularization term. f (x) is the Jacobian matrix.
[0025] further,
[0026] ,
[0027] Among them, J f (x) is the Jacobian matrix, h j (x) is the hidden layer. Let be the first-order partial derivative of the hidden layer, and i and j be the pixel values at different positions in the hidden layer.
[0028] Further, step S200 includes:
[0029] S210: Reduce the brightness of the low-light image to obtain a darkened image;
[0030] S220: Add noise to the darkened image.
[0031] Furthermore, step S210 is represented as follows:
[0032] I Low =B*I In β ,
[0033] Where B is the coefficient of image brightness, B is a constant, β is an adaptive factor, β<1, I In For the original input image, I Low This is the image after the brightness has been reduced.
[0034] Furthermore, step S220 is represented as follows:
[0035] I Noise =N(G(I) Low )),
[0036] Where N(~) is the noise function, G(~) is the gamma adjustment function, and I Low To darken the image, I Noise The image after noise has been added.
[0037] Furthermore, in steps S400 and S800, adjusting the generation strategy of the generator includes:
[0038] The generation strategy of the generator is as follows:
[0039] z t =Gen(x) t ,at-1 ,θ g ),
[0040] Among them, z t x is the random variable currently generated by the generator. t For the input sample, a t-1 Let θ be the generator's decision vector at the previous time step. g These are the parameters of the generator;
[0041] The strategy of the discriminator is as follows:
[0042] y t =Disc(z t ,a t-1 ,θ d ),
[0043] Among them, y t z is the predicted label currently generated by the discriminator. t a is the random variable currently generated by the generator. t-1 Let θ be the generator's decision vector at the previous time step. d These are the parameters of the discriminator;
[0044] The generator updates its decision vector based on the first or second judgment result of the discriminator, as follows:
[0045] a t =arg max a'P(a'|z t ,x t ,at-1,θ g ),
[0046] Among them, a t Let P(a'|z) be the generator's decision vector at the current time step. t ,x t ,at-1,θ g ) is the generator that generates random variable z based on the current value. t and the current input sample x t Predict the probability distribution of the next decision vector a';
[0047] The discriminator updates its prediction result based on the generator's generation result:
[0048] y t+1 =arg max x y P(y|z t ,x t ,a t ,θ d ),
[0049] Among them, y t+1Let P(y|z) be the predicted label of the discriminator at the next time step. t ,x t ,a t ,θ d The discriminator is based on the currently generated random variable z. t and the current input sample x t Predict the probability distribution of the next label y.
[0050] Furthermore, the discriminator network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, and a fifth convolutional layer connected in sequence. The input of the fourth convolutional layer is connected to the output of the second convolutional layer, and the input of the fifth convolutional layer is connected to the output of the third convolutional layer.
[0051] The first, second, third, and fourth convolutional layers all have 3×3 kernels, while the fifth convolutional layer has a 1×1 kernel.
[0052] The first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, and the fifth convolutional layer all include an activation function, which is the PReLU activation function.
[0053] The present invention also proposes a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, perform the method described above.
[0054] According to some embodiments of the present invention, the beneficial effects of the present invention are as follows:
[0055] The unsupervised low-light image enhancement method proposed in this invention improves the training efficiency and generalization ability of generative adversarial networks (GANs) by using compressed autoencoders, thereby reducing image distortion and noise and enhancing the performance of unsupervised low-light enhancement algorithms.
[0056] Furthermore, additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0057] Figure 1 This is a flowchart of the unsupervised low-light image enhancement method based on compression autoencoder.
[0058] Figure 2 The flowchart shows how a generator of an unsupervised low-light image enhancement method based on compression autoencoder generates a fake image.
[0059] Figure 3 This is a structural diagram of the image enhancement network for an unsupervised low-light image enhancement method based on compression autoencoders.
[0060] Figure 4 This is a schematic diagram of the generator autoencoder network for an unsupervised low-light image enhancement method based on compression autoencoders.
[0061] Figure 5 This is a diagram of the discriminator network structure for an unsupervised low-light image enhancement method based on compression autoencoders. Detailed Implementation
[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0063] The following will provide a clear and complete description of the concept, specific structure, and technical effects of the present invention in conjunction with the embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of the present invention.
[0064] It should be noted that, unless otherwise specified, when a feature is referred to as "fixed" or "connected" to another feature, it can be directly fixed or connected to the other feature, or indirectly fixed or connected to the other feature. The singular forms "a," "described," and "the" used herein are also intended to include the plural forms, unless the context clearly indicates otherwise. Furthermore, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing particular embodiments only and not for limiting the invention. The term "and / or" as used herein includes any combination of one or more of the associated listed items.
[0065] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish elements of the same type from each other. For example, a first element may also be referred to as a second element without departing from the scope of this disclosure, and similarly, a second element may also be referred to as a first element. Any and all instances or exemplary language (“e.g.,” “such as,” etc.) used herein are intended only to better illustrate embodiments of the invention and, unless otherwise required, do not impose a limitation on the scope of the invention. Furthermore, the industry term “pose” as used herein refers to the position and orientation of an element relative to a spatial coordinate system.
[0066] Reference Figures 1 to 5This invention provides an unsupervised low-light image enhancement method and apparatus based on compressed autoencoders. The image enhancement method involves inputting a low-light image into a generative adversarial network (GAN) for image enhancement. The GAN includes a generator and a discriminator connected sequentially. (Refer to...) Figure 1 The method includes the following steps:
[0067] S100. Initialize the network parameters of the generator and the discriminator;
[0068] S200: The generator acquires a low-light image, extracts a noise vector from the low-light image, and generates a fake image.
[0069] S300, the discriminator acquires the first false image and the normal illumination image, determines whether the first false image is a real data image or a false data image, and outputs a judgment result. The normal illumination image is a normal illumination image that is unrelated to the low illumination image and the first false image.
[0070] S400, The generator obtains the first judgment result, adjusts the generator's generation strategy, and obtains a new strategy, the new strategy being to make it more difficult for the discriminator to distinguish between the first fake image and the normal illumination image;
[0071] S500, Repeat steps S200 to S400 until the discriminator can hardly distinguish a fake image generated by the generator;
[0072] S600, The generator obtains the first fake image it generated and generates a second fake image;
[0073] S700, The discriminator acquires the secondary false image and the normal illumination image, determines whether the secondary false image is a real data image or a false data image, and outputs the secondary judgment result;
[0074] S800, The generator obtains the secondary judgment result, adjusts the generator's generation strategy, and obtains a new secondary strategy, which makes it more difficult for the discriminator to distinguish between the secondary false image and the normal illumination image.
[0075] S900, Repeat steps S600 to S800 until the discriminator can hardly distinguish the secondary fake images generated by the generator.
[0076] By comparing the effects of existing algorithms and their combined reconstructions on low-light nighttime image processing, this paper demonstrates that applying a compressed autoencoder to the generator of a Generative Adversarial Network (GAN) can further improve the training efficiency and generalization ability of GANs while reducing image distortion and noise. Encoding the input data using a compressed autoencoder compresses a large amount of data into a small number of vector representations, thus reducing the learning task of the GAN. Furthermore, decoding the generated image using a compressed autoencoder ensures a high similarity between the generated and original images, thereby enhancing the generalization ability of the GAN.
[0077] Generative Adversarial Networks (GANs) are network frameworks consisting of a generator network and a discriminator network. The generator network's primary function is to generate initial results based on the input data. The discriminator network's main task is to determine whether the input data is real or generated. The output of the generator network is fed to the discriminator network so that it can be trained to distinguish between real and generated data. Specifically, both the generator and discriminator networks in a GAN are deep neural networks. The generator network typically consists of multiple hidden layers, each containing many neurons. These neurons map the input data into a high-dimensional space so that the generator network can generate results similar to the input data. The main goal of the generator network is to learn the features of the input data to generate more realistic results. Therefore, we use compressed autoencoders as the generator part of the GAN.
[0078] Furthermore, by incorporating Generative Adversarial Networks (GANs) as part of the discriminator, we optimize the discriminator to address feature redundancy and reduce the number of parameters. In traditional GANs, the discriminator consists of multiple convolutional and fully connected layers, resulting in a large number of parameters and impacting performance and training speed. Therefore, by leveraging the advantages of GANs, we propose a lightweight convolutional neural network GAN model that can capture image features such as edges, textures, and shapes—essential for image classification and recognition tasks. This model utilizes residual networks (ResNet) to reduce parameter count and improve performance.
[0079] In Generative Adversarial Networks (GANs), the generator and discriminator networks are trained using the backpropagation algorithm. During training, the generator and discriminator networks interact to continuously improve their performance. The generator network improves its generation ability by continuously learning the features of real data, while the discriminator network continuously improves its judgment ability to distinguish between real and generated data. Therefore, a lightweight convolutional neural network is used as the discriminator in the GAN.
[0080] Furthermore, refer to Figure 3 and Figure 4 The generator network includes an encoder, a hidden layer, and a decoder connected in sequence, wherein the encoder maps the input low-light image to the hidden layer, as shown below:
[0081] h = f(x) = N f (Wx+b h ),
[0082] Where, N f W is a non-linear activation function, which can be either the Sigmoid function or the ReLU function, where W is a dimensionless activation function. h *d x The weight matrix, It is the bias vector;
[0083] The decoder reconstructs a primary or secondary fake image from the output of the hidden layer, as follows:
[0084] y = g(h) = N g (W'h+b y ),
[0085] Where, N g W' is a non-linear activation function, which can be either the Sigmoid function or the ReLU function, where W' is a dimensionless activation function. x *d h The weight matrix, This is the bias vector.
[0086] An autoencoder is a neural network with three or more layers. It consists of an encoder and a decoder. The encoder transforms the input signal X into an encoded signal Y, while the decoder transforms the encoded Y into an output signal X'.
[0087] Furthermore, refer to Figure 3 and Figure 4 The generator network is a compressed autoencoder network, and the process of the compressed autoencoder network is as follows:
[0088] ,
[0089] Among them, Γ CAE (θ) is the compression autoencoder, x is the input data, and D n Let x be the training set, g(f(x)) be the output graph data, and L be the loss function, which is used to calculate the minimum value between x and f(x). J is the regularization term, λ is the regularization parameter, and J is the regularization term. f (x) is the Jacobian matrix.
[0090] Furthermore, refer to Figure 3 and Figure 4 ,
[0091] ,
[0092] Among them, J f (x) is the Jacobian matrix, h j (x) is the hidden layer. Let be the first-order partial derivative of the hidden layer, and i and j be the pixel values at different positions in the hidden layer.
[0093] Compressed autoencoders (CAs) extract deep image features using an encoder-decoder approach, enabling unbiased reconstruction of input data. The CA introduces a Jacobian matrix into the loss function to ensure strong anti-interference capabilities. The proposed unsupervised low-light image enhancement method based on CA employs a multi-layered CA to form the generator within a GAN network, enhancing low-light images in nighttime scenes. Combining this with the strong feature extraction capabilities of CNNs, a lightweight network model is constructed as the discriminator within the GAN, thus addressing the computational resource consumption issue of deep network models.
[0094] Furthermore, refer to Figure 1 and Figure 2 Step S200 includes:
[0095] S210: Reduce the brightness of the low-light image to obtain a darkened image;
[0096] S220: Add noise to the darkened image.
[0097] Furthermore, refer to Figure 2 Step S210 is represented as follows:
[0098] I Low =B*I In β ,
[0099] Where B is the coefficient of image brightness, B is a constant, β is an adaptive factor, β<1, β<1 makes the image darker, β=1 makes the image unchanged, β>1 makes the image brighter, I In For the original input image, I LowThis is the image after the brightness has been reduced.
[0100] Furthermore, refer to Figure 2 Step S220 is represented as follows:
[0101] I Noise =N(G(I) Low )),
[0102] Where N(~) is the noise function, G(~) is the gamma adjustment function, and I Low To darken the image, I Noise The image after noise has been added.
[0103] Furthermore, refer to Figure 1 In steps S400 and S800, adjusting the generation strategy of the generator includes:
[0104] The generation strategy of the generator is as follows:
[0105] z t =Gen(x) t ,a t-1 ,θ g ),
[0106] Among them, z t x is the random variable currently generated by the generator. t For the input sample, a t-1 Let θ be the generator's decision vector at the previous time step. g These are the parameters of the generator;
[0107] The strategy of the discriminator is as follows:
[0108] y t =Disc(z t ,a t-1 ,θ d ),
[0109] Among them, y t z is the predicted label currently generated by the discriminator. t a is the random variable currently generated by the generator. t-1 Let θ be the generator's decision vector at the previous time step. d These are the parameters of the discriminator;
[0110] The generator updates its decision vector based on the first or second judgment result of the discriminator, as follows:
[0111] a t =arg max a'P(a'|z t ,x t ,at-1,θ g),
[0112] Among them, a t Let P(a'|z) be the generator's decision vector at the current time step. t ,x t ,at-1,θ g ) is the generator that generates random variable z based on the current value. t and the current input sample x t Predict the probability distribution of the next decision vector a';
[0113] The discriminator updates its prediction result based on the generator's generation result:
[0114] y t+1 =arg max x y P(y|z t ,x t ,a t ,θ d ),
[0115] Among them, y t+1 Let P(y|z) be the predicted label of the discriminator at the next time step. t ,x t ,a t ,θ d The discriminator is based on the currently generated random variable z. t and the current input sample x t Predict the probability distribution of the next label y.
[0116] Furthermore, refer to Figure 3 and Figure 5 The discriminator network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, and a fifth convolutional layer connected in sequence. The input of the fourth convolutional layer is connected to the output of the second convolutional layer, and the input of the fifth convolutional layer is connected to the output of the third convolutional layer.
[0117] The first, second, third, and fourth convolutional layers all have 3×3 kernels, while the fifth convolutional layer has a 1×1 kernel.
[0118] The first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, and the fifth convolutional layer all include an activation function, which is the PReLU activation function.
[0119] Specifically, in one particular embodiment, the size of the input image is 384×384.
[0120] The discriminator network consists of five convolutional layers:
[0121] First convolutional layer: 3×3 convolution, input image In size is 384×384, input image channel is 3, padding is 1, output channel is 64, then PReLu activation function is used, output Out_1;
[0122] The second convolutional layer is a 3×3 convolution with Out_1 as the input, 64 input channels, 1 padding channel, and 64 output channels. It then uses the PReLu activation function to output Out_2.
[0123] The third convolutional layer is a 3×3 convolution with 2 input channels (Out_2), 64 input channels, 1 padding channel, and 64 output channels. It then uses the PReLu activation function to output Out_3.
[0124] The fourth convolutional layer is a 3×3 convolution. The input is Out_2 and Out_3, which are concatenated by channels. The input channel is 128, the padding is 1, and the output channel is 64. Then, the PReLu activation function is used to output Out_4.
[0125] The fifth convolutional layer is a 1×1 convolution. The inputs are Out_3 and Out_4, which are concatenated by channels. The input channel has 128 channels and the padding is 1. The output channel has 3 channels. The PReLu activation function is then used to output Out_5.
[0126] The present invention also proposes a computer-readable storage medium comprising a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to implement the above-described unsupervised low-light image enhancement method based on compression autoencoder.
[0127] According to some embodiments of the present invention, the beneficial effects of the present invention are as follows:
[0128] The unsupervised low-light image enhancement method proposed in this invention improves the training efficiency and generalization ability of generative adversarial networks (GANs) by using compressed autoencoders, thereby reducing image distortion and noise and enhancing the performance of unsupervised low-light enhancement algorithms.
[0129] Specifically, the unsupervised low-light image enhancement method based on compressed autoencoders is a key innovation of this invention, which improves the training efficiency and generalization ability of generative adversarial networks (GANs) by using compressed autoencoders. In GANs, the generator's task is to generate realistic images from random noise. However, due to the massive amount of image data, the generator's training process requires significant computational resources and time. To address this issue, this invention proposes applying compressed autoencoders to the generator of a GAN. A compressed autoencoder is a neural network that compresses input data into a small number of vector representations and uses these vectors to reconstruct the data. This technique significantly reduces the amount of data representation, thereby reducing training time and computational resource requirements. Using compressed autoencoders in GANs can compress large amounts of image data into a small number of vector representations, thus alleviating the generator's training task. Simultaneously, by decoding the generated images using compressed autoencoders, a high similarity between the generated and original images can be ensured, thereby improving the generalization ability of the GAN. This technique reduces the distortion of the original image by the generator, resulting in more realistic generated images.
[0130] Furthermore, while deep neural network models demonstrate excellent performance in low-light image enhancement, limitations in storage space and power consumption necessitate exploring lightweight convolutional neural networks for this purpose. By optimizing the network structure and convolutional computation, and combining this with the strong feature extraction capabilities of generative adversarial networks (GANs), a lightweight network model is constructed to address the computational resource consumption issue of deep network models and reduce the number of network parameters.
[0131] In traditional Generative Adversarial Networks (GANs), the discriminator consists of multiple convolutional and fully connected layers. These layers have a large number of parameters, resulting in a very large parameter set, which negatively impacts model performance and training speed. To address this issue, this invention combines a GAN model with a lightweight convolutional neural network (LCNN) to replace the traditional discriminator, thereby reducing the number of parameters and improving model performance and training speed.
[0132] Lightweight Convolutional Neural Networks (LCNNs) are a more efficient convolutional neural network architecture compared to traditional convolutional neural networks. LCNNs use a set of lightweight convolutional layers instead of the convolutional and pooling layers found in traditional convolutional neural networks. These lightweight convolutional layers use small-sized kernels and can dynamically select the features to be extracted. This approach effectively reduces the number of model parameters and computational cost, thereby improving model performance and training speed.
[0133] It should be understood that the method steps in the embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable storage medium. The method can use standard programming techniques. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if necessary, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. Furthermore, for this purpose, the program can run on a programmed application-specific integrated circuit (ASIC).
[0134] Furthermore, the procedures described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by the context. The procedures described herein (or variations and / or combinations thereof) may be executed under the control of one or more computer systems configured with executable instructions, and may be implemented by hardware or a combination thereof as code (e.g., executable instructions, one or more computer programs, or one or more applications) that commonly executes on one or more processors. The computer program comprises a plurality of instructions executable by one or more processors.
[0135] Furthermore, the method can be implemented in any suitable type of computing platform, including but not limited to personal computers, minicomputers, mainframes, workstations, networked or distributed computing environments, standalone or integrated computer platforms, or in communication with charged particle tools or other imaging devices, etc. Aspects of the invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optical read and / or write storage medium, RAM, ROM, etc., such that it is readable by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein. Furthermore, the machine-readable code, or portions thereof, can be transmitted via wired or wireless networks. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media comprises instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. When programmed according to the methods and techniques described in the invention, the invention may also include the computer itself.
[0136] A computer program can be applied to input data to perform the functions described herein, thereby transforming the input data to generate output data stored in non-volatile memory. The output information can also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including specific visual depictions of physical and tangible objects generated on the display.
[0137] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention, as long as they achieve the technical effects of the present invention by the same means, should be included within the scope of protection of the present invention. Within the scope of protection of the present invention, the technical solutions and / or implementation methods can have various modifications and variations.
Claims
1. An image enhancement method, characterized in that, The image enhancement method described above is used to input a low-light image into a generative adversarial network (GAN) for image enhancement. The GAN includes a generator and a discriminator connected in sequence. The method includes the following steps: S100. Initialize the network parameters of the generator and the discriminator; S200: The generator acquires a low-light image, extracts a noise vector from the low-light image, and generates a fake image. S300, the discriminator acquires the first false image and the normal illumination image, determines whether the first false image is a real data image or a false data image, and outputs a judgment result. The normal illumination image is a normal illumination image that is unrelated to the low illumination image and the first false image. S400, The generator obtains the first judgment result, adjusts the generator's generation strategy, and obtains a new strategy, the new strategy being to make it more difficult for the discriminator to distinguish between the first fake image and the normal illumination image; S500, Repeat steps S200 to S400 until the discriminator can hardly distinguish a fake image generated by the generator; S600, The generator obtains the first fake image it generated and generates a second fake image; S700, The discriminator acquires the secondary false image and the normal illumination image, determines whether the secondary false image is a real data image or a false data image, and outputs the secondary judgment result; S800, The generator obtains the secondary judgment result, adjusts the generator's generation strategy, and obtains a new secondary strategy, which makes it more difficult for the discriminator to distinguish between the secondary false image and the normal illumination image. S900, Repeat steps S600 to S800 until the discriminator can hardly distinguish the secondary fake images generated by the generator; The generator comprises an encoder, a hidden layer, and a decoder connected in sequence, wherein, The encoder maps the input low-light image to the hidden layer, as follows: ， in, It is a non-linear activation function, which can be either the Sigmoid function or the ReLU function. For dimension The weight matrix, It is the bias vector; The decoder reconstructs a primary or secondary fake image from the output of the hidden layer, as follows: ， in, It is a non-linear activation function, which can be either the Sigmoid function or the ReLU function. For dimension The weight matrix, This is the bias vector.
2. The image enhancement method according to claim 1, characterized in that, The generator is a compressed autoencoder network, and the process of the compressed autoencoder network is as follows: ， in, For compression autoencoders, For input data, For the sample training set, To output graph data, For loss function, Used for calculation and The minimum value between For regularization terms, For regularization parameters, It is a Jacobian matrix.
3. The image enhancement method according to claim 2, characterized in that, ， in, For Jacobian matrices, As a hidden layer, Let be the first-order partial derivative of the hidden layer, and i and j be the pixel values at different positions in the hidden layer.
4. The image enhancement method according to claim 1, characterized in that, Step S200 includes: S210: Reduce the brightness of the low-light image to obtain a darkened image; S220: Add noise to the darkened image.
5. The image enhancement method according to claim 4, characterized in that, Step S210 is represented as follows: ， Where B is the coefficient of image brightness, and B is a constant. As an adaptive factor, <1, The original input image, This is the image after the brightness has been reduced.
6. The image enhancement method according to claim 4, characterized in that, Step S220 is represented as follows: ， Where N(~) is the noise function and G(~) is the gamma adjustment function. The image after dimming. The image after noise has been added.
7. The image enhancement method according to claim 1, characterized in that, In steps S400 and S800, adjusting the generation strategy of the generator includes: The generation strategy of the generator is as follows: ， in, For the random variable currently generated by the generator, For the input sample, Let be the generator's decision vector at the previous time step. These are the parameters of the generator; The strategy of the discriminator is as follows: ， in, The predicted label currently generated by the discriminator. For the random variable currently generated by the generator, Let be the generator's decision vector at the previous time step. These are the parameters of the discriminator; The generator updates its decision vector based on the first or second judgment result of the discriminator, as follows: ， in, Let this be the generator's decision vector at the current moment. The generator generates random variables based on the current value. and the current input sample Predict the next decision vector The probability distribution; The discriminator updates its prediction result based on the generator's generation result: ， in, The discriminator's predicted label at the next time step. The discriminator generates random variables based on the current data. and the current input sample Predict the next label The probability distribution.
8. The image enhancement method according to claim 1, characterized in that, The discriminator network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, and a fifth convolutional layer connected in sequence. The input of the fourth convolutional layer is connected to the output of the second convolutional layer, and the input of the fifth convolutional layer is connected to the output of the third convolutional layer. The first, second, third, and fourth convolutional layers all have 3×3 kernels, while the fifth convolutional layer has a 1×1 kernel. The first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, and the fifth convolutional layer all include an activation function, which is the PReLU activation function.
9. A computer-readable storage medium having stored thereon program instructions that, when executed by a processor, perform the method as described in any one of claims 1 to 8.