Image enhancement method, device, equipment and readable storage medium

Abstract

The present invention provides an image enhancement method, device, equipment and readable storage medium. The image enhancement method includes: performing illumination enhancement processing on each low-light image; taking each low-light image and its corresponding preliminary illumination enhancement image and normal illumination image as a training image pair; using multiple training image pairs to alternately train an enhancement generator network, a degradation generator network, an enhancement discriminator network and a degradation discriminator network to obtain a trained enhancement generator network. Through the present invention, the low-light image is subjected to preliminary illumination enhancement, a degradation generator network is added on the basis of the enhancement generator network, and the normal illumination image is subjected to reverse degradation learning training. Due to the mutual exclusion between the discriminator and the generator, the enhanced image and the degraded image are more and more similar to the corresponding real illumination image. Through the present invention, a higher quality normal illumination image can be generated.

Description

Technical Field

[0001] The present invention relates to the field of images, and in particular to an image enhancement method, device, equipment and readable storage medium. Background Art

[0002] Low-light image enhancement is one of the basic tasks in image processing and has wide applications in visual surveillance, autonomous driving, computational photography, and other fields. In the past few decades, many researchers have devoted themselves to solving the problem of low-light image enhancement. Many technologies have been developed to improve the subjective and objective quality of low-light images. With the rise of deep learning, the underlying visual tasks have developed rapidly. At present, deep learning frameworks have become an effective solution for low-light image enhancement.

[0003] Wei et al. designed a deep network called RetinexNet (Deep retinex decomposition for low-light enhancement), which combines Retinex theory with CNN to improve low-light images by optimizing the illumination component. Inspired by RetinexNet (Deep retinex decomposition for low-light enhancement), Zhang et al. improved the processing of the reflection component on this basis and proposed a simple and efficient network KinD (Kindling the darkness: A practical low-light image enhancer) to process both the illumination component and the reflection component. With the development of GAN networks, Rao et al. combined Retinex theory with GAN methods to propose a Component-GAN network (Seeing in the Dark by Component-GAN), using a discriminator to further enhance the image reconstruction effect. Although the method based on Retinex theory has achieved remarkable results in improving image contrast and noise reduction, it is difficult to maintain color naturalness, and the synthesized images usually tend to be cartoon-style. Jiang et al. proposed an unsupervised generative adversarial network EnlightenGAN (Enlightengan: Deep light enhancement without paired supervision) that does not require paired datasets, and made a series of innovations to the problem of low-light image enhancement, including a global-local discriminator structure. Chunle et al. proposed a new method that specifies illumination enhancement as a task of image-specific curve estimation using a deep network (Zero-reference deep curve estimation for low-light image enhancement), and it does not require any paired or unpaired data during training. Although the above unsupervised learning is more suitable for real-world scenarios, this learning method still makes it difficult to obtain high-precision low-light enhanced images. Summary of the invention

[0004] The main purpose of the present invention is to provide an image enhancement method, device, equipment and readable storage medium, aiming to solve the current technical problem of low-light image enhancement to generate normal-light image, resulting in low-quality image.

[0005] In a first aspect, the present invention provides an image enhancement method, the image enhancement method comprising:

[0006] Acquire a paired image dataset, wherein the paired image dataset includes multiple groups of image pairs, each group of image pairs includes a low-light image and a corresponding normal-light image;

[0007] Performing illumination enhancement processing on each low-light image to obtain a preliminary illumination enhancement image corresponding to each low-light image;

[0008] Taking each low-light image and its corresponding preliminary light-enhanced image and normal-light image as a training image pair, a plurality of training image pairs are obtained;

[0009] Using multiple training image pairs to alternately train an enhanced generator network, a degraded generator network, an enhanced discriminator network, and a degraded discriminator network to obtain a trained enhanced generator network;

[0010] The low-light image to be enhanced is input into the trained enhancement generator network to generate the enhanced image.

[0011] Optionally, before using a plurality of training image pairs to alternately train the enhanced generator network, the degraded generator network, the enhanced discriminator network, and the degraded discriminator network to obtain a trained enhanced generator network, the method further includes:

[0012] Construct and initialize an adversarial learning network, wherein the adversarial learning network includes a generator network and a discriminator network, wherein the generator network includes an enhanced generator network and a degraded generator network, and wherein the discriminator network includes an enhanced discriminator network and a degraded discriminator network.

[0013] Optionally, performing illumination enhancement processing on each low-light image to obtain a preliminary illumination enhanced image corresponding to each low-light image includes:

[0014] Decomposing each low-light image to obtain a first illumination map and a reflection map corresponding to each low-light image;

[0015] Selecting the maximum value of the three RGB channels of each low-light image as image brightness information, and increasing the brightness of each low-light image based on the image brightness information to obtain a second illumination map;

[0016] Optimizing the second illumination map using a structural prior and performing gamma correction to obtain a third illumination map;

[0017] The third illumination map and the reflection map are combined to obtain a preliminary illumination enhancement image corresponding to each low-light image.

[0018] Optionally, the step of alternately training an enhanced generator network, a degraded generator network, an enhanced discriminator network, and a degraded discriminator network using a plurality of training image pairs to obtain a trained enhanced generator network comprises:

[0019] Selecting a set of training image pairs from among a plurality of training image pairs;

[0020] Inputting the preliminary illumination enhanced image of the set of training image pairs into the enhancement generator network, and obtaining an enhanced normal illumination image through the enhancement generator of the enhancement generator network;

[0021] Performing image feature extraction based on the preliminary illumination enhanced image and the enhanced normal illumination image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a first loss function based on the positive samples, negative samples, and anchor points, and updating an enhancement generator network and a degradation generator network based on the first loss function;

[0022] Inputting the normal-light image of the set of training image pairs into the degradation generator network, and obtaining a degraded low-light image through a degradation generator of the degradation generator network;

[0023] Performing image feature extraction based on the normal-light image and the degraded low-light image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a second loss function based on the positive samples, negative samples, and anchor points, and updating an enhanced generator network and a degraded generator network based on the second loss function;

[0024] Determining the authenticity of the enhanced normal-lighting image and the normal-lighting image through an enhanced discriminator network, and updating the enhanced discriminator network based on the determination result;

[0025] Determining the authenticity of the low-light image and the degraded low-light image through a degradation discriminator network, and updating the degradation discriminator network based on the determination result;

[0026] Check whether the first loss function converges;

[0027] If the first loss function converges or not, a new set of training image pairs is selected from the multiple training image pairs, the new set of training image pairs is used as a set of training image pairs, and the step of inputting the preliminary illumination enhanced image of the set of training image pairs into the enhancement generator network, and obtaining the enhanced normal illumination image through the enhancement generator of the enhancement generator network is returned to be executed;

[0028] If the first loss function converges, the training is terminated and the latest enhanced generator network is used as the trained enhanced generator network.

[0029] Optionally, performing image feature extraction based on the preliminary illumination enhanced image and the enhanced normal illumination image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a first loss function based on the positive samples, negative samples, and anchor points, and updating the enhancement generator network and the degradation generator network based on the first loss function includes:

[0030] The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the enhancement generator are used as the first selected layer, and the preliminary illumination enhanced image and each layer of the first selected layer are respectively input into the mapping head to extract image features, so as to obtain the first four groups of enhanced image features, and image features are respectively selected from a certain position of the first four groups of enhanced image features as positive samples, and image features at other positions are respectively selected as negative samples;

[0031] The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the degradation generator are used as the second selected layers, and the enhanced normal illumination image and each layer of the second selected layer are respectively input into the mapping head to extract image features, so as to obtain the second batch of four groups of enhanced image features, and the image features of the positive samples at the same position as the first batch of four groups of enhanced image features are respectively selected from the second batch of four groups of enhanced image features as anchor points;

[0032] Combining each of the first four groups of enhanced image features with each of the second four groups of enhanced image features to obtain a third four groups of enhanced image features;

[0033] Calculating loss functions based on positive samples, negative samples and anchor points of each group of the third batch of four groups of enhanced image features, and accumulating the loss functions to obtain a first loss function;

[0034] The enhanced generator network and the degraded generator network are updated based on the first loss function.

[0035] Optionally, performing image feature extraction based on the normal-light image and the degraded low-light image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a second loss function based on the positive samples, negative samples, and anchor points, and updating the enhanced generator network and the degraded generator network based on the second loss function includes:

[0036] The 4th, 8th, 12th and 16th convolutional neural networks in the encoder of the degradation generator are used as the third selected layer, and the normal illumination image and each layer of the third selected layer are respectively transmitted to the mapping head for image feature extraction to obtain a fourth batch of four groups of enhanced image features, and image features are respectively selected from a certain position of the fourth batch of four groups of enhanced image features as positive samples, and image features at other positions are respectively selected as negative samples;

[0037] The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the enhancement generator are used as the fourth selected layer, the degraded low-light image and each layer of the fourth selected layer are respectively input into the mapping head for image feature extraction to obtain a fifth batch of four groups of enhanced image features, and image features of positive samples at the same position as the fourth batch of four groups of enhanced image features are respectively selected from the fifth batch of four groups of enhanced image features as anchor points;

[0038] Combining each of the fourth batch of four groups of enhanced image features with each of the fifth batch of four groups of enhanced image features to obtain a sixth batch of four groups of enhanced image features;

[0039] Calculating loss functions based on positive samples, negative samples and anchor points of each group of the fourth group of enhanced image features in the sixth batch respectively, and accumulating the loss functions to obtain a second loss function;

[0040] The enhanced generator network and the degraded generator network are updated based on the second loss function.

[0041] In a second aspect, the present invention further provides an image enhancement device, the image enhancement device comprising:

[0042] An acquisition module is used to acquire a paired image dataset, wherein the paired image dataset includes multiple groups of image pairs, each group of image pairs includes a low-light image and a corresponding normal-light image;

[0043] A preliminary enhancement module, used to perform illumination enhancement processing on each low-light image to obtain a preliminary illumination enhancement image corresponding to each low-light image;

[0044] A pairing module, used to use each low-light image and its corresponding preliminary light-enhanced image and normal-light image as a training image pair to obtain a plurality of training image pairs;

[0045] A training module, used for alternately training an enhanced generator network, a degraded generator network, an enhanced discriminator network and a degraded discriminator network using a plurality of training image pairs to obtain a trained enhanced generator network;

[0046] The generation module is used to input the low-light image to be enhanced into the trained enhancement generator network to generate an enhanced image.

[0047] Optionally, the training module is used to:

[0048] Selecting a set of training image pairs from among a plurality of training image pairs;

[0049] Inputting the preliminary illumination enhanced image of the set of training image pairs into the enhancement generator network, and obtaining an enhanced normal illumination image through the enhancement generator of the enhancement generator network;

[0050] Performing image feature extraction based on the preliminary illumination enhanced image and the enhanced normal illumination image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a first loss function based on the positive samples, negative samples, and anchor points, and updating an enhancement generator network and a degradation generator network based on the first loss function;

[0051] Inputting the normal-light image of the set of training image pairs into the degradation generator network, and obtaining a degraded low-light image through a degradation generator of the degradation generator network;

[0052] Performing image feature extraction based on the normal-light image and the degraded low-light image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a second loss function based on the positive samples, negative samples, and anchor points, and updating an enhanced generator network and a degraded generator network based on the second loss function;

[0053] Determining the authenticity of the enhanced normal-lighting image and the normal-lighting image through an enhanced discriminator network, and updating the enhanced discriminator network based on the determination result;

[0054] Determining the authenticity of the low-light image and the degraded low-light image through a degradation discriminator network, and updating the degradation discriminator network based on the determination result;

[0055] Check whether the first loss function converges;

[0056] If the first loss function converges or not, a new set of training image pairs is selected from the multiple training image pairs, the new set of training image pairs is used as a set of training image pairs, and the step of inputting the preliminary illumination enhanced image of the set of training image pairs into the enhancement generator network, and obtaining the enhanced normal illumination image through the enhancement generator of the enhancement generator network is returned to be executed;

[0057] If the first loss function converges, the training is terminated and the latest enhanced generator network is used as the trained enhanced generator network.

[0058] In a third aspect, the present invention also provides an image enhancement device, comprising a processor, a memory, and an image enhancement program stored in the memory and executable by the processor, wherein when the image enhancement program is executed by the processor, the steps of the image enhancement method described above are implemented.

[0059] In a fourth aspect, the present invention further provides a readable storage medium having an image enhancement program stored thereon, wherein when the image enhancement program is executed by a processor, the steps of the image enhancement method as described above are implemented.

[0060] In the present invention, a paired image data set is obtained, wherein the paired image data set includes multiple groups of image pairs, each group of image pairs includes a low-light image and a corresponding normal-light image; each low-light image is subjected to light enhancement processing to obtain a preliminary light-enhanced image corresponding to each low-light image; each low-light image and its corresponding preliminary light-enhanced image and the normal-light image are used as a training image pair to obtain multiple training image pairs; an enhancement generator network, a degradation generator network, an enhancement discriminator network and a degradation discriminator network are alternately trained using the multiple training image pairs to obtain a trained enhancement generator network; and the low-light image to be enhanced is input into the trained enhancement generator network to generate an enhanced image. According to the present invention, a preliminary illumination enhancement process is first performed on a low-light image to obtain a preliminary illumination enhancement image, and enhancement learning training is performed on the preliminary illumination enhancement image. Secondly, a degradation generator network is added on the basis of the enhancement generator network, and reverse degradation learning training is performed on the normal illumination image. Due to the mutual exclusion between the discriminator and the generator, both the enhanced image and the degraded image are increasingly similar to the corresponding real illumination image. The present invention consolidates the image generation training process by using a dual learning mechanism, and can make the parameters of the generator network more optimized, so that the low-light image can be generated into a normal illumination image with higher quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of the hardware structure of an image enhancement device according to an embodiment of the present invention;

[0062] Figure 2 A schematic diagram of a flow chart of an embodiment of an image enhancement method of the present invention;

[0063] Figure 3 FIG. 4 is a schematic diagram of functional modules of an image enhancement device according to an embodiment of the present invention.

[0064] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0065] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0066] In a first aspect, an embodiment of the present invention provides an image enhancement device, which may be a device having a data processing function, such as a personal computer (PC), a notebook computer, or a server.

[0067] Reference Figure 1 , Figure 1 The hardware structure diagram of an embodiment of the image enhancement device of the present invention. In the embodiment of the present invention, the image enhancement device may include a processor 1001 (e.g., a central processing unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components; the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard); the network interface 1004 may optionally include a standard wired interface, a wireless interface (such as a wireless fidelity WIreless-FIdelity, WI-FI interface); the memory 1005 may be a high-speed random access memory (random access memory, RAM), or a stable memory (non-volatile memory), such as a disk storage, and the memory 1005 may optionally be a storage device independent of the aforementioned processor 1001. Those skilled in the art will understand that Figure 1 The hardware structure shown in the figure does not constitute a limitation of the present invention, and may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0068] Continue to refer to Figure 1 , Figure 1 The memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module and an image enhancement program. The processor 1001 may call the image enhancement program stored in the memory 1005 and execute the image enhancement method provided in the embodiment of the present invention.

[0069] In a second aspect, an embodiment of the present invention provides an image enhancement method.

[0070] In order to more clearly demonstrate the image enhancement method provided in the embodiment of the present application, the application scenario of the image enhancement method provided in the embodiment of the present application is first introduced.

[0071] Shooting under adverse weather and lighting conditions, such as cloudy, foggy, low-light, backlight, etc., will cause the captured images to exhibit image degradation problems such as low contrast, poor clarity, and high noise. By using the image enhancement method provided in the embodiment of the present application, the captured images can be enhanced to obtain high-quality normal-light images. Low-light image enhancement is one of the basic tasks of image processing. The image enhancement method provided in the embodiment of the present application is widely used in visual surveillance, image recognition, autonomous driving, computational photography and other fields.

[0072] In one embodiment, referring to Figure 2 , Figure 2 FIG. 1 is a flow chart of an embodiment of an image enhancement method of the present invention. Figure 2 As shown, the image enhancement method includes:

[0073] Step S10: obtaining a paired image dataset, wherein the paired image dataset includes a plurality of image pairs, each image pair including a low-light image and a corresponding normal-light image.

[0074] In this embodiment, the paired image dataset adopts the LOL public dataset, which is a dataset from real scenes containing 500 pairs of low-light / normal-light image pairs.

[0075] Step S20, performing illumination enhancement processing on each low-light image to obtain a preliminary illumination enhanced image corresponding to each low-light image.

[0076] In this embodiment, illumination enhancement processing is performed on each low-light image, with the purpose of obtaining a preliminary illumination enhancement image corresponding to each low-light image, which is in an intermediate state between low-light and normal-light.

[0077] Step S30 , taking each low-light image and its corresponding preliminary light-enhanced image and normal-light image as a training image pair, to obtain a plurality of training image pairs.

[0078] In this embodiment, each low-light image and its corresponding preliminary lighting enhanced image and normal lighting image are used as a training image pair, so that each of the obtained multiple training image pairs contains a low-light image, a preliminary lighting enhanced image, and a normal lighting image of the same scene for subsequent learning and training.

[0079] Step S40, using multiple training image pairs to alternately train the enhanced generator network, the degraded generator network, the enhanced discriminator network and the degraded discriminator network to obtain a trained enhanced generator network.

[0080] In this embodiment, multiple training images are used to alternately train the enhanced generator network, the degraded generator network, the enhanced discriminator network and the degraded discriminator network. This is an iterative training process. This embodiment involves two ways of ending the training. One is to end the training when the loss function converges. The convergence of the loss function indicates that the noise loss of the generated image becomes smaller and tends to be stable. The other is to preset the number of training times according to needs. When the number of iterations reaches the preset number of training times, the training is ended to obtain a trained enhanced generator network.

[0081] Step S50: input the low-light image to be enhanced into the trained enhancement generator network to generate an enhanced image.

[0082] In this embodiment, a trained enhancement generator network is used to generate a low-light image to be enhanced into a corresponding normal-light image.

[0083] In this embodiment, 485 pairs of images are selected from 500 pairs of LOL public data sets as training data sets, and the remaining 15 pairs of images are used as test data sets. All images are converted into PNG format and adjusted to 256×256 pixels. The comparison results of the experimental results obtained using the image enhancement method of the present invention and other low-light image enhancement algorithms under the same conditions are shown in Table 1. Table 1 is an example of the comparison results of the experimental results of the present invention and other low-light image enhancement algorithms.

[0084] Table 1.

[0085] Algorithm LIME RetinexNet KinD EnlightenGAN Zero-DCE The present invention PSNR 16.08 20.12 15.99 23.05 19.83 24.05 SSIM 0.78 0.77 0.74 0.90 0.86 0.86 FSIM 0.91 0.84 0.90 0.94 0.92 0.90

[0086] The PSNR, SSIM and FSIM in Table 1 are all average values, where PSNR is the peak signal-to-noise ratio, which is the abbreviation of Peak Signal to Noise Ratio; SSIM is the abbreviation of Structural Similarity, which is an indicator for measuring the similarity between two images; FSIM uses feature similarity for quality evaluation, which is the abbreviation of feature similarity index mersure.

[0087] It can be seen from Table 1 that the image enhancement method of the present invention obtains a higher score in PSNR than the comparison algorithm and is superior to the comparison algorithm.

[0088] Furthermore, in one embodiment, before step S40, the method further includes:

[0089] Construct and initialize an adversarial learning network, wherein the adversarial learning network includes a generator network, an enhanced discriminator network, and a degraded discriminator network, wherein the generator network includes an enhanced generator network and a degraded generator network, and the discriminator network includes an enhanced discriminator network and a degraded discriminator network.

[0090] In this embodiment, the adversarial learning network constructed and initialized is different from the existing adversarial learning network in that a degraded generator network is added on the basis of the enhanced generator network, and the discriminator network is divided into an enhanced discriminator network and a degraded discriminator network.

[0091] Furthermore, in one embodiment, step S20 includes:

[0092] Decomposing each low-light image to obtain a first illumination map and a reflection map corresponding to each low-light image;

[0093] Selecting the maximum value of the three RGB channels of each low-light image as image brightness information, and increasing the brightness of each low-light image based on the image brightness information to obtain a second illumination map;

[0094] Optimizing the second illumination map using a structural prior and performing gamma correction to obtain a third illumination map;

[0095] The third illumination map and the reflection map are combined to obtain a preliminary illumination enhancement image corresponding to each low-light image.

[0096] In this embodiment, based on the Retinex theoretical method, an image can be decomposed into an illumination map and a reflection map. The low-light image is decomposed into a first illumination map and a reflection map. The maximum value of the three RGB channels of the low-light image is found and selected as the image brightness information to enhance the brightness of the low-light image to obtain a brightened second illumination map. The second illumination map is subjected to structural prior optimization processing and gamma correction to obtain a third illumination map. The third illumination map and the reflection map are then merged to obtain a preliminary illumination enhanced image. Through this processing method, the texture, contour and other structural features of the image can be better preserved.

[0097] Furthermore, in one embodiment, step S40 includes:

[0098] Step S401, selecting a set of training image pairs from a plurality of training image pairs;

[0099] Step S402, inputting the preliminary illumination enhanced image of the set of training image pairs into the enhancement generator network, and obtaining an enhanced normal illumination image through the enhancement generator of the enhancement generator network;

[0100] Step S403, performing image feature extraction based on the preliminary illumination enhanced image and the enhanced normal illumination image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a first loss function based on the positive samples, negative samples, and anchor points, and updating an enhancement generator network and a degradation generator network based on the first loss function;

[0101] Step S404, inputting the normal light image of the set of training image pairs into the degradation generator network, and obtaining a degraded low light image through the degradation generator of the degradation generator network;

[0102] Step S405, performing image feature extraction based on the normal-light image and the degraded low-light image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a second loss function based on the positive samples, negative samples, and anchor points, and updating an enhanced generator network and a degraded generator network based on the second loss function;

[0103] Step S406, judging the authenticity between the enhanced normal illumination image and the normal illumination image through an enhanced discriminator network, and updating the enhanced discriminator network based on the judgment result;

[0104] Step S407, judging the authenticity of the low-light image and the degraded low-light image through a degradation discriminator network, and updating the degradation discriminator network based on the judgment result;

[0105] Step S408, detecting whether the first loss function converges;

[0106] Step S409, if the first loss function converges or not, a new set of training image pairs is selected from the multiple training image pairs, the new set of training image pairs is used as a set of training image pairs, and the step of inputting the preliminary illumination enhanced image of the set of training image pairs into the enhancement generator network, and obtaining the enhanced normal illumination image through the enhancement generator of the enhancement generator network is returned to execute;

[0107] Step S410: If the first loss function converges, the training is terminated and the latest enhanced generator network is used as the trained enhanced generator network.

[0108] In this embodiment, a group of training image pairs includes a low-light image, a preliminary light-enhanced image, and a normal-light image of the same scene. The preliminary light-enhanced image is obtained by an enhancement generator to obtain an enhanced normal-light image. Image features are extracted based on the preliminary light-enhanced image and the enhanced normal-light image, and then positive samples, negative samples, and anchor points are selected therefrom. The image noise loss is calculated using the positive samples, negative samples, and anchor points to obtain a first loss function. The parameters returned by the first loss function are used to optimize the parameters of the enhancement generator network and the degradation generator network.

[0109] Similarly, in this embodiment, there is a reverse degradation learning process, that is, the normal lighting image is passed through the degradation generator to obtain a degraded low-light image, image features are extracted based on the normal lighting image and the degraded low-light image, and then positive samples, negative samples, and anchor points are selected therefrom, and the image noise loss is calculated through the positive samples, negative samples, and anchor points to obtain a second loss function, and the parameters returned by the second loss function are used to optimize the parameters of the enhancement generator network and the degradation generator network.

[0110] In this embodiment, the enhancement discriminator network judges the authenticity between the enhanced normal-light image and the normal-light image, and optimizes the parameters of the enhancement discriminator network based on the returned results. Similarly, the degradation discriminator network judges the authenticity between the low-light image and the degraded low-light image, and optimizes the parameters of the degradation discriminator network based on the returned results.

[0111] In this embodiment, whether the first loss function converges is used as a condition for terminating the training of the adversarial learning network because the generator network is ultimately used to generate images. If the first loss function does not reach convergence, a new set of training image pairs is selected, and the process returns to the step of inputting the preliminary illumination enhanced image into the enhanced generator network in step S40 of this embodiment to perform a new round of iterative training. If the first loss function reaches convergence, the training process is terminated, and the latest enhanced generator network is used as the trained enhanced generator network.

[0112] Furthermore, in one embodiment, step S403 includes:

[0113] The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the enhancement generator are used as the first selected layer, and the preliminary illumination enhanced image and each layer of the first selected layer are respectively input into the mapping head to extract image features, so as to obtain the first four groups of enhanced image features, and image features are respectively selected from a certain position of the first four groups of enhanced image features as positive samples, and image features at other positions are respectively selected as negative samples;

[0114] The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the degradation generator are used as the second selected layers, and the enhanced normal illumination image and each layer of the second selected layer are respectively input into the mapping head to extract image features, so as to obtain the second batch of four groups of enhanced image features, and the image features of the positive samples at the same position as the first batch of four groups of enhanced image features are respectively selected from the second batch of four groups of enhanced image features as anchor points;

[0115] Combining each of the first four groups of enhanced image features with each of the second four groups of enhanced image features to obtain a third four groups of enhanced image features;

[0116] Calculating loss functions based on positive samples, negative samples and anchor points of each group of the third batch of four groups of enhanced image features, and accumulating the loss functions to obtain a first loss function;

[0117] The enhanced generator network and the degraded generator network are updated based on the first loss function.

[0118] In this embodiment, both the enhancement generator network and the degradation generator network adopt a network based on ResNet, and the preliminary illumination enhancement image is trained together with the enhanced normal illumination image. The 4th, 8th, 12th, and 16th convolutional neural networks in the encoder of the degradation generator are used as the corresponding layers of the 4th, 8th, 12th, and 16th convolutional neural networks in the encoder of the enhancement generator. For each layer, the mapping head is used to extract image features and then select positive samples, negative samples, and anchor points. The purpose of image feature extraction is to make the features of the obtained image more obvious. The mapping head is composed of two layers of MLP (Multi-layer Perceptron). The mapping head is used to better extract image features. The purpose of selecting positive samples, negative samples, and anchor points is to calibrate learning samples to achieve better learning purposes. In this embodiment, the positive samples, negative samples, and anchor points in the four batches of image features are calculated using loss functions to evaluate image noise loss, and the parameters returned by the loss function are used to optimize the parameters of the enhancement generator network and the degradation generator network.

[0119] Furthermore, in one embodiment, step S405 includes:

[0120] The 4th, 8th, 12th and 16th convolutional neural networks in the encoder of the degradation generator are used as the third selected layer, and the normal illumination image and each layer of the third selected layer are respectively transmitted to the mapping head for image feature extraction to obtain a fourth batch of four groups of enhanced image features, and image features are respectively selected from a certain position of the fourth batch of four groups of enhanced image features as positive samples, and image features at other positions are respectively selected as negative samples;

[0121] The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the enhancement generator are used as the fourth selected layer, the degraded low-light image and each layer of the fourth selected layer are respectively input into the mapping head for image feature extraction to obtain a fifth batch of four groups of enhanced image features, and image features of positive samples at the same position as the fourth batch of four groups of enhanced image features are respectively selected from the fifth batch of four groups of enhanced image features as anchor points;

[0122] Combining each of the fourth batch of four groups of enhanced image features with each of the fifth batch of four groups of enhanced image features to obtain a sixth batch of four groups of enhanced image features;

[0123] Calculating loss functions based on positive samples, negative samples and anchor points of each group of the fourth group of enhanced image features in the sixth batch respectively, and accumulating the loss functions to obtain a second loss function;

[0124] The enhanced generator network and the degraded generator network are updated based on the second loss function.

[0125] In this embodiment, this is a reverse degradation learning process, in which the normal light image is trained together with the degraded low light image, and the 4th, 8th, 12th, and 16th convolutional neural networks in the encoder of the enhancement generator are used as the corresponding layers of the 4th, 8th, 12th, and 16th convolutional neural networks in the encoder of the degradation generator. For each layer, the mapping head is used to extract image features and then select positive samples, negative samples, and anchor points. The purpose of image feature extraction is to make the features of the obtained image more obvious. The mapping head is composed of two layers of MLP (Multi-layer Perceptron). The mapping head is used to better extract image features. The purpose of selecting positive samples, negative samples, and anchor points is to calibrate the learning samples to achieve better learning purposes. In this embodiment, the positive samples, negative samples, and anchor points in the four batches of image features are calculated using loss functions to evaluate the image noise loss, and the parameters returned by the loss function are used to optimize the parameters of the enhancement generator network and the degradation generator network.

[0126] In a third aspect, an embodiment of the present invention further provides an image enhancement device.

[0127] Reference Figure 3 , Figure 3 FIG. 4 is a schematic diagram of functional modules of an image enhancement device according to an embodiment of the present invention.

[0128] In this embodiment, the image enhancement device includes:

[0129] An acquisition module 10 is used to acquire a paired image dataset, wherein the paired image dataset includes a plurality of image pairs, each image pair includes a low-light image and a corresponding normal-light image;

[0130] A preliminary enhancement module 20, configured to perform illumination enhancement processing on each low-light image to obtain a preliminary illumination enhancement image corresponding to each low-light image;

[0131] A pairing module 30, configured to use each low-light image and its corresponding preliminary light-enhanced image and normal-light image as a training image pair to obtain a plurality of training image pairs;

[0132] A training module 40 is used to alternately train an enhanced generator network, a degraded generator network, an enhanced discriminator network, and a degraded discriminator network using a plurality of training image pairs to obtain a trained enhanced generator network;

[0133] The generation module 50 is used to input the low-light image to be enhanced into the trained enhancement generator network to generate an enhanced image.

[0134] Furthermore, in one embodiment, the image enhancement device further includes a construction module, which is used to:

[0135] Construct and initialize an adversarial learning network, wherein the adversarial learning network includes a generator network and a discriminator network, wherein the generator network includes an enhanced generator network and a degraded generator network, and wherein the discriminator network includes an enhanced discriminator network and a degraded discriminator network.

[0136] Furthermore, in one embodiment, the preliminary enhancement module 20 is used to:

[0137] Decomposing each low-light image to obtain a first illumination map and a reflection map corresponding to each low-light image;

[0138] Selecting the maximum value of the three RGB channels of each low-light image as image brightness information, and increasing the brightness of each low-light image based on the image brightness information to obtain a second illumination map;

[0139] Optimizing the second illumination map using a structural prior and performing gamma correction to obtain a third illumination map;

[0140] The third illumination map and the reflection map are combined to obtain a preliminary illumination enhancement image corresponding to each low-light image.

[0141] Furthermore, in one embodiment, the training module 40 is used to:

[0142] Selecting a set of training image pairs from among a plurality of training image pairs;

[0143] Inputting the preliminary illumination enhanced image of the set of training image pairs into the enhancement generator network, and obtaining an enhanced normal illumination image through the enhancement generator of the enhancement generator network;

[0144] Performing image feature extraction based on the preliminary illumination enhanced image and the enhanced normal illumination image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a first loss function based on the positive samples, negative samples, and anchor points, and updating an enhancement generator network and a degradation generator network based on the first loss function;

[0145] Inputting the normal-light image of the set of training image pairs into the degradation generator network, and obtaining a degraded low-light image through a degradation generator of the degradation generator network;

[0146] Performing image feature extraction based on the normal-light image and the degraded low-light image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a second loss function based on the positive samples, negative samples, and anchor points, and updating an enhanced generator network and a degraded generator network based on the second loss function;

[0147] Determining the authenticity of the enhanced normal-lighting image and the normal-lighting image through an enhanced discriminator network, and updating the enhanced discriminator network based on the determination result;

[0148] Determining the authenticity of the low-light image and the degraded low-light image through a degradation discriminator network, and updating the degradation discriminator network based on the determination result;

[0149] Check whether the first loss function converges;

[0150] If the first loss function converges or not, a new set of training image pairs is selected from the multiple training image pairs, the new set of training image pairs is used as a set of training image pairs, and the step of inputting the preliminary illumination enhanced image of the set of training image pairs into the enhancement generator network, and obtaining the enhanced normal illumination image through the enhancement generator of the enhancement generator network is returned to be executed;

[0151] If the first loss function converges, the training is terminated and the latest enhanced generator network is used as the trained enhanced generator network.

[0152] Further, in one embodiment, the training module 40 extracts image features based on the preliminary illumination enhanced image and the enhanced normal illumination image, selects positive samples, negative samples, and anchor points from the extracted image features, calculates a first loss function based on the positive samples, negative samples, and anchor points, and updates the enhancement generator network and the degradation generator network based on the first loss function, for:

[0153] The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the enhancement generator are used as the first selected layer, and the preliminary illumination enhanced image and each layer of the first selected layer are respectively input into the mapping head to extract image features, so as to obtain the first four groups of enhanced image features, and image features are respectively selected from a certain position of the first four groups of enhanced image features as positive samples, and image features at other positions are respectively selected as negative samples;

[0154] The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the degradation generator are used as the second selected layers, and the enhanced normal illumination image and each layer of the second selected layer are respectively input into the mapping head to extract image features, so as to obtain the second batch of four groups of enhanced image features, and the image features of the positive samples at the same position as the first batch of four groups of enhanced image features are respectively selected from the second batch of four groups of enhanced image features as anchor points;

[0155] Combining each of the first four groups of enhanced image features with each of the second four groups of enhanced image features to obtain a third four groups of enhanced image features;

[0156] Calculating loss functions based on positive samples, negative samples and anchor points of each group of the third batch of four groups of enhanced image features, and accumulating the loss functions to obtain a first loss function;

[0157] The enhanced generator network and the degraded generator network are updated based on the first loss function.

[0158] Further, in one embodiment, the training module 40 extracts image features based on the normal illumination image and the degraded low illumination image, selects positive samples, negative samples, and anchor points from the extracted image features, calculates a second loss function based on the positive samples, negative samples, and anchor points, and updates the enhanced generator network and the degraded generator network based on the second loss function, for:

[0159] The 4th, 8th, 12th and 16th convolutional neural networks in the encoder of the degradation generator are used as the third selected layer, and the normal illumination image and each layer of the third selected layer are respectively transmitted to the mapping head for image feature extraction to obtain a fourth batch of four groups of enhanced image features, and image features are respectively selected from a certain position of the fourth batch of four groups of enhanced image features as positive samples, and image features at other positions are respectively selected as negative samples;

[0160] The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the enhancement generator are used as the fourth selected layer, the degraded low-light image and each layer of the fourth selected layer are respectively input into the mapping head for image feature extraction to obtain a fifth batch of four groups of enhanced image features, and image features of positive samples at the same position as the fourth batch of four groups of enhanced image features are respectively selected from the fifth batch of four groups of enhanced image features as anchor points;

[0161] Combining each of the fourth batch of four groups of enhanced image features with each of the fifth batch of four groups of enhanced image features to obtain a sixth batch of four groups of enhanced image features;

[0162] Calculating loss functions based on positive samples, negative samples and anchor points of each group of the fourth group of enhanced image features in the sixth batch respectively, and accumulating the loss functions to obtain a second loss function;

[0163] The enhanced generator network and the degraded generator network are updated based on the second loss function.

[0164] Among them, the functional implementation of each module in the above-mentioned image enhancement device corresponds to each step in the above-mentioned image enhancement method embodiment, and its functions and implementation processes are no longer repeated here.

[0165] In a fourth aspect, an embodiment of the present invention further provides a readable storage medium.

[0166] The readable storage medium of the present invention stores an image enhancement program, wherein when the image enhancement program is executed by a processor, the steps of the above-mentioned image enhancement method are implemented.

[0167] The method implemented when the image enhancement program is executed can refer to the various embodiments of the image enhancement method of the present invention, and will not be described in detail here.

[0168] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0169] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0170] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present invention.

[0171] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image enhancement method, characterized in that: The image enhancement method comprises: Acquire a paired image dataset, wherein the paired image dataset includes multiple groups of image pairs, each group of image pairs includes a low-light image and a corresponding normal-light image; Performing illumination enhancement processing on each low-light image to obtain a preliminary illumination enhancement image corresponding to each low-light image; Taking each low-light image and its corresponding preliminary light-enhanced image and normal-light image as a training image pair, a plurality of training image pairs are obtained; Using multiple training image pairs to alternately train an enhanced generator network, a degraded generator network, an enhanced discriminator network, and a degraded discriminator network to obtain a trained enhanced generator network; Input the low-light image to be enhanced into the trained enhancement generator network to generate an enhanced image; The method uses a plurality of training images to alternately train an enhanced generator network, a degraded generator network, an enhanced discriminator network, and a degraded discriminator network to obtain a trained enhanced generator network, including: Selecting a set of training image pairs from among a plurality of training image pairs; Inputting the preliminary illumination enhanced image of the set of training image pairs into the enhancement generator network, and obtaining an enhanced normal illumination image through the enhancement generator of the enhancement generator network; Performing image feature extraction based on the preliminary illumination enhanced image and the enhanced normal illumination image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a first loss function based on the positive samples, negative samples, and anchor points, and updating an enhancement generator network and a degradation generator network based on the first loss function; Inputting the normal-light image of the set of training image pairs into a degradation generator network, and obtaining a degraded low-light image through a degradation generator of the degradation generator network; Performing image feature extraction based on the normal-light image and the degraded low-light image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a second loss function based on the positive samples, negative samples, and anchor points, and updating an enhanced generator network and a degraded generator network based on the second loss function; Judging the authenticity between the enhanced normal-lighting image and the normal-lighting image through an enhanced discriminator network, and updating the enhanced discriminator network based on the judgment result; Determining the authenticity of the low-light image and the degraded low-light image through a degradation discriminator network, and updating the degradation discriminator network based on the determination result; Check whether the first loss function converges; If the first loss function converges or not, a new set of training image pairs is selected from the multiple training image pairs, the new set of training image pairs is used as a set of training image pairs, and the step of inputting the preliminary illumination enhanced image of the set of training image pairs into the enhancement generator network, and obtaining the enhanced normal illumination image through the enhancement generator of the enhancement generator network is returned to be executed; If the first loss function converges, the training is terminated and the latest enhanced generator network is used as the trained enhanced generator network; The step of extracting image features based on the preliminary illumination enhanced image and the enhanced normal illumination image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a first loss function based on the positive samples, negative samples, and anchor points, and updating an enhancement generator network and a degradation generator network based on the first loss function includes: The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the enhancement generator are used as the first selected layer, and the preliminary illumination enhanced image and each layer of the first selected layer are respectively input into the mapping head to extract image features, so as to obtain the first four groups of enhanced image features, and image features are respectively selected from a certain position of the first four groups of enhanced image features as positive samples, and image features at other positions are respectively selected as negative samples; The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the degradation generator are used as the second selected layers, and the enhanced normal illumination image and each layer of the second selected layer are respectively input into the mapping head to extract image features, so as to obtain the second batch of four groups of enhanced image features, and the image features of the positive samples at the same position as the first batch of four groups of enhanced image features are respectively selected from the second batch of four groups of enhanced image features as anchor points; Combining each of the first four groups of enhanced image features with each of the second four groups of enhanced image features to obtain a third four groups of enhanced image features; Calculating loss functions based on positive samples, negative samples and anchor points of each group of the third batch of four groups of enhanced image features, and accumulating the loss functions to obtain a first loss function; Update the enhanced generator network and the degraded generator network based on the first loss function; The step of extracting image features based on the normal-light image and the degraded low-light image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a second loss function based on the positive samples, negative samples, and anchor points, and updating the enhanced generator network and the degraded generator network based on the second loss function includes: The 4th, 8th, 12th and 16th convolutional neural networks in the encoder of the degradation generator are used as the third selected layer, and the normal illumination image and each layer of the third selected layer are respectively transmitted to the mapping head for image feature extraction to obtain a fourth batch of four groups of enhanced image features, and image features are respectively selected from a certain position of the fourth batch of four groups of enhanced image features as positive samples, and image features at other positions are respectively selected as negative samples; The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the enhancement generator are used as the fourth selected layer, the degraded low-light image and each layer of the fourth selected layer are respectively input into the mapping head for image feature extraction to obtain a fifth batch of four groups of enhanced image features, and image features of positive samples at the same position as the fourth batch of four groups of enhanced image features are respectively selected from the fifth batch of four groups of enhanced image features as anchor points; Combining each of the fourth batch of four groups of enhanced image features with each of the fifth batch of four groups of enhanced image features to obtain a sixth batch of four groups of enhanced image features; Calculating loss functions based on positive samples, negative samples and anchor points of each group of the fourth group of enhanced image features in the sixth batch respectively, and accumulating the loss functions to obtain a second loss function; The enhanced generator network and the degraded generator network are updated based on the second loss function.

2. The image enhancement method according to claim 1, characterized in that: Before the method uses a plurality of training images to alternately train the enhanced generator network, the degraded generator network, the enhanced discriminator network and the degraded discriminator network to obtain a trained enhanced generator network, the method further includes: Construct and initialize an adversarial learning network, wherein the adversarial learning network includes a generator network and a discriminator network, wherein the generator network includes an enhanced generator network and a degraded generator network, and wherein the discriminator network includes an enhanced discriminator network and a degraded discriminator network.

3. The image enhancement method according to claim 1, characterized in that: The performing illumination enhancement processing on each low-light image to obtain a preliminary illumination enhancement image corresponding to each low-light image comprises: Decomposing each low-light image to obtain a first illumination map and a reflection map corresponding to each low-light image; Selecting the maximum value of the three RGB channels of each low-light image as image brightness information, and increasing the brightness of each low-light image based on the image brightness information to obtain a second illumination map; Optimizing the second illumination map using a structural prior and performing gamma correction to obtain a third illumination map; The third illumination map and the reflection map are combined to obtain a preliminary illumination enhancement image corresponding to each low-light image.

4. An image enhancement device, characterized in that: The image enhancement device comprises: An acquisition module is used to acquire a paired image dataset, wherein the paired image dataset includes multiple groups of image pairs, each group of image pairs includes a low-light image and a corresponding normal-light image; A preliminary enhancement module, used to perform illumination enhancement processing on each low-light image to obtain a preliminary illumination enhancement image corresponding to each low-light image; A pairing module, used to use each low-light image and its corresponding preliminary light-enhanced image and normal-light image as a training image pair to obtain a plurality of training image pairs; A training module, used for alternately training an enhanced generator network, a degraded generator network, an enhanced discriminator network and a degraded discriminator network using a plurality of training image pairs to obtain a trained enhanced generator network; A generation module, used for inputting the low-light image to be enhanced into the trained enhancement generator network to generate an enhanced image; The training module is used to: Selecting a set of training image pairs from among a plurality of training image pairs; Inputting the preliminary illumination enhanced image of the set of training image pairs into the enhancement generator network, and obtaining an enhanced normal illumination image through the enhancement generator of the enhancement generator network; Performing image feature extraction based on the preliminary illumination enhanced image and the enhanced normal illumination image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a first loss function based on the positive samples, negative samples, and anchor points, and updating an enhancement generator network and a degradation generator network based on the first loss function; Inputting the normal-light image of the set of training image pairs into a degradation generator network, and obtaining a degraded low-light image through a degradation generator of the degradation generator network; Performing image feature extraction based on the normal-light image and the degraded low-light image, selecting positive samples, negative samples, and anchor points from the extracted image features, calculating a second loss function based on the positive samples, negative samples, and anchor points, and updating an enhanced generator network and a degraded generator network based on the second loss function; Judging the authenticity between the enhanced normal-lighting image and the normal-lighting image through an enhanced discriminator network, and updating the enhanced discriminator network based on the judgment result; Determining the authenticity of the low-light image and the degraded low-light image through a degradation discriminator network, and updating the degradation discriminator network based on the determination result; Check whether the first loss function converges; If the first loss function converges or not, a new set of training image pairs is selected from the multiple training image pairs, the new set of training image pairs is used as a set of training image pairs, and the step of inputting the preliminary illumination enhanced image of the set of training image pairs into the enhancement generator network, and obtaining the enhanced normal illumination image through the enhancement generator of the enhancement generator network is returned to be executed; If the first loss function converges, the training is terminated and the latest enhanced generator network is used as the trained enhanced generator network; The training module extracts image features based on the preliminary illumination enhanced image and the enhanced normal illumination image, selects positive samples, negative samples, and anchor points from the extracted image features, calculates a first loss function based on the positive samples, negative samples, and anchor points, and updates an enhancement generator network and a degradation generator network based on the first loss function, for: The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the enhancement generator are used as the first selected layer, and the preliminary illumination enhanced image and each layer of the first selected layer are respectively input into the mapping head to extract image features, so as to obtain the first four groups of enhanced image features, and image features are respectively selected from a certain position of the first four groups of enhanced image features as positive samples, and image features at other positions are respectively selected as negative samples; The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the degradation generator are used as the second selected layers, and the enhanced normal illumination image and each layer of the second selected layer are respectively input into the mapping head to extract image features, so as to obtain the second batch of four groups of enhanced image features, and the image features of the positive samples at the same position as the first batch of four groups of enhanced image features are respectively selected from the second batch of four groups of enhanced image features as anchor points; Combining each of the first four groups of enhanced image features with each of the second four groups of enhanced image features to obtain a third four groups of enhanced image features; Calculating loss functions based on positive samples, negative samples and anchor points of each group of the third batch of four groups of enhanced image features, and accumulating the loss functions to obtain a first loss function; Update the enhanced generator network and the degraded generator network based on the first loss function; The training module extracts image features based on the normal illumination image and the degraded low illumination image, selects positive samples, negative samples, and anchor points from the extracted image features, calculates a second loss function based on the positive samples, negative samples, and anchor points, and updates the enhanced generator network and the degraded generator network based on the second loss function, for: The 4th, 8th, 12th and 16th convolutional neural networks in the encoder of the degradation generator are used as the third selected layer, and the normal illumination image and each layer of the third selected layer are respectively transmitted to the mapping head for image feature extraction to obtain a fourth batch of four groups of enhanced image features, and image features are respectively selected from a certain position of the fourth batch of four groups of enhanced image features as positive samples, and image features at other positions are respectively selected as negative samples; The 4th, 8th, 12th and 16th convolutional neural network layers in the encoder of the enhancement generator are used as the fourth selected layer, the degraded low-light image and each layer of the fourth selected layer are respectively input into the mapping head for image feature extraction to obtain a fifth batch of four groups of enhanced image features, and image features of positive samples at the same position as the fourth batch of four groups of enhanced image features are respectively selected from the fifth batch of four groups of enhanced image features as anchor points; Combining each of the fourth batch of four groups of enhanced image features with each of the fifth batch of four groups of enhanced image features to obtain a sixth batch of four groups of enhanced image features; Calculating loss functions based on positive samples, negative samples and anchor points of each group of the fourth group of enhanced image features in the sixth batch respectively, and accumulating the loss functions to obtain a second loss function; The enhanced generator network and the degraded generator network are updated based on the second loss function.

5. An image enhancement device, characterized in that: The image enhancement device comprises a processor, a memory, and an image enhancement program stored in the memory and executable by the processor, wherein when the image enhancement program is executed by the processor, the steps of the image enhancement method according to any one of claims 1 to 3 are implemented.

6. A readable storage medium, characterized in that: The readable storage medium stores an image enhancement program, wherein when the image enhancement program is executed by a processor, the steps of the image enhancement method according to any one of claims 1 to 3 are implemented.

Images