Image enhancement method and device, electronic equipment and storage medium

Abstract

The application provides an image enhancement method and device, electronic equipment and storage medium. A single low-light image to be processed is acquired. The single low-light image to be processed is input into an image enhancement network. The image enhancement network is trained by defining a regularization loss of prior guidance to a zero-shot learning enhancer. An expected exposure image output by the image enhancement network for the single low-light image to be processed is acquired. The application proposes a flexible and controllable zero-shot learning enhancer, which can learn an expected exposure image only by using a single low-light image itself and without relying on statistical characteristics of training data. Therefore, the zero-shot learning enhancer has good generalization ability for a real scene. Correspondingly, an image enhancement network based on the zero-shot learning enhancer can learn an expected exposure image only by inputting a single low-light image for training.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and more specifically, to an image enhancement method, apparatus, electronic device, and storage medium. Background Technology

[0002] Images captured in low-light environments often suffer from severe visual degradation, such as low brightness, color distortion, and unclear texture. To address this issue, mainstream research employs supervised learning for low-light image enhancement, with these models trained on carefully collected datasets containing a large number of paired low-light / normal-light images. While these methods have achieved significant performance, a key issue is their tedious and time-consuming nature in collecting a vast number of paired images. Furthermore, unlike high-level vision tasks where labels are well-defined, the definition of the ground truth image (i.e., the normally lit image) in low-light image enhancement tasks is subjective and vague, which affects the representation of the enhanced image and makes it more difficult to achieve satisfactory generalization.

[0003] To alleviate the aforementioned problems, many unsupervised learning methods have been developed. Generally, these methods mainly fall into two technical categories. One uses unpaired low-light / normal-light paired images for training, where the low-light images are used for coarse enhancement, while the unpaired images are used to refine the global style with the help of adversarial learning; the other utilizes multiple low-light images for training, where image priors are used to help recover normal-light images. Both research approaches can achieve good visibility. However, to achieve satisfactory performance, the data distribution of the external low-light (and normal-light) images used for training must be highly correlated with the test images captured from real-world scenes; otherwise, the generalization ability of the low-light image enhancement model cannot be guaranteed. Summary of the Invention

[0004] In view of this, to solve the above problems, the present invention provides an image enhancement method, apparatus, electronic device, and storage medium, the technical solution of which is as follows:

[0005] An image enhancement method, the method comprising:

[0006] Acquire a single low-light image to be processed;

[0007] The single low-light image to be processed is input into an image enhancement network, which is trained on a zero-shot learner by defining a prior-guided regularization loss.

[0008] Obtain the expected exposure image output by the image enhancement network for the single low-light image to be processed.

[0009] Preferably, the process of training the zero-shot learner to obtain the image enhancement network by defining a prior-guided regularization loss includes:

[0010] Construct the zero-shot learning enhancer:

[0011]

[0012] in, The output image is the one with the expected exposure. To map the output illumination ratio and satisfy piecewise smoothness constraints. L represents the illuminance diagram. I represents the desired illumination level; I is the input single low-light image. R is the reflectivity map; N is the output noise map; φ(N) represents the regularization terms for the output expected exposure image, illumination ratio map, and noise map, respectively; λ1, λ2, and λ3 represent the regularization terms for the output image, illumination ratio map, and noise map, respectively. The respective balancing weights of φ(N) are set as follows: λ1 is set to 1, λ2 is set to 0.5, and λ3 is set to 0.1. These are texture regularization terms, color regularization terms, and illuminance regularization terms, respectively.

[0013] Construct the loss function for the zero-shot learner:

[0014]

[0015] Among them, L all L is the overall loss function; rec To rebuild the losses, It is the regularization loss of the illuminance map. For the gradient operator, λ g λ is the balance factor. g Set to 10; L N L is the regularization loss for the noisy map. N =||M(N)||, where M(·) is the function for calculating the mean of the input; L T For texture loss, F(·) is the feature extractor, T(·) is the illumination transformation, D(·) is the denoising operation, and γt is a learnable factor; L C For color loss, α is the pixel-to-pixel transformation function, α and β are learnable parameters, and S(·) is the function for calculating the input variance. for The corresponding r, g, b channels, I i ∈{I r ,I g ,I b} represents the r, g, and b channels corresponding to I; λ cf The constant parameter used to enhance image color is set to 0.025; E n ω is a constant parameter used to determine the image color level, set to 0.32; ω is a weighting parameter, set to 0.3; L I For illuminance loss, δ(·) represents the operation of generating a local image from the input image, B is the number of image patches, and λ il λ is a parameter used to balance the local contrast of the image under desired exposure. il Set to 2; ε is a constant parameter set in the range [0.25, 0.75].

[0016] The zero-shot learning enhancer is trained by iteratively optimizing its loss function, and the trained zero-shot learning enhancer is used as the image enhancement network.

[0017] Preferably, training the zero-shot learning enhancer includes:

[0018] The zero-shot learning enhancer is trained in the first stage using an input-guided enhancement method: a single low-light image is acquired for iterative training; the single low-light image for iterative training is input into the zero-shot learning enhancer to obtain the corresponding expected exposure image.

[0019] The zero-shot learning enhancer is trained in the second stage using output-guided refinement:

[0020] The expected exposure images obtained from each iteration of training in the first stage are stored in the historical data pool, and the average value of the multiple expected exposure images most recently stored in the historical data pool is used as the coarsely enhanced exposure image output from the first stage. The coarsely enhanced exposure image is input into the zero-shot learning enhancer to obtain the corresponding expected exposure image. The expected exposure images obtained from each iteration of training in the second stage are stored in the historical data pool.

[0021] Preferably, the illumination transformation T(·) is gamma correction and the denoising operation D(·) is zero-sample denoising operation.

[0022] An image enhancement apparatus, the apparatus comprising:

[0023] The image acquisition module is used to acquire a single low-light image to be processed;

[0024] An image enhancement module is used to input the single low-light image to be processed into an image enhancement network, which is trained on a zero-shot learner by defining a prior-guided regularization loss; and to obtain the expected exposure image output by the image enhancement network for the single low-light image to be processed.

[0025] Preferably, the process by which the image enhancement module trains the zero-shot learner to obtain the image enhancement network by defining a prior-guided regularization loss includes:

[0026] Construct the zero-shot learning enhancer:

[0027]

[0028] in, The output image is the one with the expected exposure. To map the output illumination ratio and satisfy piecewise smoothness constraints. L represents the illuminance diagram. I represents the desired illumination level; I is the input single low-light image. R is the reflectivity map; N is the output noise map; φ(N) represents the regularization terms for the output expected exposure image, illumination ratio map, and noise map, respectively; λ1, λ2, and λ3 represent the regularization terms for the output image, illumination ratio map, and noise map, respectively. The respective balancing weights of φ(N) are set as follows: λ1 is set to 1, λ2 is set to 0.5, and λ3 is set to 0.1. These are texture regularization terms, color regularization terms, and illuminance regularization terms, respectively.

[0029] Construct the loss function for the zero-shot learner:

[0030]

[0031] Among them, L all L is the overall loss function; rec To rebuild the losses, It is the regularization loss of the illumination map. For the gradient operator, λ g λ is the balance factor. g Set to 10; L N L is the regularization loss for the noise map. N =||M(N)||, where M(·) is the function for calculating the mean of the input; L T For texture loss, F(·) is the feature extractor, T(·) is the illumination transformation, D(·) is the denoising operation, and γt is a learnable factor; LC For color loss, α is the pixel-to-pixel transformation function, α and β are learnable parameters, and S(·) is the function for calculating the input variance. for The corresponding r, g, b channels, I i ∈{I r ,I g ,I b} represents the r, g, and b channels corresponding to I; λ cf The constant parameter used to enhance image color is set to 0.025; E n ω is a constant parameter used to determine the image color level, set to 0.32; ω is a weighting parameter, set to 0.3; L I For illuminance loss, δ(·) represents the operation of generating a local image from the input image, B is the number of image patches, and λ il λ is a parameter used to balance the local contrast of the image under desired exposure. il Set to 2; ε is a constant parameter set in the range [0.25, 0.75].

[0032] The zero-shot learning enhancer is trained by iteratively optimizing its loss function, and the trained zero-shot learning enhancer is used as the image enhancement network.

[0033] Preferably, the process of the image enhancement module training the zero-shot learning enhancer includes:

[0034] The zero-shot learning enhancer is trained in the first stage using an input-guided enhancement method: a single low-light image is acquired for iterative training; the single low-light image for iterative training is input into the zero-shot learning enhancer to obtain the corresponding expected exposure image.

[0035] The zero-shot learning enhancer is trained in the second stage using output-guided refinement:

[0036] The expected exposure images obtained from each iteration of training in the first stage are stored in the historical data pool, and the average value of the multiple expected exposure images most recently stored in the historical data pool is used as the coarsely enhanced exposure image output from the first stage. The coarsely enhanced exposure image is input into the zero-shot learning enhancer to obtain the corresponding expected exposure image. The expected exposure images obtained from each iteration of training in the second stage are stored in the historical data pool.

[0037] Preferably, the illumination transformation T(·) is gamma correction and the denoising operation D(·) is zero-sample denoising operation.

[0038] An electronic device includes: at least one memory and at least one processor; the memory stores an application program, and the processor invokes the application program stored in the memory, the application program being used to implement the image enhancement method.

[0039] A storage medium storing computer program code, which, when executed, implements the image enhancement method.

[0040] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0041] This invention provides an image enhancement method, apparatus, electronic device, and storage medium. The method involves acquiring a single low-light image to be processed; inputting the single low-light image to be processed into an image enhancement network, which is trained on a zero-shot learning enhancer using a defined prior-guided regularization loss; and obtaining the expected exposure image output by the image enhancement network for the single low-light image to be processed. This invention proposes a flexible and controllable zero-shot learning enhancer that can learn the expected exposure image using only the single low-light image itself, without relying on the statistical features of the training data. Therefore, it has good generalization ability for real-world scenes. Correspondingly, the image enhancement network constructed based on the zero-shot learning enhancer only needs a single low-light image for training to learn the expected exposure image. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0043] Figure 1 This is a flowchart of an image enhancement method provided in an embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram of the structure of an image enhancement network provided in an embodiment of the present invention;

[0045] Figure 3 λ provided for embodiments of the present invention cf and λ il Parameter analysis diagram;

[0046] Figure 4Visual comparison results of the image enhancement network provided in this embodiment of the invention with other low-light image enhancement models;

[0047] Figure 5 The image enhancement network provided in this embodiment of the invention is compared with other low-light image enhancement models on a non-reference dataset.

[0048] Figure 6 This is a schematic diagram of the structure of the image enhancement device provided in an embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] In order to solve the problem of visual degradation in low-light images in existing technologies, the inventors discovered that:

[0052] Researchers have proposed many methods, which can be mainly divided into non-learning methods, supervised learning methods, and unsupervised learning methods.

[0053] Non-learning methods include gamma correction, histogram equalization, and Retinex-based methods. Gamma correction methods transform the image illumination using a gamma curve, improving contrast but easily leading to overexposure or underexposure. Histogram equalization first calculates the histogram of the low-light image and then improves contrast by stretching the dynamic range; this method is simple and effective but easily amplifies local noise. Retinex-based methods decompose the low-light image into an illuminance map and a reflectance map, then use the reflectance map as the final enhanced image. This technique improves the detail and naturalness of the enhanced image but remains susceptible to strong noise.

[0054] In recent years, supervised learning methods have dominated research on low-light image enhancement tasks. Low-light Net (LLNet) was the earliest attempt, achieving image enhancement by training a deep autoencoder using synthetic dark images. Inspired by this work, many studies have used synthetic datasets to train low-light image enhancement models. However, synthetic datasets cannot reflect the characteristics of real low-light images, hindering the applicability of these methods in real-world scenes. To address this issue, Wei et al. collected a low-light dataset (LOL) containing paired low-light / normal-light images. Yang et al. expanded the LOL dataset, constructing a larger paired low-light image dataset with more realistic scenes. Chen et al. introduced a real paired image dataset containing original short-exposure low-light images and corresponding long-exposure reference images. Based on these real paired datasets, more supervised learning methods have been proposed. For example, Zhang et al. proposed KinD (kindling the darkness) and KinD++ (kindling the darkness plus), which not only illuminate dark areas but also remove hidden artifacts. Xu et al. utilized a signal-to-noise ratio-aware Transformer and convolutional models to dynamically enhance pixels through spatially varying operations. While these methods have made significant breakthroughs in enhancing quality, they rely on large-scale low-light / normal-light paired images, which are very time-consuming to acquire.

[0055] To reduce the pairing requirements for low-light / normal-light images, much work has focused on developing unsupervised learning methods. Typically, Jiang et al. proposed a heuristic adversarial network, a lightweight yet efficient single-path generative adversarial network. Ni et al. proposed an unsupervised enhancement generative adversarial network (UEGAN), which learns corresponding image-to-image mappings from a set of images with desired features. Jin et al. integrated a decomposition network and a light effect suppression network in a unified framework to achieve unsupervised nighttime image enhancement. These methods fully utilize unpaired images and achieve encouraging results. Furthermore, some methods attempt to achieve unsupervised low-light image enhancement using only low-light images. For example, Guo et al. proposed zero-reference deep curve estimation (Zero-DCE++), defining low-light image enhancement as an image-specific curve estimation task. Liu et al. proposed a retinex-inspired unrolling with architecture search (RUAS), achieving unsupervised learning through the design of cooperative no-reference learning strategies. All of these methods can achieve good enhancement quality. However, they still need to collect a large number of training images. Recently, two papers have further developed low-light image enhancement models trained on a single low-light image based on deep image prior (DIP). However, DIP mainly focuses on the restoration of the overall texture and color of the image and cannot effectively restore the ideal illumination.

[0056] To address the problems of the prior art, this invention proposes an image enhancement method that learns the desired exposure image from a single low-light image. This method requires no training dataset and can recover the desired exposure image given a low-light image and the desired brightness.

[0057] See Figure 1 , Figure 1 This is a flowchart illustrating the image enhancement method provided in an embodiment of the present invention. Figure 1 As shown, the image enhancement method includes the following steps:

[0058] S10, acquire the single low-light image to be processed.

[0059] In this embodiment of the invention, the single low-light image to be processed is an image taken in a low-light environment, which can be specified by the user.

[0060] S20, the single low-light image to be processed is input into the image enhancement network, which is trained on a zero-shot learner by defining a prior-guided regularization loss.

[0061] This invention proposes a flexible and controllable zero-shot learning enhancer that can learn from the expected exposure image, requiring only the low-light image itself as input. This zero-shot learning enhancer does not rely on the statistical features of the training data, thus exhibiting good generalization ability for real-world scenes.

[0062] To address this, the present invention proposes a set of prior-guided regularization losses based on illumination priors and natural image priors, which introduce a learnable flexibility factor to reduce the adverse effects of illumination differences between low-light images and enhanced images.

[0063] This invention proposes an image enhancement strategy that learns a desired exposure image from a single low-light image and constructs an image enhancement network for learning this desired exposure image from a single low-light image. In low-light image enhancement tasks, the definition of normal illumination is subjective, while the potential colors and textures of the image are objectively determined by the scene. Inspired by this, the zero-shot learning enhancer proposed in this invention introduces a desired illumination to recover the desired exposure image and utilizes natural images before learning potential colors and textures from the input low-light image itself. From this perspective, this invention formulates a set of prior-guided regularization losses and designs a two-stage progressive learning scheme to train the zero-shot learning enhancer step by step, helping to enhance images to achieve clear textures, vivid colors, and adjustable illumination.

[0064] Based on this, the process of training the zero-shot learner to obtain the image enhancement network by defining a prior-guided regularization loss includes the following steps:

[0065] Building zero-shot learning augmenters:

[0066]

[0067] in, The output image is the one with the expected exposure. To map the output illumination ratio and satisfy piecewise smoothness constraints. L represents the illuminance diagram. I represents the desired illumination level; I is the input single low-light image. R is the reflectivity map; N is the output noise map; φ(N) represents the regularization terms for the output expected exposure image, illumination ratio map, and noise map, respectively; λ1, λ2, and λ3 represent the regularization terms for the output image, illumination ratio map, and noise map, respectively. The respective balancing weights of φ(N) are set as follows: λ1 is set to 1, λ2 is set to 0.5, and λ3 is set to 0.1. These are texture regularization terms, color regularization terms, and illuminance regularization terms, respectively.

[0068] The loss function for constructing a zero-shot learner augmenter is:

[0069]

[0070] Among them, L all L is the overall loss function; rec To rebuild the losses, It is the regularization loss of the illumination map. For the gradient operator, λ g λ is the balance factor. g Set to 10; L N L is the regularization loss for the noise map. N =||M(N)||, where M(·) is the function for calculating the mean of the input; L T For texture loss, F(·) is the feature extractor, T(·) is the illumination transformation, D(·) is the denoising operation, and γt is a learnable factor; L C For color loss, α is the pixel-to-pixel transformation function, α and β are learnable parameters, and S(·) is the function for calculating the input variance. for The corresponding r, g, b channels, I i ∈{I r ,I g ,I b} represents the r, g, and b channels corresponding to I; λ cf The constant parameter used to enhance image color is set to 0.025; E n ω is a constant parameter used to determine the image color level, set to 0.32; ω is a weight parameter, set to 0.3.

[0071] L I For illuminance loss, δ(·) represents the operation of generating a local image from the input image, B is the number of image patches, and λ il λ is a parameter used to balance the local contrast of the image under desired exposure. il Set to 2; ε is a constant parameter set in the range [0.25, 0.75].

[0072] The zero-shot learning augmenter is trained by iteratively optimizing the loss function of the zero-shot learning augmenter, and the trained zero-shot learning augmenter is used as an image augmentation network.

[0073] In the specific implementation process, 1) the process of constructing an image intensifier that learns the expected exposure image from a single low-light image can include the following steps:

[0074] This invention is based on learning an image enhancement strategy for the desired exposure image from a single low-light image and utilizes a zero-shot learning enhancer to construct an image enhancement network that learns the desired exposure image from a single low-light image. Based on Retinex theory, a low-light image can be modeled as:

[0075]

[0076] Where I is the low-light input image, R is the reflectance map, L is the illuminance map, N is the noise map, and ο represents element-wise multiplication. Given a low-light image I, the goal of the low-light image enhancement task is to recover a normal-light image. However, as previously explained, the definition of a normal-light image is quite subjective. To address this issue, this invention introduces a desired illuminance value. And aims to restore an image with the desired exposure. Therefore, equation (1) can be restated as:

[0077]

[0078] in, For the expected exposure image, It is a illumination ratio map that satisfies piecewise smoothness constraints. Then, the present invention can recover from I by solving a regularization minimization problem.

[0079]

[0080] in, φ(N) represents the regularization terms for the expected exposure image, illumination ratio map, and noise map, respectively, and λ1, λ2, and λ3 represent the regularization terms for the image, illumination ratio map, and noise map, respectively. φ(N) represents the respective balancing weights.

[0081] As explained above, The intended exposure image. Therefore, the core of this invention is... The design of this invention is based on illumination priors and the objectivity of texture and color (i.e., natural image priors). Inspired by an illumination prior, this invention introduces a desired illumination into the regularization term; and inspired by a natural image prior, this invention maintains texture and color consistency with the input low-light image. However, the illumination difference between the desired exposure image and the input low-light image hinders texture and color consistency. To address this issue, this invention further... Decoupled into an illuminance regularization term A color regular expression And a texture regularization term Right now:

[0082]

[0083] in, It is designed using illuminance priors, while ψ c and ψ T It is designed based on natural image priors. Specifically, this invention focuses on ψ c and ψ T The invention introduces a learnable flexibility factor (a learnable parameter that changes continuously during training iterations) to help ensure the flexibility consistency of texture and color. Therefore, by setting λ1 to 1, the basic idea of ​​the zero-shot learning enhancer can be expressed as:

[0084]

[0085] Based on the above analysis, this invention constructs a flexible and controllable zero-shot learning enhancer, which can be called ZSEer (Zero-shot Enhancer). See also Figure 2 , Figure 2 This is a schematic diagram of the image enhancement network provided in an embodiment of the present invention. Figure 2 As shown, a single low-light image is input into a zero-shot learning enhancer (which can be a deep neural network Enhancer(·)), and then outputs... And N, the zero-shot learning augmenter can be represented as:

[0086]

[0087] 2) The process of constructing the loss function for the zero-shot learner augmenter can include the following steps:

[0088] This invention proposes a set of loss functions to aid in reconstruction. And N. According to equation (5), the present invention will use the overall loss function L all Defined as:

[0089]

[0090] Among them, Lrec Losses due to reconstruction:

[0091]

[0092] It is the regularization loss of the illumination map, λ2, λ3, λ T , λ C , λ I The values ​​are set to 0.5, 0.1, 0.2, 1.0, and 0.1 respectively. In this invention, piecewise smoothness constraints are used for definition.

[0093]

[0094] in, For the gradient operator, λ g Set the balance factor to 10.

[0095] L N This is the regularization loss of the noise map. Here, the invention does not require a priori noise distribution, but simply assumes the noise mean is 0. Therefore, L N Defined as:

[0096] L N =||M(N)|| (10)

[0097] Where M(·) is a function that calculates the mean of the input.

[0098] L I L C and L T These represent the prior-guided illuminance loss, color loss, and texture loss, respectively.

[0099] Texture loss L T The design aims to encourage zero-shot learning enhancers to learn texture details from input low-light images. However, the low illumination and high noise of low-light images hinder texture learning. To address this issue, this invention performs an illumination transformation T(·) and a denoising operation D(·) on the input low-light image. Specifically, given a single low-light image I, this invention first calculates the transformed image D(T(I)) and uses it as a reference. Then, the perceptual features of the enhanced image are constrained to maintain a flexible consistency with the perceptual features of the transformed image, which can be described in the following form:

[0100]

[0101] Where F(·) is the feature extractor (specifically a perceptual feature extractor), which can be implemented using a pre-trained VGG model; γt is a learnable factor that can reduce the adverse effects of the illumination difference between the expected exposure image and the transformed image; the illumination transformation T(·) can be implemented in various ways, such as gamma correction, histogram equalization, and linear stretching. In this embodiment of the invention, gamma correction is simply used, i.e., T(I) = I γi γi is a learnable parameter; for the denoising operation D(·), the present invention employs a zero-sample denoising operation, which is trained by using the input low-light image itself to achieve image denoising.

[0102] Color loss L C Design. The red (r), green (g), and blue (b) components of the enhanced image are constrained to match the components of the input low-light image. Simultaneously, inspired by color measurement methods, this invention introduces a colorimetric regularization term to ensure color output. C The formal definition of is:

[0103]

[0104] in:

[0105]

[0106]

[0107] α is a pixel-to-pixel transformation function that adjusts the intensity of the enhanced image, where α and β are both learnable parameters; M(·) is a function for calculating the input mean, and S(·) is a function for calculating the input variance; for The corresponding r, g, b channels, I i ∈{I r ,I g ,I b} represents the r, g, and b channels corresponding to I; λ cf For a constant parameter used to enhance image color, see [link / reference]. Figure 3 , Figure 3 λ provided for embodiments of the present invention cf and λ il Parameter analysis diagram, such as Figure 3 As shown, in this invention, λ cf The value is 0.025; E n ω is a constant parameter that determines the color level of the image to be enhanced, set to 0.32; ω is a weight parameter, set to 0.3.

[0108] Illuminance loss L IThe design aims to force the image's illuminance to approach the expected illuminance and improve image contrast through illuminance loss. To this end, this invention employs global and local constraints to define the illuminance loss. Given the desired illuminance ε, L... I It can be defined as:

[0109]

[0110] Where δ(·) represents the operation of generating a local image from the input image. B is the number of local image blocks, and λ il Parameters used to balance the local contrast of the image under desired exposure, such as Figure 3 As shown, based on experience, it is set to 2 in this embodiment of the invention. ε is a constant parameter, usually set in the range of [0.25, 0.75], which helps the zero-shot learning enhancer overcome the subjectivity of the definition of normal illumination.

[0111] 3) The process of training the zero-shot learner by iteratively optimizing the loss function of the zero-shot learner can include the following steps:

[0112] This invention employs seven datasets for comparative experiments: the real low-light dataset (LOL-real), the test low-light dataset (LOL-test), the multi-exposure image fusion dataset (MEF), the low-light image enhancement dataset (LIME-D), the digital image contrast enhancement commercial dataset (DICM), the commonly used Vasileios Vonikakis dataset (VV), and the naturalness preserved enhancement dataset (NPE). LOL-real and LOL-test have normal-light images paired with low-light images; therefore, this invention uses peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and the natural image quality evaluator (NIQE) for evaluation. MEF, LIME, DICM, VV, and NPE do not have normal-light images paired with low-light images; therefore, this invention uses the non-reference metric NIQE for evaluation.The comparison methods include low-light image enhancement (LIME), Low-Light Image Enhancement Using the Camera Response Model, Retinex Network (Retinex-Net), KinD++, SNR-aware low-light image enhancement (Xu et al.), UEGAN, Enlighten Generative Adversarial Network (GAN), Unsupervised Night Enhancement (Jin et al.), Self-calibrated Illumination (SCI), Semantic-guided Zero-shot Learning (SGZ), Zero-DCE++, RetinexDIP, and a self-supervised method based on differentially untrained networks (Liang et al.).

[0113] In practical enhancement applications, to further reduce the adverse effects of the illumination difference between the expected exposure image and the input single low-light image, such as... Figure 2 As shown, this invention employs a two-stage progressive learning strategy to train the enhancer, namely, input-guided enhancement and output-guided refinement:

[0114] The zero-shot learning enhancer is trained in the first stage using an input-guided enhancement method: a single low-light image is acquired for iterative training; the single low-light image for iterative training is input into the zero-shot learning enhancer to obtain the corresponding expected exposure image.

[0115] The zero-shot learning enhancer is trained in the second stage using output-guided refinement:

[0116] The expected exposure images obtained from each iteration of training in the first stage are stored in the historical data pool, and the average of the multiple expected exposure images stored in the historical data pool that are closest to the current time is used as the coarsely enhanced exposure image output by the first stage. The coarsely enhanced exposure image is input into the zero-shot learning enhancer to obtain the corresponding expected exposure image. The expected exposure images obtained from each iteration of training in the second stage are stored in the historical data pool.

[0117] Specifically, input-guided enhancement utilizes the input low-light image to guide the learning of the expected exposure image. In this stage, due to the effects of low light and noise amplification, a flexible constraint needs to be applied to the texture loss. Furthermore, to stabilize the training process, a historical data pool is constructed to store the expected exposure images output in each iteration. Then, this invention uses the average of the latest 200 expected exposure images to construct the output of the first stage. After this stage, the network outputs a coarsely enhanced exposure image.

[0118] Output-guided refinement, which utilizes the first-stage coarse enhancement of the exposed image. The image is then refined using historical enhancement techniques to restore the expected exposure. At this stage, the texture loss is modified as follows:

[0119]

[0120] Here, the flexibility factor γt is removed because the first stage has learned an appropriate illuminance, and the illuminance difference between R and the final expected exposure image is quite small. Note that the output-guided refinement stage can be executed multiple times. In each execution, the latest expected exposure image is used to update...

[0121] Therefore, this invention designs a two-stage progressive learning scheme that uses the input low-light image and historical expected exposure images to gradually recover the ideal image, helping the enhancer generate images with clear textures and vibrant colors.

[0122] S30: Obtain the expected exposure image output by the image enhancement network for the single low-light image to be processed.

[0123] In this embodiment of the invention, based on an image enhancement network, the expected exposure image corresponding to a single low-light image to be processed can be obtained.

[0124] Furthermore, to verify the effectiveness of the image enhancement method provided in this embodiment of the invention, the following experimental analysis was conducted. This invention establishes three versions of zero-shot learning enhancers based on different expected illumination standards, and demonstrates that the established zero-shot learning enhancers can achieve competitiveness and visibility compared to existing methods:

[0125] This invention compares the performance of the proposed method with several state-of-the-art image enhancement methods. In this embodiment, three versions of the zero-shot learning enhancer are built: ZSEer-V1 is the first version, configured with a fixed expected illumination of 0.5; ZSEer-V2 is the second version, configured with 10 expected illuminations, and the final result is determined by a non-reference metric; ZSEer-V3 is the third version, configured with 10 expected illuminations, and the final result is determined by subjective evaluation through user interaction.

[0126] Results on the LOL-real and LOL-test datasets. See Table 1 for the results on the LOL-real and LOL-test datasets. As shown in Table 1, the proposed method achieves better performance on PSNR, SSIM, and NIQE compared to non-learning methods. Given appropriate expected illumination, ZSEer-V3 can even achieve performance comparable to supervised and unsupervised learning methods. Compared to methods trained only using low-light images, the proposed method achieves more competitive performance even without user interaction. These results can be attributed to the introduced expected illumination and the proposed regularization loss, which reduces the ill-posedness of image reconstruction. Furthermore, see... Figure 4 , Figure 4 The image enhancement network provided in this embodiment of the invention is compared with other low-light image enhancement models through visualization. For example... Figure 4 As shown, existing methods tend to amplify noise and over-enhance colors. In contrast, the proposed method effectively suppresses noise and maintains the naturalness of the image. Furthermore, ZSEer-V3 significantly improves PSNR and SSIM, but has a similar NIQE as ZSEer-V1 and ZSEerV2. This result is because ZSEer-V1, ZSEer-V2, and ZSEer-V3 recover similar textures and colors.

[0127] Generality Evaluation on Non-Reference Real-World Datasets. This invention further evaluates low-light image enhancement models on real-world scenes using the MEF, LIME, DICM, VV, and NPE datasets. See Table 2, which shows the NIQE results on the non-reference datasets. Comparing Tables 1 and 2, it can be seen that although supervised and unsupervised learning methods achieve very competitive results on the LOL-real and LOL-test datasets, they fail to achieve satisfactory performance on some real-world scenes. This is because the data distribution of the test images may differ significantly from that of the training images. In contrast, this method does not rely on the training data, allowing it to generalize well to unknown scenes. Furthermore, compared to methods trained with a single low-light image, the proposed method achieves more competitive visibility. See Table 2. Figure 5 , Figure 5The image enhancement network provided in this embodiment of the invention is compared with other low-light image enhancement models on a non-reference dataset.

[0128] Table 1

[0129]

[0130] Table 2

[0131]

[0132] Based on the image enhancement method provided in the above embodiments, the present invention provides a corresponding apparatus for performing the above image enhancement method, the structural schematic diagram of which is shown below. Figure 6 As shown, it includes:

[0133] Image acquisition module 10 is used to acquire a single low-light image to be processed;

[0134] The image enhancement module 20 is used to input a single low-light image to be processed into the image enhancement network, which is trained on a zero-shot learner by defining a prior-guided regularization loss; and to obtain the expected exposure image output by the image enhancement network for the single low-light image to be processed.

[0135] Optionally, the process by which the image enhancement module 20 trains the zero-shot learner to obtain the image enhancement network by defining a prior-guided regularization loss includes:

[0136] Building zero-shot learning augmenters:

[0137]

[0138] in, The output image is the one with the expected exposure. To map the output illumination ratio and satisfy piecewise smoothness constraints. L represents the illuminance diagram. I represents the desired illumination level; I is the input single low-light image. R is the reflectivity map; N is the output noise map; φ(N) represents the regularization terms for the output expected exposure image, illumination ratio map, and noise map, respectively; λ1, λ2, and λ3 represent the regularization terms for the output image, illumination ratio map, and noise map, respectively. The respective balancing weights of φ(N) are set as follows: λ1 is set to 1, λ2 is set to 0.5, and λ3 is set to 0.1. These are texture regularization terms, color regularization terms, and illuminance regularization terms, respectively.

[0139] The loss function for constructing a zero-shot learner augmenter is:

[0140]

[0141] Among them, L all L is the overall loss function; rec To rebuild the losses, It is the regularization loss of the illumination map. For the gradient operator, λ g λ is the balance factor. g Set to 10; L N L is the regularization loss for the noise map. N =||M(N)||, where M(·) is the function for calculating the mean of the input; L T For texture loss, F(·) is the feature extractor, T(·) is the illumination transformation, D(·) is the denoising operation, and γt is a learnable factor; L C For color loss, α is the pixel-to-pixel transformation function, α and β are learnable parameters, and S(·) is the function for calculating the input variance. for The corresponding r, g, b channels, I i ∈{I r ,I g ,I b} represents the r, g, and b channels corresponding to I; λ cf The constant parameter used to enhance image color is set to 0.025; E n ω is a constant parameter used to determine the image color level, set to 0.32; ω is a weight parameter, set to 0.3.

[0142] L I For illuminance loss, δ(·) represents the operation of generating a local image from the input image, B is the number of image patches, and λ il λ is a parameter used to balance the local contrast of the image under desired exposure. il Set to 2; ε is a constant parameter set in the range [0.25, 0.75].

[0143] The zero-shot learning augmenter is trained by iteratively optimizing the loss function of the zero-shot learning augmenter, and the trained zero-shot learning augmenter is used as an image augmentation network.

[0144] Optionally, the process of training the zero-shot learning enhancer by the image enhancement module 20 includes:

[0145] The zero-shot learning enhancer is trained in the first stage using an input-guided enhancement method: a single low-light image is acquired for iterative training; the single low-light image for iterative training is input into the zero-shot learning enhancer to obtain the corresponding expected exposure image.

[0146] The zero-shot learning enhancer is trained in the second stage using output-guided refinement:

[0147] The expected exposure images obtained from each iteration of training in the first stage are stored in the historical data pool, and the average of the multiple expected exposure images stored in the historical data pool that are closest to the current time is used as the coarsely enhanced exposure image output by the first stage. The coarsely enhanced exposure image is input into the zero-shot learning enhancer to obtain the corresponding expected exposure image. The expected exposure images obtained from each iteration of training in the second stage are stored in the historical data pool.

[0148] Optionally, illumination transformation T(·) is gamma correction and denoising operation D(·) is zero-sample denoising operation.

[0149] It should be noted that the detailed functions of each module in the embodiments of the present invention can be found in the corresponding disclosures of the above-described image enhancement method embodiments, and will not be repeated here.

[0150] Based on the image enhancement method provided in the above embodiments, this invention provides an electronic device, which includes: at least one memory and at least one processor; the memory stores an application program, and the processor calls the application program stored in the memory, the application program being used to implement the image enhancement method.

[0151] Based on the image enhancement method provided in the above embodiments, this embodiment of the invention provides a storage medium that stores computer program code, which implements the image enhancement method when executed.

[0152] The above provides a detailed description of the image enhancement method, apparatus, electronic device, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

[0153] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0154] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that elements inherent to a process, method, article, or apparatus that comprises a list of elements, or elements inherent to such processes, methods, articles, or apparatus, are also included. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0155] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image enhancement method, characterized in that, The method includes: Acquire a single low-light image to be processed; The single low-light image to be processed is input into an image enhancement network, which is trained on a zero-shot learner by defining a prior-guided regularization loss. Obtain the expected exposure image output by the image enhancement network for the single low-light image to be processed; The process of training the zero-shot learner to obtain the image enhancement network by defining a prior-guided regularization loss includes: Construct the zero-shot learning enhancer: in, ; The output image is the one with the expected exposure. To map the output illumination ratio and satisfy piecewise smoothness constraints. L represents the illuminance map. The desired illuminance; The input is a single low-light image. , Reflectance map; This is the output noise graph; , , These are the regularization terms for the output expected exposure image, illumination ratio map, and noise map, respectively; , , They are respectively , , Their respective balancing weights, Set to 1 Set to 0.5 Set to 0.1; , , These are texture regularization terms, color regularization terms, and illuminance regularization terms, respectively. Construct the loss function for the zero-shot learner: in, The overall loss function; To rebuild the losses, ; It is the regularization loss of the illumination map. , For gradient operators, As a balance factor, Set to 10; The regularization loss is for the noise map. , A function to calculate the mean of the input; For texture loss, , For feature extractors, For lighting changes, For noise reduction operation, It is a learnable factor; For color loss, , , , Pixel-to-pixel transformation function and For learnable parameters, A function for calculating the input variance; for corresponding , , aisle, for corresponding , , aisle; This is a constant parameter used to enhance image color; set it to 0.

025. The constant parameter used to determine the color level of the image is set to 0.32; The weight parameter is set to 0.3; For illuminance loss, , This is an operation that generates a local image from an input image. Number of image blocks These are parameters used to balance the local contrast of the image under the desired exposure. Set to 2; This is a constant parameter set within the range of [0.25, 0.75]. The zero-shot learning enhancer is trained by iteratively optimizing its loss function, and the trained zero-shot learning enhancer is used as the image enhancement network.

2. The method according to claim 1, characterized in that, Training the zero-shot learning enhancer includes: The zero-shot learning enhancer is trained in the first stage using an input-guided enhancement method: a single low-light image is acquired for iterative training; the single low-light image for iterative training is input into the zero-shot learning enhancer to obtain the corresponding expected exposure image. The zero-shot learning enhancer is trained in the second stage using output-guided refinement: The expected exposure images obtained from each iteration of training in the first stage are stored in the historical data pool, and the average value of the multiple expected exposure images most recently stored in the historical data pool is used as the coarsely enhanced exposure image output from the first stage. The coarsely enhanced exposure image is input into the zero-shot learning enhancer to obtain the corresponding expected exposure image. The expected exposure images obtained from each iteration of training in the second stage are stored in the historical data pool.

3. The method according to claim 1, characterized in that, Lighting changes For gamma correction and noise reduction operations This is a zero-sample denoising operation.

4. An image enhancement device, characterized in that, The device includes: The image acquisition module is used to acquire a single low-light image to be processed; An image enhancement module is used to input the single low-light image to be processed into an image enhancement network, which is trained on a zero-shot learner by defining a prior-guided regularization loss; and to obtain the expected exposure image output by the image enhancement network for the single low-light image to be processed. The process by which the image enhancement module trains the zero-shot learner to obtain the image enhancement network by defining a prior-guided regularization loss includes: Construct the zero-shot learning enhancer: in, ; The output image is the one with the expected exposure. To map the output illumination ratio and satisfy piecewise smoothness constraints. L represents the illuminance map. The desired illuminance; The input is a single low-light image. , Reflectance map; This is the output noise graph; , , These are the regularization terms for the output expected exposure image, illumination ratio map, and noise map, respectively; , , They are respectively , , Their respective balancing weights, Set to 1 Set to 0.5 Set to 0.1; , , These are texture regularization terms, color regularization terms, and illuminance regularization terms, respectively. Construct the loss function for the zero-shot learner: in, The overall loss function; To rebuild the losses, ; It is the regularization loss of the illumination map. , For gradient operators, As a balance factor, Set to 10; The regularization loss is for the noise map. , A function to calculate the mean of the input; For texture loss, , For feature extractors, For lighting changes, For noise reduction operation, It is a learnable factor; For color loss, , , , Pixel-to-pixel transformation function and For learnable parameters, A function for calculating the input variance; for corresponding , , aisle, for corresponding , , aisle; This is a constant parameter used to enhance image color; set it to 0.

025. The constant parameter used to determine the color level of the image is set to 0.32; The weight parameter is set to 0.3; For illuminance loss, , This is an operation that generates a local image from an input image. Number of image blocks These are parameters used to balance the local contrast of the image under the desired exposure. Set to 2; This is a constant parameter set within the range of [0.25, 0.75]. The zero-shot learning enhancer is trained by iteratively optimizing its loss function, and the trained zero-shot learning enhancer is used as the image enhancement network.

5. The apparatus according to claim 4, characterized in that, The process by which the image enhancement module trains the zero-shot learning enhancer includes: The zero-shot learning enhancer is trained in the first stage using an input-guided enhancement method: a single low-light image is acquired for iterative training; the single low-light image for iterative training is input into the zero-shot learning enhancer to obtain the corresponding expected exposure image. The zero-shot learning enhancer is trained in the second stage using output-guided refinement: The expected exposure images obtained from each iteration of training in the first stage are stored in the historical data pool, and the average value of the multiple expected exposure images most recently stored in the historical data pool is used as the coarsely enhanced exposure image output from the first stage. The coarsely enhanced exposure image is input into the zero-shot learning enhancer to obtain the corresponding expected exposure image. The expected exposure images obtained from each iteration of training in the second stage are stored in the historical data pool.

6. The apparatus according to claim 4, characterized in that, Lighting changes For gamma correction and noise reduction operations This is a zero-sample denoising operation.

7. An electronic device, characterized in that, The electronic device includes: at least one memory and at least one processor; the memory stores an application program, and the processor calls the application program stored in the memory, the application program being used to implement the image enhancement method according to any one of claims 1-3.

8. A storage medium, characterized in that, The storage medium stores computer program code, which, when executed, implements the image enhancement method according to any one of claims 1-3.

Images