Unsupervised image enhancement method and device, equipment and storage medium

Abstract

The invention discloses an unsupervised image enhancement method and device, equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: inputting an obtained low-illumination image into a pre-trained unsupervised enhancement model for network cooperative processing, and obtaining an enhanced image, the unsupervised enhancement model is obtained by training based on a preset loss function and a pre-trained generative adversarial network. According to the method, the illumination distribution of the low-illumination image is accurately adjusted through the pre-trained unsupervised enhancement model, so that efficient enhancement of the single low-illumination image is realized, and the enhancement result is ensured to have relatively high visual quality and naturalness in global brightness and local detail levels; and a good balance is obtained between image detail keeping and noise suppression.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and in particular, to an unsupervised image enhancement method, apparatus, device, and storage medium. Background Art

[0002] Currently, the core challenge in low-light image enhancement lies in how to effectively restore image details and suppress noise in the absence of paired training data. Traditional methods (such as histogram equalization and the classic Retinex model) usually rely on manually designed prior information or strictly aligned low-light - normal-light image pairs, and it is difficult to handle complex and changing real-world scenarios.

[0003] In recent years, unsupervised methods (such as EnlightenGAN) have achieved the mapping from a single low-light image to a normal-light image through a generative adversarial network (GAN), and can be modeled without paired training samples. However, such methods are still prone to problems such as noise amplification or color distortion under extremely low-light conditions. On the other hand, PairLIE (paired low-light image enhancement) uses the reflection consistency constraint between paired low-light images to optimize the Retinex (retina-cortex model) decomposition process, and to a certain extent, improves the image enhancement quality, but its high dependence on image alignment limits its practical application in dynamic scenarios. Summary of the Invention

[0004] The main purpose of this application is to provide an unsupervised image enhancement method, apparatus, device, and storage medium, aiming to solve the technical problem of how to achieve efficient enhancement of a single low-light image and achieve a good balance between maintaining image details and suppressing noise.

[0005] To achieve the above object, this application proposes an unsupervised image enhancement method, and the unsupervised image enhancement method includes: Input the obtained low-light image into a pre-trained unsupervised enhancement model for network collaborative processing to obtain an enhanced image, where the unsupervised enhancement model is trained based on a preset loss function and a pre-trained generative adversarial network.

[0006] In an embodiment, the pre-trained unsupervised enhancement model includes an image mapping network, an illuminance estimation network, and a reflectance estimation network. The step of inputting the obtained low-light image into a pre-trained unsupervised enhancement model for network collaborative processing to obtain an enhanced image includes: Decompose the low-light image into an intermediate image through the image mapping network; Process the intermediate image through the illuminance estimation network to obtain an illumination component; Process the intermediate image through the reflectivity estimation network to obtain a reflectivity component; Calculate the enhanced image based on the illumination component and the reflectivity component.

[0007] In a feasible embodiment, the preset loss function includes a first preset loss function and a second preset loss function. Before the step of inputting the obtained low-illumination image into a pre-trained unsupervised enhancement model for network collaborative processing to obtain an initial enhanced image, it includes: Input the obtained low-illumination image into a pre-trained generative adversarial network to obtain a preliminary enhanced image, where the pre-trained generative adversarial network is trained based on the first preset loss function; Combine the preliminary enhanced image and the second preset loss function to train the unsupervised enhancement model to obtain a pre-trained unsupervised enhancement model.

[0008] In a feasible embodiment, the pre-trained generative adversarial network includes a generator, a global discriminator, and a local discriminator. The step of inputting the obtained low-illumination image into a pre-trained generative adversarial network to obtain a preliminary enhanced image includes: Through the generator, perform feature enhancement and spatial attention guidance on the low-illumination image to generate an enhanced candidate image; Through the global discriminator, evaluate the global illumination distribution and structural consistency of the enhanced candidate image to generate a global discrimination result; Through the local discriminator, discriminate multiple local regions randomly cropped from the enhanced candidate image to generate a local discrimination result; Based on the global discrimination result and the local discrimination result, optimize the generator, the global discriminator, and the local discriminator, and generate the preliminary enhanced image through the optimized generator.

[0009] In a feasible embodiment, the step of performing feature enhancement and spatial attention guidance on the low-illumination image through the generator to generate the enhanced candidate image includes: Through the encoder of the generator, perform multi-level feature extraction on the low-illumination image to generate an intermediate feature map; Generate a spatial attention map based on the illumination component decomposed from the low-illumination image through normalization processing; Perform a channel-wise multiplication operation on the spatial attention map and the intermediate feature map to generate an adaptive feature enhancement map; Through the decoder of the generator, spatial restoration and detail reconstruction are performed on the adaptive feature enhancement map to generate the enhanced candidate image.

[0010] In a feasible embodiment, the first preset loss function includes a self-feature preservation loss function and an adversarial network loss function. Before the step of inputting the obtained low-illumination image into a pre-trained generative adversarial network to obtain a preliminary enhanced image, it includes: Combining the self-feature preservation loss function and the adversarial network loss function, training the generative adversarial network to obtain a pre-trained generative adversarial network.

[0011] In a feasible embodiment, the self-feature preservation loss function includes a global feature preservation loss function and a local feature preservation loss function, and the adversarial network loss function includes a global adversarial loss function and a local adversarial loss function. The step of combining the self-feature preservation loss function and the adversarial network loss function, training the generative adversarial network to obtain a pre-trained generative adversarial network includes: Input the low-illumination image into the generator to generate an enhanced candidate image; Obtain a real image with normal illumination, and input the real image and the enhanced candidate image into the global discriminator respectively, and calculate to obtain the global adversarial loss function; Input multiple local regions of the randomly cropped real image and the enhanced candidate image into the local discriminator respectively, and calculate to obtain the local adversarial loss function; Extract the deep features of the low-illumination image and the enhanced candidate image, and calculate the global feature preservation loss function based on the deep features; Calculate the local feature difference between the corresponding regions of the low-illumination image and the enhanced candidate image to obtain the local feature preservation loss function; According to the global adversarial loss function, the local adversarial loss function, the global feature preservation loss function, and the local feature preservation loss function, construct a total loss function, and update the parameters of the generative adversarial network based on the total loss function to obtain a pre-trained generative adversarial network.

[0012] In a feasible embodiment, the second preset function includes a projection loss function, a reflectance consistency loss function, and a Retinex loss function of the retina-cortex theory; The step of combining the preliminary enhanced image and the second preset loss function, training the unsupervised enhancement model to obtain a pre-trained unsupervised enhancement model includes: The low - illumination image is preliminarily enhanced by the generator to generate the preliminarily enhanced image; The low - illumination image and the preliminarily enhanced image are subjected to mapping processing by the image mapping network in combination with the projection loss function to generate a first mapped image and a second mapped image; The first mapped image and the second mapped image are decomposed by the illuminance estimation network to obtain the illumination component; The first mapped image and the second mapped image are decomposed and aligned by the reflectance estimation network in combination with the reflectance consistency loss function to obtain the reflectance component; The Retinex loss function is used to constrain the reconstruction relationship between the illumination component and the reflectance component to obtain an optimized image decomposition result; The unsupervised enhancement model is trained based on the optimized image decomposition result to obtain a pre - trained unsupervised enhancement model.

[0013] In addition, to achieve the above object, the present application also proposes an unsupervised image enhancement device, which includes: An image enhancement module, configured to input the acquired low - illumination image into a pre - trained unsupervised enhancement model for network collaborative processing to obtain an enhanced image, where the unsupervised enhancement model is trained based on a preset loss function and a pre - trained generative adversarial network.

[0014] In addition, to achieve the above object, the present application also proposes an unsupervised image enhancement device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the unsupervised image enhancement method as described above.

[0015] In addition, to achieve the above object, the present application also proposes a storage medium, which is a computer - readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, it implements the steps of the unsupervised image enhancement method as described above.

[0016] In addition, to achieve the above object, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the unsupervised image enhancement method as described above.

[0017] An unsupervised image enhancement method, device, equipment and storage medium proposed by this application. The method includes inputting the obtained low-light image into a pre-trained unsupervised enhancement model for network collaborative processing to obtain an enhanced image, where the unsupervised enhancement model is trained based on a preset loss function and a pre-trained generative adversarial network. This method can precisely adjust the illumination distribution of the low-light image through the pre-trained unsupervised enhancement model, thereby realizing the efficient enhancement of a single low-light image, and achieving a good balance between maintaining image details and suppressing noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.

[0019] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0020] Figure 1 It is a schematic flowchart provided for the first embodiment of the unsupervised image enhancement method of this application; Figure 2 It is a schematic flowchart provided for the second embodiment of the unsupervised image enhancement method of this application; Figure 3 It is a schematic diagram of the structure of the generative adversarial network module provided for the second embodiment of this application; Figure 4 It is a schematic diagram of the training process of the RetinexGAN-LIE model provided for the second embodiment of this application; Figure 5 It is a schematic diagram of the testing process of the RetinexGAN-LIE model provided for the second embodiment of this application; Figure 6 It is a schematic flowchart provided for the third embodiment of the unsupervised image enhancement method of this application; Figure 7 It is a schematic diagram of the training process of the generative adversarial network provided for the third embodiment of this application; Figure 8 It is a schematic diagram of the module structure of the unsupervised image enhancement device of this application; Figure 9 It is a schematic diagram of the device structure of the hardware operating environment involved in the unsupervised image enhancement method in the embodiments of this application.

[0021] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with embodiments with reference to the accompanying drawings. Detailed implementation manners

[0022] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not used to limit this application.

[0023] To better understand the technical solutions of this application, the following will be described in detail in conjunction with the drawings of the specification and specific implementation manners.

[0024] The main solution of the embodiment of this application is: input the obtained low-light image into a pre-trained unsupervised enhancement model for network collaborative processing to obtain an enhanced image, where the unsupervised enhancement model is trained based on a preset loss function and a pre-trained generative adversarial network.

[0025] In this embodiment, for the convenience of description, a personal computer is used as the execution subject for elaboration below.

[0026] Currently, the core challenge in low-light image enhancement lies in: how to effectively restore image details and suppress noise in the absence of paired training data. Traditional methods (such as histogram equalization and the classic Retinex model) usually rely on manually designed prior information or strictly aligned low-light - normal-light image pairs and are difficult to handle complex and changing real scenes. For example, methods based on the Retinex theory achieve enhancement by decomposing an image into an illumination component and a reflection component, but their performance is often limited by the fixed Gaussian filter design or the dependence on paired data. Although supervised learning methods based on deep convolutional networks perform excellently under the condition of sufficient paired data, in practical applications, the acquisition cost of high-quality reference images is relatively high, and even difficult to achieve in some scenarios. In recent years, unsupervised methods have realized the mapping from a single low-light image to a normal-light image through generative adversarial networks (GANs), and can perform modeling without paired training samples. However, such methods are still prone to problems such as noise amplification or color distortion under extremely low-light conditions. On the other hand, PairLIE uses the reflection consistency constraint between paired low-light images to optimize the Retinex decomposition process, which improves the image enhancement quality to a certain extent, but its high dependence on image alignment limits its practical application in dynamic scenes.

[0027] This application provides a solution. Through a pre-trained unsupervised enhancement model, combining the physical interpretability of the Retinex theory and generative adversarial networks (GANs), it can achieve efficient enhancement of a single low-light image without paired data, achieving a good balance between maintaining image details and suppressing noise, and having strong generality and practical application potential.

[0028] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of implementing the above functions. Hereinafter, a personal computer will be taken as an example to illustrate this embodiment and the following embodiments.

[0029] Based on this, the embodiments of the present application provide an unsupervised image enhancement method. Refer to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the unsupervised image enhancement method of the present application.

[0030] In this embodiment, the unsupervised image enhancement method includes step S10: Step S10: Input the obtained low-light image into a pre-trained unsupervised enhancement model for network collaborative processing to obtain an enhanced image, where the unsupervised enhancement model is trained based on a preset loss function and a pre-trained generative adversarial network.

[0031] It should be noted that the unsupervised enhancement model adopted in this embodiment is the RetinexGAN-LIE model. This model is an unsupervised enhancement model that combines the Retinex theory and the idea of generative adversarial network (GAN). Among them, the RetinexGAN-LIE model is trained based on a preset loss function and a pre-trained generative adversarial network.

[0032] Among them, the Retinex theory is a theory used to explain how the human visual system perceives color and brightness. Its core assumption is that the human visual system achieves color constancy perception by separating the illumination and reflectance components of the scene.

[0033] Among them, the core goal of the generative adversarial network is to effectively enhance low-light images in an unsupervised learning framework, while ensuring that the enhancement results have high visual quality and naturalness at both the global brightness and local detail levels.

[0034] In addition, it should be noted that the pre-trained RetinexGAN-LIE model includes an image mapping network, an illumination estimation network, and a reflectance estimation network.

[0035] Optionally, since the traditional Retinex theory provides a physically driven theoretical basis for low-light image enhancement, it is considered that the observed image can be modeled as the element-wise product of the illumination component and the reflectance component :

[0036] Among them, describe the spatial distribution of ambient light, characterize the intrinsic color and texture attributes of an object, and x and y represent the spatial coordinates of pixels in the image. Based on this decomposition, the Retinex theory aims to restore the true details of the reflection component by suppressing the influence of uneven illumination, thereby enhancing the visual effect of low-light images.

[0037] According to the traditional Retinex decomposition theory, the observed image I(x, y) can be modeled as the element-wise product of the illumination component L(x, y) and the reflection component R(x, y). However, existing methods separate L and R through manually designed prior constraints (such as illumination smoothness, reflection sparsity) to suppress illumination interference and restore image details, but it is difficult to adapt to complex and variable natural scenes, especially in extremely low-light or non-uniform illumination conditions, which are prone to decomposition failure.

[0038] Therefore, in this embodiment, through the collaborative design of a dual-branch network and a generative adversarial network, the unsupervised low-light image enhancement task and the Retinex theory constraint are deeply integrated. Specifically, the RetinexGAN-LIE model adopts a parallel and isomorphic dual-branch structure, namely the illumination estimation network and the reflectance estimation network, both of which adopt a lightweight five-layer convolutional network architecture to achieve efficient Retinex decomposition.

[0039] In a feasible implementation manner, step S10 may include steps S11 to S14: Step S11, decompose the low-light image into an intermediate image through the image mapping network; It should be noted that the intermediate image includes but is not limited to the first intermediate image and the second intermediate image.

[0040] In addition, it should also be noted that the image mapping network is a deep learning model for processing and analyzing digital images, which can convert or map the input image into another representation form. In this specific application, the image mapping network is specifically designed to generate two intermediate images based on the input low-light image: the illumination component and the reflectance component.

[0041] Specifically, first, the input low-light image is converted into a form suitable for Retinex decomposition through an image mapping network. Among them, the image mapping network includes deep learning components such as convolutional layers and pooling layers to extract key features in the low-light image and convert them into two intermediate representations that are easy to process. Subsequently, the image mapping network outputs an intermediate image, which includes a first intermediate image and a second intermediate image. The first intermediate image and the second intermediate image are respectively used for subsequent estimation of the illumination component and the reflectance component.

[0042] Step S12, process the intermediate image through the illumination estimation network to obtain the illumination component; It should be noted that the illumination estimation network adopts a lightweight five-layer convolutional network architecture to achieve efficient Retinex decomposition. Among them, the first four layers of the illumination estimation network use the ReLU (Rectified Linear Unit) activation function to extract multi-scale illumination features, and the last layer uses the Sigmoid activation function to output a single-channel illumination map, thereby limiting the output value between [0,1]. Its principle matches the physical characteristics of the illumination component describing the brightness distribution in the Retinex theory.

[0043] Specifically, after obtaining the first intermediate image through step S12, use the illumination estimation network to extract the single-channel illumination component L in the first intermediate image. Since the illumination component is usually related to the overall brightness of the scene, this illumination component is often a grayscale image (single-channel), representing the light intensity distribution at each position in the low-light image.

[0044] Step S13, process the intermediate image through the reflectance estimation network to obtain the reflectance component; It should be noted that the reflectance estimation network usually also adopts a convolutional neural network (CNN) architecture, aiming to separate the reflectance component from the input low-light image. Different from the illumination estimation network, the reflectance estimation network needs to restore the inherent color and texture details of the scene, so its output is a three-channel reflectance map to match the RGB color space.

[0045] Specifically, at the same time, another network - the reflectance estimation network - is responsible for extracting the three-channel reflectance component R from the second intermediate image. Among them, the reflectance component contains the color information and surface attributes of the object, so it is a color image (three-channel).

[0046] Step S14, calculate the initial enhanced image according to the illumination component and the reflectance component.

[0047] Specifically, combining the single-channel illumination component L output by the illumination estimation network and the three-channel reflectance component R output by the reflectance estimation network, the final enhanced image is calculated by the following formula :

[0048] where the symbol ° represents per-pixel multiplication operation, and λ is the illumination correction coefficient, which is used to control the degree of brightness enhancement of the enhanced image represents the finally output enhanced image

[0049] Through the above steps, by combining the Retinex theory with the non-linear brightness mapping function g(L)=Lλ, on the basis of retaining the original structure and color information of the image, the natural enhancement of low-light images is realized, especially improving the visibility and detail restoration ability of the dark area

[0050] Through the above implementation method, the obtained low-illumination image is input into a pre-trained unsupervised enhancement model for network collaborative processing to obtain an enhanced image. Among them, the unsupervised enhancement model is trained based on a preset loss function and a pre-trained generative adversarial network. This method can precisely adjust the illumination distribution of the low-illumination image through the pre-trained unsupervised enhancement model, thereby realizing the efficient enhancement of a single low-illumination image, and also achieving a good balance between maintaining image details and suppressing noise

[0051] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar content as the above-mentioned embodiment one can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 2 , before step S10, the unsupervised image enhancement method further includes steps S01~S02 Step S01, input the obtained low-illumination image into a pre-trained generative adversarial network to obtain a preliminary enhanced image, where the pre-trained generative adversarial network is trained based on the first preset loss function It should be noted that during the training of the RetinexGAN-LIE model, the generative adversarial network (GAN) module plays a key role. Its core goal is to effectively enhance low-illumination images in an unsupervised learning framework, while ensuring that the enhancement results have high visual quality and naturalness at both the global brightness and local detail levels

[0052] Among them, the generative adversarial network module is mainly composed of a generator and a global-local discriminator. The two are collaboratively optimized through adversarial training to enhance the network's ability to model brightness distribution and structural details. As Figure 3 shown Figure 3Schematic diagram of the structure of the generative adversarial network module.

[0053] Among them, the generator adopts an improved U-Net (U-shaped network) structure based on the self-regularized spatial attention mechanism, fully integrating multi-scale features and the prior of the Retinex theory, guiding the network to focus on underexposed regions in the image at different scales, and suppressing the generation of over-enhancement and artifacts.

[0054] The global discriminator is constructed based on the PatchGAN (Patch Generative Adversarial Network) structure, acting on the complete image to judge the overall illumination distribution and structural consistency of the image at a higher resolution.

[0055] The local discriminator is also constructed based on the PatchGAN structure, aiming to randomly crop a fixed number (such as 5) of sub-regions with a size of 128×128 from the image to evaluate the authenticity of microscopic details and the continuity of textures.

[0056] It can be understood that since the initially enhanced image may still have problems of insufficient brightness or detail loss in some regions, especially the images generated under extremely low illuminance conditions may have local over-darkness or uneven contrast, so performing step S01 can further optimize the overall visual effect of the image, ensuring that the final enhanced image meets higher quality standards in terms of global brightness uniformity and local detail clarity.

[0057] In a feasible embodiment, step S01 may include steps S011 to S014: Step S011, through the generator, perform feature enhancement and spatial attention guidance on the low-illuminance image to generate an enhanced candidate image; It should be noted that the generator includes three parts: an encoder, a spatial attention guidance module, and a decoder.

[0058] Specifically, first, the encoder is used to extract multi-layer features of the initially enhanced image. Among them, the encoder consists of 8 convolutional modules, each module contains two 3×3 convolutional layers, a LeakyReLU activation function, and batch normalization (BatchNormalization), and spatial dimensionality reduction is performed through a max-pooling layer. This process gradually extracts information about the image from edges to textures and then to deep semantic features, providing rich context information for subsequent processing.

[0059] Then, a self-regularized spatial attention mechanism is introduced to solve the problems of uneven brightness and loss of dark area details in low-light images. This mechanism uses the illumination component L obtained in the Retinex decomposition to generate an attention map A through normalization, and its calculation is shown in the following formula:

[0060] Among them, Norm(L) represents normalizing the illumination map to the range [0,1], so that the attention value in the darker area of the low-light image is close to 1, while the attention in the bright area approaches 0.

[0061] Subsequently, the attention map is then upsampled to the same spatial size as each layer of feature maps and multiplied with the intermediate feature maps of the encoder or decoder channel by channel to generate an adaptive feature enhancement map. This process not only realizes the dynamic enhancement of dark area features and the redundant suppression of bright area calculations, but also requires no external supervision. Depending only on the Retinex physical decomposition results, it can achieve adaptive modeling of illumination changes.

[0062] Next, through the decoder based on the skip connection mechanism, the adaptive feature enhancement map generated by the encoder is fused with the corresponding decoding layer, effectively restoring image details and maintaining spatial consistency. Finally, the decoder outputs an enhanced candidate image, whose brightness is more natural, details are clearer, and the visual realism is significantly improved.

[0063] Through the above steps, by using the multi-level encoder of the generator, the self-regularized spatial attention mechanism, and the technical means of decoder and feature fusion, more detailed feature enhancement and spatial attention guidance are achieved, and finally an enhanced candidate image with higher quality is generated. Such a process design not only solves the problems of uneven brightness and detail loss in low-light images, but also greatly improves the visual quality and practicality of the images.

[0064] Step S012, through the global discriminator, evaluate the global illumination distribution and structural consistency of the enhanced candidate image to generate a global discrimination result; It should be noted that the global discriminator is usually a convolutional neural network (CNN), which contains multiple convolutional layers, activation functions (such as LeakyReLU) and pooling layers, and gradually extracts the semantic information of the image.

[0065] Specifically, the enhanced candidate image and the real image with normal illumination are simultaneously input into the global discriminator. Through the global discriminator, multi-layer convolutional operations are performed to extract the global features of the image, including illumination distribution, edge information and texture patterns.

[0066] Specifically, the global discriminator calculates the global illumination differences between the enhanced candidate image and the real image, such as the brightness histogram distribution, the smoothness of the illumination transition, etc. If the illumination distribution of the enhanced image is too concentrated or too dispersed, this usually means there are problems with the image quality, so the discriminator will give a lower score for such cases.

[0067] Finally, the final output of the global discriminator is normalized by the sigmoid function to produce a scalar value between 0 and 1. This scalar value represents the authenticity score of the enhanced candidate image relative to the real image, and this scalar value is used as the global discriminant result.

[0068] In addition, the global discriminant result is used as one of the training signals for the generator to adjust the parameters of the generator to make the generated image closer to the real image. Through continuous iterative optimization, the gap between the generated image and the real image is gradually narrowed, thereby improving the quality of the generated image.

[0069] Through the above steps, the global discriminator can not only identify the differences in global features between the enhanced candidate image and the real image, but also effectively convert these differences into specific improvement directions for the generator to learn and optimize.

[0070] Step S013: Use the local discriminator to discriminate multiple local regions randomly cropped from the enhanced candidate image to generate local discriminant results; Specifically, first, input the enhanced candidate image into a preset local discriminator. The local discriminator divides the enhanced candidate image into multiple local regions (for example, by means of a sliding window or random sampling) for separate analysis of these local regions.

[0071] Among them, the enhanced candidate image is segmented into several local regions (such as small blocks or sub - graphs), and these regions can be regularly divided (for example, evenly grid - divided) or based on specific targets (such as focusing on key regions such as faces and object edges).

[0072] Once the enhanced candidate image is segmented into several local regions, for each local region, the local discriminator uses a multi - layer convolutional neural network (CNN) or other feature extraction methods to extract the key features of this region. Among them, the key features include but are not limited to: Texture pattern: The variation pattern of pixel values within the local region.

[0073] Edge information: The clarity and sharpness of the boundaries within the local region.

[0074] Illumination distribution: Whether the brightness change within the local region is natural.

[0075] Color consistency: Whether the color transition within the local area is smooth.

[0076] Subsequently, the local discriminator will independently discriminate each local area, calculating the difference between the enhanced candidate image and the real image in this area. For example, it judges whether the texture of the enhanced candidate image is consistent with the real image, whether the illumination distribution of the enhanced candidate image matches the real image, whether the edges and shapes of the enhanced candidate image are natural and reasonable, etc. It should be noted that if there are significant differences in a certain local area (such as blurred texture, unnatural illumination, edge distortion, etc.), the local discriminator will give a lower score.

[0077] Step S014, based on the global discrimination result and the local discrimination result, optimize the generator, the global discriminator, and the local discriminator, and generate a preliminary enhanced image through the optimized generator.

[0078] Specifically, according to the global discrimination result and the local discrimination result, adjust the parameters in the generator, the global and local discriminators, so that the optimized generator performs global brightness adjustment on the low-light image based on the global discrimination result, and performs local detail optimization on the low-light image based on the local discrimination result, and finally outputs a preliminary enhanced image.

[0079] Through the above steps, combining the global discrimination result and the local discrimination result, comprehensively optimize the low-light image. This process not only solves the problems of uneven global brightness and local detail loss, but also effectively enhances the low-light image, while ensuring that the enhancement result has high visual quality and naturalness at both the global brightness and local detail levels.

[0080] Step S02, combine the preliminary enhanced image and the second preset loss function to train the unsupervised enhancement model to obtain a pre-trained unsupervised enhancement model.

[0081] It should be noted that the second preset loss function includes a projection loss function, a reflectance consistency loss function, and a Retinex loss function.

[0082] Among them, the projection loss function is mainly used to measure the difference between the input low-light image and the intermediate image after its mapping optimization.

[0083] The reflectance consistency loss function is mainly used to constrain the consistency of the reflectance estimation results of paired images.

[0084] The Retinex loss function is mainly used to jointly constrain the estimation processes of the illumination map and the reflectance map.

[0085] It is understandable that since the first preset loss function aims to ensure that the generated image is consistent with the input low-illumination image in terms of basic physical characteristics such as brightness and reflectivity, and follows the decomposition principle of the Retinex theory, therefore, in this embodiment, by comprehensively using these loss functions for joint optimization, the ability of the model to enhance images under complex lighting conditions can be effectively improved.

[0086] In a feasible embodiment, step S02 may include steps S021 to S026: Step S021, preliminarily enhancing the low-illumination image through the generator to generate a preliminarily enhanced image; Step S022, through the image mapping network combined with the projection loss function, performing mapping processing on the low-illumination image and the preliminarily enhanced image to generate a first mapped image and a second mapped image; Step S023, using the illuminance estimation network to decompose the first mapped image and the second mapped image to obtain the illumination component; Step S024, using the reflectance estimation network combined with the reflectance consistency loss function to decompose and align the first mapped image and the second mapped image to obtain the reflectance component; Step S025, using the Retinex loss function to constrain the reconstruction relationship between the illumination component and the reflectance component to obtain an optimized image decomposition result; Step S026, training the unsupervised enhancement model based on the optimized image decomposition result to obtain a preliminarily trained unsupervised enhancement model.

[0087] Specifically, please refer to Figure 4 , Figure 4 which is a schematic diagram of the training process of the RetinexGAN-LIE model. During the training process, first, the input low-illumination image is first preliminarily enhanced through a generator based on the U-Net structure to generate a preliminarily enhanced image . Then, through the image mapping network, and are subjected to mapping processing to respectively generate intermediate images and suitable for Retinex decomposition. Among them, the mapping processing is as follows:

[0088] Among them, P represents the image mapping network, represents the low-illumination image, represents the preliminarily enhanced image, represents the first intermediate image, Represents the second intermediate image.

[0089] Next, the illuminance estimation network and the reflectance estimation network respectively perform Retinex decomposition on these two mapped images to extract the corresponding illumination components ( , ), and reflectance components ( , ). Among them, the decomposition expressions of the illuminance estimation network and the reflectance estimation network are as follows:

[0090] Among them, Z represents the illuminance estimation network, F represents the reflectance estimation network, R represents the reflectance component, L represents the illumination component, represents the first intermediate image, represents the second intermediate image.

[0091] Moreover, during the entire model training process, the network parameters are jointly optimized through three specially designed loss functions to ensure the image enhancement quality and the physical consistency of the Retinex decomposition results. Specifically: Projection loss function: In order to effectively guide the joint optimization process of the parallel isomorphic double-branch network in the unsupervised low-light image enhancement task, this embodiment designs a variety of loss functions to supervise the training of each sub-module. Among them, the projection loss is mainly used to measure the difference between the original low-light image and its mapped and optimized intermediate image. By minimizing this difference, the image mapping network is guided to remove inappropriate features in the low-light image while retaining the structural information, thereby providing a more stable and physically meaningful input image for the subsequent illuminance estimation and reflectance estimation branches. The definition of this projection loss function is shown in the following formula:

[0092] Among them, represents the input low-light image, represents the enhanced image variant, and are the intermediate images generated by the image mapping network. The goal of this projection loss is to make the mapped image retain the structural semantics of the original image while removing components that are irrelevant to Retinex modeling or have strong interference, so that the network focuses more on physically interpretable brightness and reflectance decomposition. Compared with traditional methods, which usually suppress the noise amplification problem during the enhancement process by applying edge smoothing or post-processing denoising operations in the reflectance branch.

[0093] Reflectance Consistency Loss Function: To ensure the network's ability to model the essential attributes of images during the Retinex decomposition process, this embodiment also introduces a reflectance consistency loss to constrain the consistency of the reflectance estimation results for paired images. Since two images and only differ in brightness under the same scene, while the image content remains unchanged, their corresponding reflectance components and should be consistent. This reflectance consistency loss constrains the network output by measuring the difference between the reflectance estimation results, thereby improving the accuracy and stability of reflectance prediction. Its mathematical definition is shown in the following formula:

[0094] where and respectively represent the prediction results of the input image and the image enhanced by the generator in the reflectance estimation network.

[0095] Since and are generated by the generative adversarial network transformation and share the same scene semantics, they should have consistent reflectance properties. By constraining the reflectance output of the network under different brightness inputs through the norm of the reflectance consistency loss function, the robustness of the reflectance branch in modeling the inherent image attributes is effectively improved, and it helps to avoid reflectance offset caused by brightness perturbation.

[0096] Retinex Loss Function: To ensure the rationality and accuracy of the Retinex decomposition results at the physical and perceptual levels, this embodiment introduces a Retinex loss function to jointly constrain the estimation processes of the illumination map and the reflectance map. According to the Retinex theory, the illumination of an image should reflect the overall illumination trend and have global smoothness, while the reflectance should retain the details and texture features of the object and have stronger edge structure expression ability. Therefore, during the training process, by modeling the relationship between the illumination, reflectance, and their combined results and the optimized image, the stability and generalization ability of the Retinex decomposition module can be further improved. This embodiment designs the Retinex loss as a multi-term composite loss, and its specific form is shown in the following formula:

[0097] Among them, L and R represent the estimated illumination map and reflectance map obtained by the network respectively, i represents the intermediate image output by the image mapping module, and stopgrad(L) represents the illumination map with gradient propagation stopped in reflectance estimation, which is used to prevent gradient backpropagation from interfering with illumination calculation and improve the numerical stability of the decomposition process. The first term constrains that the reconstruction result of illumination and reflectance decomposition should be consistent with the input image; the second term emphasizes that the illumination residue should be eliminated as much as possible in the reflectance; the third term constrains the reasonable initialization of illumination through the loss with the initial illumination estimation of ; while the fourth term introduces the Total Variation regularization of illumination , which is used to enhance the smoothness of the illumination map and avoid high-frequency noise and structural artifacts in the estimation results. As shown in the following formula, where the initial illumination is obtained by taking the maximum value of the three color channels (R, G, B) of the input image:

[0098] This definition method can be regarded as a heuristic estimation of brightness, which is simple and efficient, and can provide a good initialization reference for the illumination network. Guided by the above Retinex loss, the network can better follow the physical prior to complete the image decomposition process, thereby significantly improving the interpretability and quality stability of low-light image enhancement without relying on strong supervision information.

[0099] Finally, the projection loss, reflectance consistency loss and Retinex loss are combined in a weighted linear manner, aiming to comprehensively guide the joint optimization process of each module of the network. And by minimizing the overall loss, the RetinexGAN-LIE model can adaptively learn the illumination and reflectance estimation results that are more in line with the Retinex decomposition theory while maintaining physical consistency and structural accuracy, thereby effectively improving the enhancement quality of low-light images. Among them, the form of the overall loss function is defined as follows:

[0100] Among them, , and represent the weighted coefficients of the projection loss function, reflectance consistency loss function and Retinex loss function respectively. By reasonably setting and adjusting these parameters, the relative contributions of different loss terms to the network learning objective can be flexibly controlled during the training process, realizing the optimal trade-off among enhancement effect, decomposition accuracy and physical rationality.

[0101] Based on the above-optimized image decomposition results, use this overall loss function to train the unsupervised enhancement model, gradually adjust the model parameters until the convergence criterion is reached, and finally obtain the preliminarily trained unsupervised enhancement model. This preliminarily trained model already has a certain low-light image enhancement ability and lays the foundation for the further training of the generative adversarial network module.

[0102] In addition, in the test stage, the RetinexGAN-LIE model is significantly simplified, only retaining the core image mapping network, illuminance estimation network, and reflectance estimation network. Given a low-light image, first obtain an intermediate image suitable for Retinex decomposition through the image mapping network. Subsequently, the illuminance estimation network outputs a single-channel illumination component L, and the reflectance estimation network outputs a three-channel reflectance component R. The final enhanced image. Please refer to Figure 5 , Figure 5 For the schematic diagram of the test process of the RetinexGAN-LIE model.

[0103] Through the method of the above embodiments, combined with the preset loss function to train the image mapping network, illuminance estimation network, and reflectance estimation network to train the unsupervised enhancement model, it can effectively enhance the unsupervised enhancement model to better retain the detail information and color consistency of the original image during the image enhancement process, making the enhanced image more natural.

[0104] Based on the second embodiment of the present application, in the third embodiment of the present application, the content that is the same as or similar to the above-mentioned first and second embodiments can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 6 , before step S01, the unsupervised image enhancement method further includes step A01: Step A01, combining the self-feature preservation loss function and the adversarial network loss function, train the generative adversarial network to obtain a pre-trained generative adversarial network.

[0105] It should be noted that the self-feature preservation loss function includes a global feature preservation loss function and a local feature preservation loss function. The adversarial network loss function includes a global adversarial loss function and a local adversarial loss function.

[0106] Among them, the global feature preservation loss function aims to ensure that the overall structure of the generated image is consistent with the input image.

[0107] The local feature preservation loss function is different from the global feature preservation loss function. The local feature preservation loss function focuses on specific parts or details of the image, such as edges, textures, etc.

[0108] The global adversarial loss function is used to evaluate the overall distribution similarity of the generated image relative to the real image set.

[0109] The local adversarial loss function focuses on certain parts of the image rather than the whole. It can enhance the ability of the generative model to generate high-quality details in specific regions.

[0110] It can be understood that the second preset loss function can further ensure that the generated image can not only retain the overall structure and detail features of the original image, but also improve the authenticity and naturalness of the image through the adversarial learning mechanism. Therefore, by performing step A01 and training the generative adversarial network with the second preset loss function, it can be ensured that the unsupervised enhancement model can maximize the retention of important information and visual effects of the image while ensuring the image quality.

[0111] In a feasible embodiment, step A01 may include steps A011 to A016: Step A011, input the low-illumination image into the generator to generate an enhanced candidate image; Step A012, obtain a real image with normal illumination, and input the real image and the enhanced candidate image into a preset global discriminator respectively, and calculate the global adversarial loss function; Step A013, input multiple local regions randomly cropped from the real image and the enhanced candidate image into a preset local discriminator respectively, and calculate the local adversarial loss function; Step A014, extract the deep features of the low-illumination image and the enhanced candidate image, and calculate the global feature retention loss function based on the deep features; Step A015, calculate the local feature difference between the corresponding regions of the low-illumination image and the enhanced candidate image to obtain the local feature retention loss function; Step A016, construct a total loss function according to the global adversarial loss function, the local adversarial loss function, the global feature retention loss function, and the local feature retention loss function, and update the parameters of the generative adversarial network based on the total loss function to obtain a pre-trained generative adversarial network.

[0112] In this embodiment, by adopting a hierarchical adversarial training mechanism, it can effectively realize the organic combination of global brightness correction and local detail optimization in the low-illumination image enhancement process. Please refer to Figure 7 , Figure 7Schematic diagram of the training process of the generative adversarial network. During the training of the generative adversarial network, the first generator receives the low-light input image and generates an enhanced image; the global discriminator is responsible for evaluating the naturalness and brightness distribution of the synthesized image as a whole to ensure that the enhancement result conforms to the global visual characteristics of the real image; while the local discriminator takes multiple local regions of the image as units and carefully detects the authenticity of local textures and edge structures, thereby effectively suppressing artifacts or inconsistent regions that may appear in the image. This global-local discriminator cooperation mechanism enables the generator to not only learn the global brightness mapping relationship but also have the ability to finely model local details, thus breaking through the performance trade-off bottleneck between "brightness enhancement" and "detail preservation" in traditional methods.

[0113] Specifically, in order to further improve the discriminator's ability to model the authenticity of images and provide more stable and discriminative training feedback, a relativistic discriminant mechanism is introduced during the training process of the RetinexGAN-LIE model. In particular, this mechanism is integrated into the global discriminator. This mechanism evaluates whether a sample is "more real than another" by comparing the relative relationship between the discriminator outputs of real samples and generated samples, thus breaking through the limitation of the traditional discriminator's judgment of "absolute true or false". Specifically, the output of the global discriminator is normalized by the sigmoid function and finally calculated based on the relative authenticity difference. Its mathematical expression is shown in the following formula:

[0114] where represents the difference between the feature output of the real image and the expected feature output of the generated image , represents the difference between the feature output of the generated image and the expected feature output of the real image , C(·) represents the discriminator feature output, σ(·) is the sigmoid activation function, and represent the images sampled from the real distribution and the generated distribution respectively, represents the expected feature output of the generated image xf, represents the real image 's expected feature output.

[0115] Moreover, combined with the least squares adversarial loss (Least Squares GAN, LSGAN), this relativistic discriminant mechanism further stabilizes the training process and improves gradient propagation. Among them, the global adversarial loss function of the generator (LG) and the discriminator (LD) at the global level is defined as follows:

[0116] Among them, represents the difference between the characteristic output of the real image xr and the expected characteristic output of the generated image ; represents the difference between the characteristic output of the generated image and the expected characteristic output of the real image xr; represents the expected characteristic output of the generated image ; represents the expected characteristic output of the real image .

[0117] For the local discriminator, its training objective directly adopts the original least - squares adversarial loss function, independently discriminates the local regions randomly cropped (5 patches) from the generated image and the real image, and calculates the local adversarial loss function. The specific calculation expression is:

[0118] Among them, represents the expected characteristic output of the generated image xf, represents the expected characteristic output of the real image , D(·) represents the output of the local discriminator, represents the local image patch sampled from the real image, represents the local image patch sampled from the generated image, represents the loss of the discriminator on the local image, represents the loss of the generator on the local image.

[0119] Through this global - local collaborative adversarial mechanism, this embodiment not only improves the naturalness performance of the enhanced image in terms of overall structure and brightness distribution, but also strengthens the modeling ability of local textures and details, thus effectively alleviating the performance bottleneck existing between brightness enhancement and detail retention in traditional image enhancement methods, and significantly improving the comprehensive performance of the network generation results in subjective visual quality and objective evaluation indicators.

[0120] In addition, in the unsupervised training framework, due to the lack of real paired samples, the generative model is prone to problems such as semantic shift and structural distortion in the enhanced images. To alleviate this problem, in the unsupervised training process of the RetinexGAN-LIE model in this embodiment, a self-feature preservation loss function is introduced. By constraining the semantic consistency between the input low-light image and its enhanced output in the deep feature space, information loss and semantic distortion are effectively suppressed. Specifically, this loss function uses a pre-trained VGG-16 (Visual Geometry Group - 16 layers) network as a feature extractor, fixes the parameters obtained by training on the ImageNet dataset, and calculates the difference in feature maps between the enhanced image and the input image on a specific convolutional layer, thereby measuring the consistency of their high-level semantic information. Let the input low-light image be , and the output image of the generator be G( ), then its self-feature preservation loss is defined as shown in the following formula:

[0121] where ϕi,j(·) represents the feature map extracted by the j-th convolutional layer after the i-th max-pooling layer in the VGG network, and the size of the feature map is width × height . In this embodiment, i = 5 and j = 1 are default selected, that is, the first convolutional layer after the 5th max-pooling layer is used as the measurement standard for semantic features. This layer can capture deeper structure and texture information and has strong semantic discrimination ability.

[0122] In addition, to further improve the model's ability to preserve semantic information in local regions, the same form of feature preservation loss is also introduced in the local discriminator part to constrain the corresponding consistency of local regions from the input image and the enhanced image in the feature space. Through feature constraints at both the global and local levels, RetinexGAN-LIE can still achieve semantic structure fidelity in the enhanced results under the condition of lacking supervision signals. The total loss function of the final generative adversarial network consists of the following four parts:

[0123] where is the feature preservation loss function, is the local feature preservation loss function, is the global adversarial loss function, is the local adversarial loss function.

[0124] By using the method of the above embodiments, training the generative adversarial network in combination with the global feature preservation loss function and the global adversarial loss function, thereby ensuring that the RetinexGAN-LIE model can effectively improve the overall brightness and contrast of the entire image, while ensuring the consistency and authenticity of the overall image structure. Moreover, training the generative adversarial network by using the local feature preservation loss function and the local adversarial loss function enables the RetinexGAN-LIE model to be more refined in detail processing. Therefore, under the synergistic effect of multiple loss functions, the generative adversarial network can not only help the RetinexGAN-LIE model improve the perceptual quality of brightness enhancement, but also maintain the high fidelity of the enhanced image in terms of structure, texture, and semantics.

[0125] It should be noted that the above examples are only for understanding the present application and do not constitute a limitation to the unsupervised image enhancement method of the present application. Based on this technical concept, more forms of simple transformations are within the protection scope of the present application.

[0126] The present application also provides an unsupervised image enhancement device. Please refer to Figure 8 , the unsupervised image enhancement device includes: An image enhancement module 10, configured to input the acquired low-illumination image into a pre-trained unsupervised enhancement model for network collaborative processing to obtain an enhanced image, where the unsupervised enhancement model is trained based on a preset loss function and a pre-trained generative adversarial network.

[0127] The unsupervised image enhancement device provided by the present application adopts the unsupervised image enhancement method in the above embodiments, and can solve the technical problem of how to efficiently enhance a single low-illumination image and achieve a good balance between maintaining image details and suppressing noise. Compared with the prior art, the beneficial effects of the unsupervised image enhancement device provided by the present application are the same as those of the unsupervised image enhancement method provided by the above embodiments, and other technical features in the unsupervised image enhancement device are the same as those disclosed in the above embodiment method, and will not be elaborated here.

[0128] The present application provides an unsupervised image enhancement device, and the unsupervised image enhancement device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the unsupervised image enhancement method in the first embodiment above.

[0129] Next, refer to Figure 9, which shows a schematic structural diagram of an unsupervised image enhancement device suitable for implementing the embodiments of the present application. The unsupervised image enhancement device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions: tablet computers), PMPs (Portable Media Players), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 9 The shown unsupervised image enhancement device is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present application.

[0130] As Figure 9 shown, the unsupervised image enhancement device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory 1002 or the program loaded from the storage device 1003 into the random access memory 1004. In the random access memory 1004, various programs and data required for the operation of the unsupervised image enhancement device are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. The input / output interface 1006 is also connected to the bus. Generally, the following systems may be connected to the input / output interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the unsupervised image enhancement device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an unsupervised image enhancement device with various systems, it should be understood that it is not required to implement or have all the shown systems. Instead, more or fewer systems may be implemented or had.

[0131] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by a processing device 1001, the above functions defined in the methods of the embodiments disclosed in the present application are executed.

[0132] The unsupervised image enhancement device provided in the present application adopts the unsupervised image enhancement method in the above embodiments, and can solve the technical problem of how to efficiently enhance a single low-illumination image and achieve a good balance between maintaining image details and suppressing noise. Compared with the prior art, the beneficial effects of the unsupervised image enhancement device provided in the present application are the same as those of the unsupervised image enhancement method provided in the above embodiments, and other technical features in the unsupervised image enhancement device are the same as those disclosed in the method of the previous embodiment, and will not be elaborated here.

[0133] It should be understood that the various parts disclosed in the present application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0134] As described above, the above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0135] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the unsupervised image enhancement method in the above embodiments.

[0136] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0137] The above computer-readable storage medium can be included in an unsupervised image enhancement device; or it can exist separately without being assembled into the unsupervised image enhancement device.

[0138] The above computer-readable storage medium carries one or more programs. When the one or more programs are executed by an unsupervised image enhancement device, the unsupervised image enhancement device is caused to: input the obtained low-light image into a pre-trained unsupervised enhancement model for network collaborative processing to obtain an enhanced image, where the unsupervised enhancement model is trained based on a preset loss function and a pre-trained generative adversarial network.

[0139] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN: Local Area Network) or a wide area network (WAN: Wide Area Network), or it can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0140] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and this module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks can occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0141] The modules described in the embodiments of this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the unit itself in some cases.

[0142] The readable storage medium provided by this application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for performing the above-mentioned unsupervised image enhancement method, and can solve the technical problem of how to efficiently enhance a single low-illumination image and achieve a good balance between maintaining image details and suppressing noise. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by this application are the same as those of the unsupervised image enhancement method provided by the above embodiments, and will not be elaborated here.

[0143] The present application also provides a computer program product, including a computer program, which implements the steps of the unsupervised image enhancement method as described above when executed by a processor.

[0144] The computer program product provided by the present application can solve the technical problem of how to efficiently enhance a single low-illumination image and achieve a good balance between maintaining image details and suppressing noise. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the unsupervised image enhancement method provided by the above embodiments, and will not be elaborated here.

[0145] The above are only partial embodiments of the present application, and thus do not limit the patent scope of the present application. Any equivalent structural transformation made under the technical concept of the present application by using the content of the specification and drawings of the present application, or direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. An unsupervised image enhancement method, characterized in that, The unsupervised image enhancement method includes: Inputting the obtained low-light image into a pre-trained unsupervised enhancement model for network collaborative processing to obtain an enhanced image, where the unsupervised enhancement model is trained based on a preset loss function and a pre-trained generative adversarial network.

2. The unsupervised image enhancement method according to claim 1, wherein The pre-trained unsupervised enhancement model includes an image mapping network, an illuminance estimation network, and a reflectance estimation network. The step of inputting the obtained low-light image into a pre-trained unsupervised enhancement model for network collaborative processing to obtain an enhanced image includes: Decomposing the low-light image into an intermediate image through the image mapping network; Processing the intermediate image through the illuminance estimation network to obtain an illumination component; Processing the intermediate image through the reflectance estimation network to obtain a reflectance component; Calculating the enhanced image according to the illumination component and the reflectance component.

3. The unsupervised image enhancement method according to claim 1, characterized in that The preset loss function includes a first preset loss function and a second preset loss function. Before the step of inputting the obtained low-light image into a pre-trained unsupervised enhancement model for network collaborative processing to obtain an enhanced image, it includes: Inputting the obtained low-light image into a pre-trained generative adversarial network to obtain a preliminary enhanced image, where the pre-trained generative adversarial network is trained based on the first preset loss function; Combining the preliminary enhanced image and the second preset loss function to train the unsupervised enhancement model to obtain a pre-trained unsupervised enhancement model.

4. The unsupervised image enhancement method according to claim 3, characterized in that The pre-trained generative adversarial network includes a generator, a global discriminator, and a local discriminator. The step of inputting the obtained low-light image into a pre-trained generative adversarial network to obtain a preliminary enhanced image includes: Through the generator, performing feature enhancement and spatial attention guidance on the low-light image to generate an enhanced candidate image; Through the global discriminator, evaluating the global illumination distribution and structural consistency of the enhanced candidate image to generate a global discrimination result; Through the local discriminator, discriminating multiple local regions randomly cropped from the enhanced candidate image to generate a local discrimination result; Based on the global discrimination result and the local discrimination result, optimizing the generator, the global discriminator, and the local discriminator, and generating the preliminary enhanced image through the optimized generator.

5. The unsupervised image enhancement method according to claim 4, wherein, The step of performing feature enhancement and spatial attention guidance on the low-light image through the generator to generate the enhanced candidate image includes: Through the encoder of the generator, performing multi-level feature extraction on the low-light image to generate an intermediate feature map; Generating a spatial attention map based on the illumination component decomposed from the low-light image through normalization processing; Performing a channel-wise multiplication operation on the spatial attention map and the intermediate feature map to generate an adaptive feature enhancement map; Through the decoder of the generator, performing spatial restoration and detail reconstruction on the adaptive feature enhancement map to generate the enhanced candidate image.

6. The unsupervised image enhancement method according to any one of claims 3-4, characterized in that, The first preset loss function includes a self-feature preservation loss function and an adversarial network loss function. Before the step of inputting the obtained low-illumination image into a pre-trained generative adversarial network to obtain a preliminary enhanced image, it includes: Combining the self-feature preservation loss function and the adversarial network loss function to train the generative adversarial network to obtain a pre-trained generative adversarial network.

7. The unsupervised image enhancement method according to claim 6, wherein The self-feature preservation loss function includes a global feature preservation loss function and a local feature preservation loss function, and the adversarial network loss function includes a global adversarial loss function and a local adversarial loss function. The step of combining the self-feature preservation loss function and the adversarial network loss function to train the generative adversarial network to obtain a pre-trained generative adversarial network includes: Inputting the low-illumination image into the generator to generate an enhanced candidate image; Obtaining a real image with normal illumination, and inputting the real image and the enhanced candidate image into the global discriminator respectively to calculate and obtain the global adversarial loss function; Inputting multiple local regions randomly cropped from the real image and the enhanced candidate image into the local discriminator respectively to calculate and obtain the local adversarial loss function; Extracting the deep features of the low-illumination image and the enhanced candidate image, and calculating the global feature preservation loss function based on the deep features; Calculating the local feature difference between the corresponding regions of the low-illumination image and the enhanced candidate image to obtain the local feature preservation loss function; Constructing a total loss function according to the global adversarial loss function, the local adversarial loss function, the global feature preservation loss function, and the local feature preservation loss function, and updating the parameters of the generative adversarial network based on the total loss function to obtain a pre-trained generative adversarial network.

8. The unsupervised image enhancement method according to any one of claims 2-4, characterized in that, The second preset function includes a projection loss function, a reflectance consistency loss function, and a Retinex loss function based on the retina theory; The step of combining the preliminary enhanced image and the second preset loss function to train the unsupervised enhancement model to obtain a pre-trained unsupervised enhancement model includes: Performing preliminary enhancement on the low-illumination image through the generator to generate the preliminary enhanced image; Performing mapping processing on the low-illumination image and the preliminary enhanced image through the image mapping network in combination with the projection loss function to generate a first mapped image and a second mapped image; Using the illumination estimation network to decompose the first mapped image and the second mapped image to obtain the illumination component; Using the reflectance estimation network in combination with the reflectance consistency loss function to decompose and align the first mapped image and the second mapped image to obtain the reflectance component; Using the Retinex loss function to constrain the reconstruction relationship between the illumination component and the reflectance component to obtain an optimized image decomposition result; Training the unsupervised enhancement model based on the optimized image decomposition result to obtain a pre-trained unsupervised enhancement model.

9. An unsupervised image enhancement device, characterized in that, The unsupervised image enhancement device includes: An image enhancement module for inputting the acquired low-light image into a pre-trained unsupervised enhancement model for network collaborative processing to obtain an enhanced image, wherein the unsupervised enhancement model is trained based on a preset loss function and a pre-trained generative adversarial network.

10. An unsupervised image enhancement device, characterized in that, The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the unsupervised image enhancement method according to any one of claims 1 to 8.

11. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the unsupervised image enhancement method according to any one of claims 1 to 8 are implemented.

Images