An image enhancement method, system, device and storage medium

Abstract

The application provides an image enhancement method, system, device and storage medium, relates to the technical field of artificial intelligence, and the method comprises the following steps: preprocessing a to-be-optimized image to generate an initial image; obtaining an initial hidden vector of the initial image through a preset encoder model; obtaining an optimization parameter of the initial image; adjusting the initial hidden vector through the preset encoder model on the basis of the optimization parameter to generate a target hidden vector of the initial image; and inputting the target hidden vector into a preset generator model to generate a target image corresponding to the optimization parameter. The application has the technical effect of improving the image enhancement quality.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to an image enhancement method, system, device, and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, image enhancement technology plays an increasingly important role in the field of image processing. However, in practical applications, due to factors such as shooting conditions and equipment limitations, the acquired images often suffer from insufficient sharpness, loss of detail, and poor quality, which seriously affects the visual effect and subsequent applications of the images. Therefore, how to effectively improve image quality and achieve intelligent image enhancement has become an urgent technical problem to be solved.

[0003] Current technologies typically employ end-to-end training methods based on convolutional neural networks for image enhancement, which directly learn the mapping relationship between low-quality and high-quality images. While this approach can improve image quality to some extent, the lack of a deep understanding and precise control over image features limits the quality of image enhancement. Summary of the Invention

[0004] This application provides an image enhancement method for improving image enhancement quality.

[0005] In a first aspect, this application provides an image enhancement method, the method comprising: preprocessing an image to be optimized to generate an initial image; obtaining an initial latent vector of the initial image through a preset encoder model; obtaining optimization parameters of the initial image; adjusting the initial latent vector based on the optimization parameters through the preset encoder model to generate a target latent vector of the initial image; and inputting the target latent vector into a preset generator model to generate a target image corresponding to the optimization parameters.

[0006] By employing the aforementioned technical solution and designing a preprocessing and encoder-generator framework, accurate conversion from low-quality to high-quality images is achieved. Specifically, a pre-defined encoder model maps the initial image to a latent space, obtaining initial latent vectors. These latent vectors are then specifically adjusted using optimization parameters, ensuring that the generated target latent vectors accurately capture the key features of the image. By inputting the adjusted target latent vectors into the pre-defined generator model, a high-quality target image meeting the optimization requirements can be generated. This latent space optimization-based enhancement strategy not only preserves the original semantic information of the image but also achieves precise control over image quality, effectively improving the image enhancement effect.

[0007] Optionally, the method further includes: constructing a generative adversarial network model comprising a generator network and a discriminator network; setting a first loss function for the generator network and a second loss function for the discriminator network, wherein the first loss function is a weighted combination of a conventional adversarial loss and a discriminator gradient penalty term, and the second loss function is a conventional binary classification cross-entropy loss; and training the generative adversarial network model using a preset image dataset based on the first loss function and the second loss function, and using the trained generator network as a preset generator model.

[0008] By employing the aforementioned technical solution, constructing a generative adversarial network model, and designing a composite loss function for training, this approach effectively enhances the generation capability of the pre-defined generator model. Specifically, the first loss function, combining conventional adversarial loss and a gradient penalty term, enhances the stability of the generator network, while the discriminator network uses binary cross-entropy loss, improving its ability to distinguish between real and generated images. This adversarial learning-based training strategy enables the pre-defined generator model to learn the essential feature distribution of real images, thereby generating more realistic and natural target images in subsequent image enhancement processes, effectively avoiding the distortion problem of the generated results.

[0009] Optionally, training the generative adversarial network model using a preset image dataset based on the first loss function and the second loss function includes: randomly selecting real images from the preset image dataset and sampling random noise from the latent space; inputting the random noise into the generator network to generate corresponding synthetic images; inputting the real images and the synthetic images into the discriminator network respectively, calculating the second loss function, and updating the parameters of the discriminator network; fixing the parameters of the discriminator network, calculating the first loss function, and updating the parameters of the generator network, until the generative adversarial network model converges or reaches a first preset number of iterations.

[0010] By employing the aforementioned technical solution and designing an alternating training strategy to optimize the generative adversarial network (GAN) model, in each iteration, the discriminator network is first trained using both real and synthetic images to enhance its ability to distinguish image authenticity. Then, the generator network is trained with fixed discriminator parameters to improve its generation quality. This training mechanism, based on random sampling and alternating optimization, enables the model to fully learn the distribution characteristics of real images and establish a stable mapping relationship between the latent space and the image space. Through iterative optimization until convergence, the generator network is ensured to possess the ability to convert random noise into high-quality images, laying the foundation for subsequent image enhancement tasks.

[0011] Optionally, the method further includes: using a feature pyramid network structure as the main framework of the encoder model, wherein the pyramid network structure is used to receive the input real image and extract features of the real image at different scales through multiple downsampling and upsampling layers; setting multiple learnable affine transformation layers on the basis of the pyramid network structure, wherein the affine transformation layers receive feature maps output from different levels of the feature pyramid network structure and generate corresponding latent vectors; wherein the number of affine transformation layers is inversely proportional to the scale of the feature maps output by the feature pyramid network structure; constructing a loss function for the encoder model; and training the encoder model based on the loss function according to a preset image dataset, and using the trained encoder model as a preset encoder model.

[0012] By employing the aforementioned technical solution, a feature pyramid network structure is used to effectively extract multi-scale features from images, and a learnable affine transformation layer maps features at different levels to the latent space. The inverse ratio of the number of affine transformation layers to the feature map scale ensures full utilization of high-level semantic features while reducing redundant representations of low-level detail features. This encoder architecture based on multi-scale feature extraction and adaptive transformation accurately captures key information of images at different scales and encodes it into semantically rich latent vector representations, providing a reliable feature foundation for subsequent image enhancement. A loss function-guided training process further optimizes the encoder model's feature extraction and transformation capabilities, enabling it to better serve image enhancement tasks.

[0013] Optionally, the loss function includes image reconstruction loss, perceptual loss, and regularization loss. The step of training the encoder model based on the loss function and a preset image dataset includes: randomly selecting real images from the preset image dataset and inputting them into the encoder model to obtain corresponding latent vectors through forward propagation; inputting the corresponding latent vectors into the preset generator model to obtain a reconstructed image; calculating the image reconstruction loss, perceptual loss, and regularization loss of the reconstructed image; and updating the parameters of the encoder model using a gradient backpropagation algorithm based on the image reconstruction loss, perceptual loss, and regularization loss until the encoder model converges or reaches a second preset number of iterations.

[0014] By employing the aforementioned technical solution and designing a composite loss function that includes image reconstruction loss, perceptual loss, and regularization loss, a comprehensive optimization of the encoder model is achieved. Specifically, the image reconstruction loss ensures pixel-level consistency between the reconstructed image and the original image, the perceptual loss guarantees the preservation of high-level semantic features, and the regularization loss constrains the distribution characteristics of the latent vectors. Through an iterative training process involving random sampling and gradient backpropagation, the encoder model learns the ability to accurately map real images to the latent space while maintaining the visual quality and semantic consistency of the generated results. This multi-objective constrained training strategy enables the encoder model to provide high-quality feature representations for subsequent image enhancement while preserving the essential features of the image.

[0015] Optionally, the step of adjusting the initial latent vector based on the optimized parameters using the preset encoder model to generate the target latent vector of the initial image includes: inputting the optimized parameters into a preset parameter mapping network so that the parameter mapping network calculates the parameter offset vector in the latent space through forward propagation; performing a weighted summation of the initial latent vector and the parameter offset vector to obtain an intermediate latent vector; and inputting the intermediate latent vector into the preset encoder model to calculate the target latent vector through forward propagation of the preset encoder model.

[0016] By employing the aforementioned technical solution and introducing a parameter mapping network and a weighted summation mechanism, precise adjustment of the initial latent vector is achieved. The parameter mapping network converts the optimization parameters into offset vectors in the latent space, generates intermediate latent vectors through weighted combinations with the initial latent vector, and then obtains the target latent vector through further processing by a pre-defined encoder model. This parameter-driven latent vector adjustment strategy not only preserves the basic features of the original image but also enables controllable adjustment of image features based on the optimization parameters. This establishes a precise correspondence between image features and the optimization target in the latent space, providing an effective guarantee for generating high-quality target images.

[0017] Optionally, after inputting the target latent vector into the preset generator model to generate a target image corresponding to the optimization parameters, the method further includes: comparing the target image with the initial image; determining whether the target image meets the preset quality requirements based on the comparison result; if the target image meets the preset quality requirements, then outputting the target image; if the target image does not meet the preset quality requirements, then optimizing the preset encoder model and / or the preset generator model until the target image meets the preset quality requirements.

[0018] By adopting the above technical solution, and establishing a comparison mechanism between the target image and the initial image, along with an iterative optimization strategy, dynamic evaluation and continuous improvement of image enhancement quality are achieved. By comparing the target image with preset quality requirements, the system can promptly identify deficiencies in the enhancement effect and continuously improve the image enhancement effect through targeted optimization of the preset encoder and generator models. This adaptive optimization mechanism based on quality feedback ensures that the final output target image meets the preset quality standards, effectively guaranteeing the reliability and applicability of the image enhancement results.

[0019] Secondly, this application provides an image enhancement system, the system comprising: a processing module, a first acquisition module, a second acquisition module, an adjustment module, and an output module; wherein, the processing module is used to preprocess the image to be optimized to generate an initial image; the first acquisition module is used to acquire the initial latent vector of the initial image through a preset encoder model; the second acquisition module is used to acquire optimization parameters of the initial image; the adjustment module is used to adjust the initial latent vector based on the optimization parameters through the preset encoder model to generate a target latent vector of the initial image; the output module is used to input the target latent vector into a preset generator model to generate a target image corresponding to the optimization parameters.

[0020] Thirdly, this application provides an electronic device that adopts the following technical solution: it includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to enable the electronic device to execute a computer program of any of the above-described image enhancement methods.

[0021] Fourthly, this application provides a computer-readable storage medium that stores a computer program capable of being loaded by a processor and executing any of the above-mentioned image enhancement methods.

[0022] In summary, this application includes at least one of the following beneficial technical effects:

[0023] 1. Through preprocessing and the design of an encoder-generator framework, accurate conversion from low-quality to high-quality images is achieved. Specifically, a pre-defined encoder model maps the initial image to the latent space, obtaining initial latent vectors. These latent vectors are then specifically adjusted using optimization parameters, ensuring that the generated target latent vectors accurately capture the key features of the image. By inputting the adjusted target latent vectors into the pre-defined generator model, a high-quality target image meeting the optimization requirements can be generated. This latent space optimization-based enhancement strategy not only preserves the original semantic information of the image but also achieves precise control over image quality, effectively improving the image enhancement effect. Attached Figure Description

[0024] Figure 1 This is a schematic flowchart of an image enhancement method provided in an embodiment of this application;

[0025] Figure 2 This is a schematic diagram of the structure of an image enhancement system provided in an embodiment of this application;

[0026] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0027] Explanation of reference numerals in the attached figures: 1000, electronic device; 1001, processor; 1002, communication bus; 1003, user interface; 1004, network interface; 1005, memory. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0029] In the description of the embodiments in this application, words such as "illustrative," "for example," or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "illustrative," "for example," or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of words such as "illustrative," "for example," or "for example" is intended to present the relevant concepts in a specific manner.

[0030] Figure 1 This is a schematic flowchart of an image enhancement method provided in an embodiment of this application. Figure 1 As shown, the method includes S101-S105:

[0031] S101, preprocess the image to be optimized to generate an initial image.

[0032] In a specific implementation, the image to be optimized can be a face image in RGB format. Since images captured by different devices may have differences in brightness and contrast, directly inputting the original image may affect the stability and robustness of subsequent processing. Therefore, it is necessary to preprocess the image to be optimized, converting it into a standardized initial image.

[0033] Specifically, for each pixel value in the image to be optimized, it can be transformed from the original range of [0, 255] to the range of [-1, 1] by dividing the pixel value by 127.5 and then subtracting 1. This preprocessing method normalizes the pixel values ​​of different images to the same numerical range, enhancing the model's robustness to different brightness and contrast levels, allowing the subsequent encoder model to better extract image features. Furthermore, this preprocessing method is reversible; when finally generating the target image, an inverse transformation can be used to restore the original pixel value range, ensuring the practicality of the processing results.

[0034] S102, obtain the initial latent vector of the initial image through the preset encoder model.

[0035] Latent vectors can be understood as a set of numerical values; in this invention, they are specifically 18 sets of 512-dimensional vectors, each containing 512 numbers. These numbers represent the encoded representation of the image in the feature space.

[0036] For example, suppose we have a human face image, which contains many visual features (such as skin color, facial feature position, expression, etc.). Latent vectors are a way of encoding these visual features into numerical representations. For example, in a 512-dimensional vector, some dimensions might be responsible for encoding the width of the face, some for encoding the size of the eyes, and some for encoding the smoothness of the skin, etc.

[0037] In this invention, the 18 sets of latent vectors are divided as follows: the first 3 sets (0-2 layers) of vectors mainly encode basic features, such as face shape and facial feature layout; the middle 4 sets (3-6 layers) of vectors encode medium-scale features, such as eye shape and nose outline; and the last 12 sets (7-18 layers) of vectors encode fine features, such as skin texture and pore details.

[0038] When the preset generator model receives this set of latent vectors, it "decodes" the corresponding visual features based on these values, thereby reconstructing the image. By adjusting the values ​​in the latent vectors, various features of the generated image can be precisely controlled, which provides an operational basis for subsequent image enhancement.

[0039] Specifically, after obtaining the initial image, it needs to be mapped to the latent space of the preset generator model for processing. Since direct optimization in the image space suffers from high dimensionality and strong non-convexity, converting the initial image into latent vectors through the preset encoder model simplifies the subsequent optimization process. Specifically, the preset encoder model uses a feature pyramid network structure as its main framework, which includes multiple downsampling and upsampling layers to extract multi-scale features from the initial image. For an initial image with an input size of 256×256, its corresponding latent space dimension is 18×512, containing 18 independent 512-dimensional vectors. The preset encoder model incorporates 18 learnable affine transformation layers, which receive feature maps from different levels of the feature pyramid network: layers 0-2 use the smallest output feature maps, layers 3-6 use medium-sized feature maps, and layers 7-18 use the largest feature maps. Through this hierarchical processing, the preset encoder model can more accurately capture the feature information of the initial image at different scales, thereby generating more accurate initial latent vectors.

[0040] S103, obtain the optimization parameters of the initial image.

[0041] In one example, after obtaining the initial latent vectors, it is necessary to determine the specific direction and extent of image enhancement, which requires obtaining the optimization parameters of the initial image. The optimization parameters are a set of values ​​used to guide the adjustment of the latent vectors, including brightness parameters, sharpness parameters, and texture parameters.

[0042] Specifically, the initial image is analyzed to calculate its overall brightness value, image sharpness index, and texture complexity index. For the brightness parameter, it is determined by calculating the difference between the average pixel value of the initial image and the standard brightness value; for the sharpness parameter, the Laplacian operator is used to calculate the image's edge sharpness score; for the texture parameter, the gray-level co-occurrence matrix is ​​used to analyze the image's texture features. These parameters are normalized to the interval [-1, 1], where 0 indicates no adjustment is needed, positive values ​​indicate enhancement is needed, and negative values ​​indicate attenuation is needed.

[0043] For example, when the initial image is detected to be too dark, the brightness parameter is set to a positive value; when the image is blurry, the sharpness parameter is set to a positive value. These optimization parameters not only reflect the aspects of the initial image that need improvement, but also quantify the degree of improvement, providing precise guidance for subsequent latent vector adjustments.

[0044] S104, based on the optimized parameters, adjusts the initial latent vector through a preset encoder model to generate the target latent vector of the initial image.

[0045] In one example, after obtaining the optimization parameters, these parameters need to be translated into specific adjustments in the W+ latent space to precisely modify the initial latent vectors. Compared to the traditional Z-space, the W+ latent space newly introduced in the StyleGAN series models has better decoupling and editability, making feature adjustments at different levels more independent and controllable. The StyleGAN series models are a series of generative adversarial network models developed by the NVIDIA research team, mainly used for high-quality image generation, and have achieved breakthroughs, especially in the field of face image generation.

[0046] First, the optimized parameters are input into a preset parameter mapping network, which is composed of a multilayer perceptron. By learning the mapping relationship between the optimized parameters and the W+ latent space adjustment, a 18×512-dimensional parameter offset vector is output, where the 18 independent 512-dimensional vectors correspond to different levels of style control in StyleGAN2.

[0047] Specifically, for latent vectors in the W+ latent space, the parameter mapping network generates offsets of corresponding dimensions. This fully utilizes the decoupling characteristics of different levels of features in the W+ latent space, ensuring that the sharpness parameter primarily affects the offset values ​​at higher levels, while the brightness parameter primarily affects the offset values ​​at mid-to-lower levels. Next, the initial latent vector and the parameter offset vector are weighted and summed in the W+ latent space to obtain an intermediate latent vector. The weight coefficients control the intensity of the adjustment. To ensure that the adjusted features still conform to the distribution characteristics of the W+ latent space, the intermediate latent vector is input into a preset encoder model for re-encoding. The target latent vector is then calculated through forward propagation of the preset encoder model. This dual processing mechanism based on the W+ latent space enables more precise feature control and a more natural image editing effect.

[0048] The W+ latent space can be understood as an 18-layer image feature control system, with each layer responsible for controlling different aspects of the image. This is like an 18-layer mixing console, with 512 fine-tuning knobs on each layer. Specifically: the bottom layer control (layers 0-2) is like adjusting the overall layout; controlling basic features such as facial features and head orientation; similar to adjusting the basic composition of a painting.

[0049] Mid-level controls (layers 3-6): like adjusting knobs for the main facial features; controlling eye size, nose shape, lip contour, etc.; similar to refining the main elements of the painting. High-level controls (layers 7-18): like adjusting knobs for details; controlling skin texture, pore details, local lighting, etc.; similar to adding fine details to the painting.

[0050] Each layer's 512-dimensional vector acts like 512 fine-tuning knobs, each controlling a specific visual feature. When you want to adjust a feature of an image, you only need to adjust the relevant "knob" at the corresponding layer without affecting other features. This embodies the feature decoupling of the W+ latent space. This hierarchical, multi-dimensional control mechanism makes image editing more precise and controllable, which is the key advantage of the W+ latent space compared to the traditional Z latent space.

[0051] Based on the above embodiments, as an optional implementation, in S104, adjusting the initial latent vector using a preset encoder model based on the optimized parameters to generate the target latent vector of the initial image specifically includes S401-S403:

[0052] S401, the optimized parameters are input into the preset parameter mapping network so that the parameter mapping network can calculate the parameter offset vector in the latent space through forward propagation.

[0053] S402, the initial latent vector and the parameter offset vector are weighted and summed to obtain the intermediate latent vector.

[0054] S403, input the intermediate latent vector into the preset encoder model, and calculate the target latent vector through the forward propagation of the preset encoder model.

[0055] In one example, to achieve precise adjustment of the initial latent vector, this implementation introduces a two-stage adjustment strategy based on parameter mapping and encoder optimization. First, the optimized enhancement parameters are input into a pre-defined parameter mapping network. This network employs a multilayer perceptron structure with three fully connected layers, each connected by a LeakyReLU activation function. The input enhancement parameters are mapped to parameter offset vectors of the same dimension as the W+ latent space. This mapping process ensures that the enhancement parameters generate meaningful adjustment directions in the latent space. Subsequently, the initial latent vector and the parameter offset vector are weighted and summed, with a weight coefficient λ set to 0.8 to balance the contributions of the original features and the enhancement effect, resulting in an intermediate latent vector that incorporates the enhancement information. Finally, the intermediate latent vector is input into a pre-trained pre-defined encoder model. Through the encoder's feature pyramid network and affine transformation layer, the final target latent vector is generated. This two-stage adjustment strategy fully leverages the advantages of the pre-defined encoder model in image feature extraction and latent space mapping, enabling precise enhancement of detail and quality while preserving the original visual features of the image.

[0056] S105: Input the target latent vector into the preset generator model to generate the target image with corresponding optimized parameters.

[0057] After obtaining the adjusted target latent vectors, they need to be transformed back into image space to achieve the final enhancement effect. The preset generator model adopts the generator structure of StyleGAN2, which can accurately decode the latent vectors in the W+ latent space into high-quality images.

[0058] Specifically, the 18×512 dimensional latent vector of the target image is input into a pre-defined generator model. Through multiple adaptive instance normalization layers within the model, style information from different levels is progressively injected into the generation process. The latent vectors at lower levels (layers 0-2) control the basic structural features of the generated image, those at mid-levels (layers 3-6) control larger-scale feature changes, and those at higher levels (layers 7-18) control fine texture details. The pre-defined generator model integrates these feature information progressively through upsampling and convolution operations, ultimately outputting a target image with a resolution of 256×256. To ensure the generated image meets normal display requirements, the output pixel values ​​are inversely transformed from the [-1, 1] interval to the [0, 255] interval, i.e., the pixel value is incremented by 1 and multiplied by 127.5.

[0059] Based on the above embodiments, as an optional implementation method, constructing a preset generator model specifically includes S201-S203:

[0060] S201, Construct a generative adversarial network model that includes a generator network and a discriminator network;

[0061] To obtain a high-quality pre-built generator model, a generative adversarial network (GAN) model needs to be constructed and trained first. This model adopts the StyleGAN2 network architecture, comprising two main components: a generator network and a discriminator network. The generator network uses a style-based generative architecture, consisting of a mapping network and a synthesis network. The mapping network maps the input random noise vector to the W+ latent space, while the synthesis network transforms the latent vectors in the W+ latent space into realistic images through multiple adaptive instance normalization layers.

[0062] Specifically, the generator network contains 18 style injection layers, each receiving a 512-dimensional style vector as input, and generates a 256×256 resolution image through progressive upsampling and convolution operations. The discriminator network adopts a symmetrical downsampling structure, progressively extracting features from the input image through multiple residual blocks and convolutional layers, and finally outputting a discrimination score.

[0063] During training, the generator and discriminator networks continuously optimize through adversarial learning: the generator learns to generate more realistic images to deceive the discriminator, while the discriminator learns to distinguish between real and generated images. This adversarial learning mechanism enables the generator network to learn the distribution characteristics of real images, producing high-quality generated results. Meanwhile, the StyleGAN2-based network architecture ensures the diversity and controllability of the generated images, laying the foundation for subsequent image augmentation tasks.

[0064] S202, set the first loss function for the generator network and the second loss function for the discriminator network. The first loss function is a weighted combination of the conventional adversarial loss and the discriminator gradient penalty term, and the second loss function is the conventional binary classification cross-entropy loss.

[0065] After constructing the generative adversarial network (GAN) model, a suitable loss function needs to be designed to guide the network's training process. For the generator network, its first loss function is a weighted combination of conventional adversarial loss and discriminator gradient penalty term. Conventional adversarial loss measures the ability of the generated image to deceive the discriminator, and is calculated as the negative logarithm of the probability that the generated image is classified as a real image by the discriminator. The discriminator gradient penalty term, by calculating the gradient norm of the discriminator with respect to the input image, is used to constrain the discriminator's Lipschitz continuity and improve training stability; its calculation formula is the mean squared error between the gradient norm and the target value of 1. These two losses are weighted by a weight coefficient λ, where λ is set to 10 to balance the contributions of the two losses.

[0066] For the discriminator network, its second loss function employs a conventional binary classification cross-entropy loss to measure the discriminator's ability to distinguish between real and generated images. Specifically, the label for real images is set to 1, and the label for generated images is set to 0. The discriminative performance is measured by calculating the cross-entropy between the discriminator's output and the real labels. This loss function design ensures that the generator can produce high-quality images while guaranteeing that the discriminator has good discriminative ability, and also improves the stability of training through a gradient penalty mechanism.

[0067] S203, based on the first and second loss functions, train the generative adversarial network model using a preset image dataset, and use the trained generator network as the preset generator model.

[0068] After setting the loss function, the parameters of the generative adversarial network (GAN) model need to be optimized through iterative training. The training process employs an alternating optimization strategy, with each iteration first training the discriminator network and then the generator network. Specifically, a batch of real images is randomly sampled from a pre-defined image dataset containing 500,000 high-quality face images, while the same number of random noise vectors are sampled from a standard normal distribution. These noise vectors are first transformed into the W+ latent space by the generator network's mapping network, and then synthesized into synthetic images by the synthesis network. The real and synthetic images are then input into the discriminator network to obtain discrimination scores, and a second loss function is calculated based on the binary cross-entropy loss. The parameters of the discriminator network are updated using backpropagation to better distinguish between real and synthetic images. Subsequently, with the discriminator network parameters fixed, the generator network is reused to generate synthetic images, and a first loss function containing conventional adversarial loss and gradient penalty terms is calculated. The parameters of the generator network are updated again using backpropagation to improve its ability to generate real images. This alternating training process continues until the model reaches convergence or completes 1 million iterations, at which point the trained generator network is used as the preset generator model.

[0069] Based on the above embodiments, as an optional implementation method, constructing a preset encoder model specifically includes S301-S304:

[0070] S301 uses a feature pyramid network structure as the main framework of the encoder model. The pyramid network structure is used to receive the input real image and extract features of the real image at different scales through multiple downsampling and upsampling layers.

[0071] To achieve efficient feature extraction and encoding of the input image, the encoder model employs a feature pyramid network structure as its main framework. This structure is designed based on the multi-scale characteristics of image features, effectively capturing information at different levels, from global structure to local details.

[0072] Specifically, the Feature Pyramid Network first progressively reduces the spatial resolution of the feature maps through a bottom-up downsampling path while increasing the number of feature channels, forming multi-scale feature maps ranging from 256×256 to 8×8. Each downsampling layer consists of a residual block and a convolutional layer with a stride of 2. The residual block contains two 3×3 convolutional layers and skip connections to extract feature information at the current scale. Throughout the downsampling path, the number of feature channels gradually increases from 64 to 512 to enhance the network's feature representation capabilities. Next, a top-down upsampling path progressively fuses high-level semantic features into low-level features. Each upsampling layer doubles the feature map size through nearest-neighbor interpolation and fuses it with downsampling feature maps of the same scale. This bidirectional feature transfer mechanism ensures the effective integration of features at different scales, enabling the encoder to comprehensively understand the visual content of the input image. Finally, the Feature Pyramid Network outputs 18 sets of 512-dimensional feature vectors, corresponding to style information at different levels in the W+ latent space.

[0073] S302, based on the pyramid network structure, sets up multiple learnable affine transformation layers. The affine transformation layers receive feature maps output from different levels of the feature pyramid network structure and generate corresponding latent vectors. The number of affine transformation layers is inversely proportional to the scale of the feature maps output by the feature pyramid network structure.

[0074] To map the multi-scale features extracted by the feature pyramid network to the W+ latent space, learnable affine transformation layers need to be added after the feature maps at different levels. The design of these affine transformation layers follows the principle of decreasing feature complexity, that is, lower-level feature maps contain richer visual information, thus requiring more affine transformation layers for feature transformation.

[0075] Specifically, four affine transformation layers are configured for feature maps with resolutions ranging from 256×256 to 32×32, three for 16×16, and two for 8×8. Each affine transformation layer contains a fully connected layer and a normalization layer. The fully connected layer transforms the input features into a 512-dimensional vector, while the normalization layer ensures that the output vector conforms to the distribution characteristics of the W+ latent space. During feature transformation, the affine transformation layer transforms the spatial feature map into a compact feature vector through learnable weight matrices and bias terms. Lower-level affine transformation layers progressively refine the feature representation through cascading, while higher-level, fewer affine transformation layers primarily focus on the transformation of abstract semantic features. This hierarchical affine transformation design not only improves the accuracy of feature mapping but also maintains computational efficiency. Ultimately, the outputs of all affine transformation layers together constitute a complete 18×512-dimensional latent vector, accurately mapping the representation of the input image in the W+ latent space.

[0076] S303, construct the loss function for the encoder model.

[0077] To train the encoder model to accurately map the input image to the W+ latent space, a multi-objective loss function is needed to guide model optimization. This loss function comprises three key components: pixel reconstruction loss, feature reconstruction loss, and perceptual loss. The pixel reconstruction loss uses the L2 norm to calculate the difference between the encoder-reconstructed image and the input image in the RGB space, ensuring pixel-level accuracy. The feature reconstruction loss measures the similarity between the reconstructed image and the input image in the semantic feature space by calculating the L2 distance between features from different layers of the pre-trained VGG19 network, specifically comparing feature maps from layers conv1_1, conv2_1, conv3_1, and conv4_1. The perceptual loss utilizes features from the intermediate layers of the discriminator network to evaluate the realism and detail fidelity of the reconstructed image. These three losses are weighted by coefficients α, β, and γ, where α is set to 1.0, β to 0.8, and γ to 0.1 to balance the contributions of different loss terms.

[0078] Specifically, the encoder first maps the input image to latent vectors in the W+ latent space, then reconstructs the image using a pre-defined generator model, and calculates the three losses mentioned above. This multi-objective loss function design ensures pixel-level accuracy of the reconstructed image while preserving semantic features and improving visual quality.

[0079] S304, Based on the loss function, the encoder model is trained according to the preset image dataset, and the trained encoder model is used as the preset encoder model.

[0080] After constructing the loss function, the parameters of the encoder model need to be optimized through an iterative training process. The training process adopts an end-to-end optimization strategy based on a pre-defined image dataset, and each iteration includes two stages: forward propagation and back propagation.

[0081] Specifically, firstly, a batch of real images is randomly sampled from a pre-defined image dataset containing 500,000 high-quality face images, and these images are input into the encoder model. The encoder model generates corresponding 18×512-dimensional latent vectors through a feature pyramid network and an affine transformation layer. Then, these latent vectors are input into a pre-trained pre-defined generator model to generate reconstructed images. After obtaining the reconstructed images, three key losses are calculated: the image reconstruction loss uses the L2 norm to measure the difference between the reconstructed image and the input image in pixel space; the perceptual loss evaluates semantic consistency through multi-layer features of a pre-trained VGG19 network; and the regularization loss constrains the distribution characteristics of the latent vectors, ensuring they conform to the statistical properties of the W+ latent space. These three losses are weighted and combined using pre-defined weight coefficients to form the final optimization objective. Then, the gradient of the loss function with respect to the model parameters is calculated using the gradient backpropagation algorithm, and the parameters of the encoder model are updated using the Adam optimizer. This training process continues until the model reaches convergence or completes 500,000 iterations. The trained encoder model is then used as the pre-defined encoder model.

[0082] After inputting the target latent vector into the preset generator model to generate the target image with corresponding optimized parameters, the process also includes:

[0083] The target image is compared with the initial image; based on the comparison result, it is determined whether the target image meets the preset quality requirements; if the target image meets the preset quality requirements, the target image is output; if the target image does not meet the preset quality requirements, the preset encoder model and / or preset generator model are optimized until the target image meets the preset quality requirements.

[0084] In one example, to ensure that the quality of the generated image meets the requirements of the actual application, quality assessment and model optimization are required after obtaining the target image.

[0085] First, the generated target image is compared with the initial input image using a multi-dimensional comparative analysis. This includes calculating the Structural Similarity Index (SSIM) to assess the degree of image structure preservation, the Peak Signal-to-Noise Ratio (PSNR) to measure the degree of image distortion, and calculating a perceptual quality score using a pre-trained image quality assessment network. These metrics collectively constitute the evaluation criteria for preset quality requirements, with an SSIM threshold set at 0.85, a PSNR threshold at 30 dB, and a perceptual quality score threshold at 7.5. If all metrics of the target image exceed the corresponding thresholds, it is considered to meet the preset quality requirements and can be directly output and used. If any metrics fail to meet the standards, targeted optimization of the preset encoder and generator models is required.

[0086] Specifically, by adding quality-oriented constraints to the loss function, such as increasing the loss weight corresponding to unmet criteria, while maintaining the contribution of other loss terms, the model parameters are fine-tuned using mini-batch stochastic gradient descent. This optimization process continues, with the target image regenerated and quality metrics evaluated after each iteration, until all metrics meet the preset requirements.

[0087] Based on the above method, this application also discloses an image enhancement system, such as... Figure 2 As shown, Figure 2 This is a schematic diagram of an image enhancement system provided in an embodiment of this application. The system includes: a processing module, a first acquisition module, a second acquisition module, an adjustment module, and an output module. The processing module is used to preprocess the image to be optimized to generate an initial image. The first acquisition module is used to acquire the initial latent vector of the initial image through a preset encoder model. The second acquisition module is used to acquire the optimization parameters of the initial image. The adjustment module is used to adjust the initial latent vector based on the optimization parameters through the preset encoder model to generate the target latent vector of the initial image. The output module is used to input the target latent vector into a preset generator model to generate a target image with corresponding optimization parameters.

[0088] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0089] Please see Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 3 As shown, the electronic device 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002.

[0090] The communication bus 1002 is used to realize the connection and communication between these components.

[0091] The user interface 1003 may include a display screen and a camera. Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface.

[0092] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0093] The processor 1001 may include one or more processing cores. The processor 1001 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and by calling data stored in the memory 1005. Optionally, the processor 1001 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 1001 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 1001 and may be implemented as a separate chip.

[0094] The memory 1005 may include random access memory (RAM) or read-only memory. Optionally, the memory 1005 may include a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. Figure 3 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for an image enhancement method.

[0095] exist Figure 3In the electronic device 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 1001 can be used to call an application program storing an image enhancement method in the memory 1005. When executed by one or more processors, the electronic device performs one or more of the methods described in the above embodiments.

[0096] An electronic device readable storage medium stores instructions that, when executed by one or more processors, cause the electronic device to perform one or more of the methods described in the above embodiments.

[0097] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0098] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some service interfaces; indirect couplings or communication connections between devices or units may be electrical or other forms.

[0100] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0102] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0103] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. An image enhancement method, characterized in that, The method includes: The process involves preprocessing the image to be optimized to generate an initial image; obtaining the initial latent vector of the initial image using a preset encoder model; obtaining optimization parameters for the initial image; and adjusting the initial latent vector based on the optimization parameters using the preset encoder model to generate the target latent vector of the initial image. This includes: inputting the optimization parameters into a preset parameter mapping network, so that the parameter mapping network calculates the parameter offset vector in the latent space through forward propagation; the parameter mapping network contains three fully connected layers, each using a LeakyReLU activation function, mapping the input optimization parameters to a parameter offset vector of the same dimension as the W+ latent space; and then mapping the initial latent vector to the parameter offset vector. Weighted summation is performed to obtain intermediate latent vectors; these intermediate latent vectors are input into the preset encoder model, and the target latent vector is calculated through forward propagation of the preset encoder model; the target latent vector is input into the preset generator model to generate a target image corresponding to the optimized parameters; the target latent vector is input into the preset generator model, and style information at different levels is gradually injected into the generation process through multiple adaptive instance normalization layers in the model; wherein, low-level latent vectors control the basic structural features of the generated image, mid-level latent vectors control large-scale feature changes, and high-level latent vectors control fine texture details; the preset generator model integrates feature information through upsampling and convolution operations to output a target image; The method further includes: constructing a generative adversarial network (GAN) model comprising a generator network and a discriminator network; setting a first loss function for the generator network and a second loss function for the discriminator network, wherein the first loss function is a weighted combination of a conventional adversarial loss and a discriminator gradient penalty term, and the second loss function is a conventional binary classification cross-entropy loss; training the GAN model using a preset image dataset based on the first and second loss functions, and using the trained generator network as a preset generator model, including: randomly selecting real images from the preset image dataset and sampling random noise from the latent space; inputting the random noise into the generator network to generate corresponding synthetic images; inputting the real images and the synthetic images into the discriminator network respectively, calculating the second loss function, and updating the parameters of the discriminator network; fixing the parameters of the discriminator network, calculating the first loss function, and updating the parameters of the generator network, until the GAN model converges or reaches a first preset number of iterations; The method further includes: using a feature pyramid network structure as the main framework of the encoder model, wherein the pyramid network structure receives the input real image and extracts features of the real image at different scales through multiple downsampling and upsampling layers; setting multiple learnable affine transformation layers on the basis of the pyramid network structure, wherein the affine transformation layers receive feature maps output from different levels of the feature pyramid network structure and generate corresponding latent vectors; wherein the number of affine transformation layers is inversely proportional to the scale of the feature maps output by the feature pyramid network structure; constructing a loss function for the encoder model; and training the encoder model based on the loss function according to a preset image dataset, using the trained encoder model as a preset... An encoder model; the loss function includes image reconstruction loss, perceptual loss, and regularization loss. The step of training the encoder model based on the loss function and a preset image dataset includes: randomly selecting real images from the preset image dataset and inputting them into the encoder model to obtain corresponding latent vectors through forward propagation; inputting the corresponding latent vectors into the preset generator model to obtain a reconstructed image; calculating the image reconstruction loss, perceptual loss, and regularization loss of the reconstructed image; and updating the parameters of the encoder model using a gradient backpropagation algorithm based on the image reconstruction loss, perceptual loss, and regularization loss until the encoder model converges or reaches a second preset number of iterations.

2. The image enhancement method according to claim 1, characterized in that, After inputting the target latent vector into the preset generator model to generate a target image corresponding to the optimization parameters, the method further includes: comparing the target image with the initial image; determining whether the target image meets the preset quality requirements based on the comparison result; if the target image meets the preset quality requirements, then outputting the target image; if the target image does not meet the preset quality requirements, then optimizing the preset encoder model and / or the preset generator model until the target image meets the preset quality requirements.

3. An image enhancement system, characterized in that, The system includes: a processing module, a first acquisition module, a second acquisition module, an adjustment module, and an output module; wherein, the processing module is used to preprocess the image to be optimized to generate an initial image; the first acquisition module is used to acquire the initial latent vector of the initial image through a preset encoder model; the second acquisition module is used to acquire the optimization parameters of the initial image; the adjustment module is used to adjust the initial latent vector based on the optimization parameters through the preset encoder model to generate the target latent vector of the initial image, including: inputting the optimization parameters into a preset parameter mapping network, so that the parameter mapping network obtains the parameters in the latent space through forward propagation. The parameter mapping network comprises three fully connected layers, each using the LeakyReLU activation function to map the input optimization parameters to parameter offset vectors of the same dimension as the W+ latent space. The initial latent vector and the parameter offset vector are weighted and summed to obtain an intermediate latent vector. This intermediate latent vector is input to the preset encoder model, and the target latent vector is calculated through forward propagation. The output module is used to input the target latent vector into a preset generator model to generate a target image corresponding to the optimization parameters. The target latent vector is input into the preset generator model, and through multiple adaptive instance normalization layers in the model, different levels of wind... The information is gradually injected into the generation process; among them, low-level latent vectors control the basic structural features of the generated image, mid-level latent vectors control large-scale feature changes, and high-level latent vectors control fine texture details; the preset generator model integrates feature information through upsampling and convolution operations to output a target image; the method also includes: constructing a generative adversarial network model containing a generator network and a discriminator network; setting a first loss function for the generator network and a second loss function for the discriminator network, wherein the first loss function is a weighted combination of conventional adversarial loss and discriminator gradient penalty term, and the second loss function is conventional binary classification cross-entropy loss; in the first loss function Based on the first loss function and the second loss function, the generative adversarial network model is trained using a preset image dataset. The trained generator network is used as the preset generator model. The process includes: randomly selecting real images from the preset image dataset and sampling random noise from the latent space; inputting the random noise into the generator network to generate corresponding synthetic images; inputting the real images and the synthetic images into the discriminator network respectively, calculating the second loss function, and updating the parameters of the discriminator network; fixing the parameters of the discriminator network, calculating the first loss function, and updating the parameters of the generator network until the generative adversarial network model converges or reaches a first preset number of iterations.The method further includes: using a feature pyramid network structure as the main framework of the encoder model, wherein the pyramid network structure receives the input real image and extracts features of the real image at different scales through multiple downsampling and upsampling layers; setting multiple learnable affine transformation layers on the basis of the pyramid network structure, wherein the affine transformation layers receive feature maps output from different levels of the feature pyramid network structure and generate corresponding latent vectors; wherein the number of affine transformation layers is inversely proportional to the scale of the feature maps output by the feature pyramid network structure; constructing a loss function for the encoder model; and training the encoder model based on the loss function according to a preset image dataset, using the trained encoder model as a preset... An encoder model; the loss function includes image reconstruction loss, perceptual loss, and regularization loss. The process of training the encoder model based on a preset image dataset, using the loss function, includes: randomly selecting real images from the preset image dataset and inputting them into the encoder model to obtain corresponding latent vectors through forward propagation; inputting the corresponding latent vectors into the preset generator model to obtain a reconstructed image; calculating the image reconstruction loss, perceptual loss, and regularization loss of the reconstructed image; and updating the parameters of the encoder model using a gradient backpropagation algorithm based on the image reconstruction loss, perceptual loss, and regularization loss until the encoder model converges or reaches a second preset number of iterations.

4. An electronic device, characterized in that, The device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-2.

5. A computer-readable storage medium, characterized in that, The computer program is stored that can be loaded by a processor and executed as described in any one of claims 1-2.

Images