Image generation model training method and device, equipment and storage medium

Abstract

The invention discloses an image generation model training method and device, equipment and a storage medium, and relates to the technical field of computers. The method comprises the steps that at least one training sample of an image generation model is acquired, each training sample comprises a first sample image and a second sample image, the first sample image is an image meeting the resolution requirement, and the second sample image is an image obtained through actual shooting; performing degradation processing on the first sample image to obtain a first sample image after degradation processing; generating a first prediction image corresponding to the first sample image after degradation processing through an image generation model according to the first sample image after degradation processing; and according to the difference between the first prediction image and the first sample image and the difference between the first prediction image and the second sample image, adjusting parameters of an image generation model to obtain a trained image generation model. According to the invention, the high resolution and authenticity of the generated image are considered at the same time, and the image generation quality is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and particularly relates to a training method and device of an image generation model, equipment and a storage medium. BACKGROUND

[0002] Image super-resolution technology is an important research topic in the field of computer vision, which aims to restore a high-resolution image from a low-resolution image.

[0003] In the related art, an image super-resolution image with strong realism can be generated through an adversarial neural network, the adversarial neural network includes a generator and a discriminator, the generator is used to receive a low-resolution image, extract image features through a convolution layer, combine a residual network or an up-sampling layer to gradually restore high-resolution details, and output a high-resolution image, the discriminator is used to output a probability of image authenticity, and through the game between the generator and the discriminator, a super-resolution image close to a real image is generated.

[0004] However, the super-resolution image generated by the above method is prone to blurring, and the super-resolution effect of local details is poor. SUMMARY

[0005] The present application provides a training method and device of an image generation model, equipment and a storage medium. The technical solutions provided by the embodiments of the present application are as follows:

[0006] According to an aspect of the embodiments of the present application, a training method of an image generation model is provided, and the method comprises:

[0007] At least one training sample of the image generation model is obtained, each training sample comprising a first sample image and a second sample image, the first sample image being an image meeting a resolution requirement, and the second sample image being an actually photographed image;

[0008] The first sample image is subjected to degradation processing to obtain a first sample image after degradation processing;

[0009] A first prediction image corresponding to the first sample image after degradation processing is generated by the image generation model according to the first sample image after degradation processing, the resolution of the first prediction image being greater than the resolution of the first sample image after degradation processing;

[0010] The parameters of the image generation model are adjusted according to the difference between the first prediction image and the first sample image, and the difference between the first prediction image and the second sample image, to obtain a trained image generation model.

[0011] According to an aspect of the embodiments of the present application, a training device of an image generation model is provided, and the device comprises:

[0012] a sample obtaining module configured to obtain at least one training sample of the image generation model, each training sample comprising a first sample image and a second sample image, the first sample image being an image meeting a resolution requirement, and the second sample image being an actually captured image;

[0013] an image degradation module configured to perform degradation processing on the first sample image to obtain a degraded first sample image;

[0014] an image generation module configured to generate, by using the image generation model, a first predicted image corresponding to the degraded first sample image according to the degraded first sample image, the first predicted image having a resolution greater than that of the degraded first sample image;

[0015] a model training module configured to adjust parameters of the image generation model according to differences between the first predicted image and the first sample image and between the first predicted image and the second sample image, to obtain a trained image generation model.

[0016] According to an aspect of the embodiments of the present application, a computer device is provided, which comprises a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-mentioned image generation model training method.

[0017] According to an aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores a computer program, the computer program being loaded and executed by a processor to implement the above-mentioned image generation model training method.

[0018] According to an aspect of the embodiments of the present application, a computer program product is provided, which comprises a computer program, the computer program being loaded and executed by a processor to implement the above-mentioned image generation model training method.

[0019] The technical solutions provided by the embodiments of the present application can bring the following beneficial effects:

[0020] By setting the first sample image meeting the resolution requirement and the second sample image actually captured in the training sample, after the image generation model generates the first predicted image, the parameters of the image generation model can be adjusted based on the difference between the first predicted image and the first sample image, so that the trained image generation model can restore a high-resolution image meeting the resolution requirement from a low-resolution image, and the parameters of the image generation model are adjusted based on the difference between the first predicted image and the second sample image, so that the trained image generation model can generate an image with higher authenticity. Compared with the way of considering only the authenticity of the generated image by using the adversarial neural network in the related art, the technical solution provided in the present application simultaneously considers the high resolution of the generated image and the authenticity of the generated image, so that the image generated by the trained image generation model has both high resolution and authenticity, improves the generation quality of the generated image, avoids the problem of blurred image details in the related art, improves the training effect of the image generation model, and improves the image resolution and image authenticity of the generated image. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is a schematic diagram of a computer system provided by an embodiment of the present application;

[0022] Figure 2 is a flowchart of a training method of an image generation model provided by an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of a training process of an image generation model provided by an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of another training process of an image generation model provided by an embodiment of the present application;

[0025] Figure 5 is a comparison diagram of the effects of QNN and Python quantization inference provided by an embodiment of the present application;

[0026] Figure 6 is a block diagram of a training device of an image generation model provided by an embodiment of the present application;

[0027] Figure 7 is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] To make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0029] Reference should be made to Figure 1Fig. 1 shows a schematic diagram of a computer system according to an embodiment of the present application. The computer system can include a model training device 10 and a model using device 20.

[0030] The model training device 10 can be an electronic device such as a mobile phone, a tablet computer, a notebook computer, a desktop computer, a smart television, a multimedia playing device, a vehicle terminal, a server, a smart robot, or other electronic devices with strong computing capability. The model training device 10 is configured to train an image generation model.

[0031] In the embodiments of the present application, the image generation model is a machine learning model trained based on a training method of an image generation model, and is configured to generate a high-resolution image according to an input low-resolution image. The model training device 10 can train the image generation model in a machine learning manner to enable the image generation model to generate a high-resolution image. The specific model training method can refer to the following embodiments.

[0032] In the embodiments of the present application, the image generation model includes an image encoder, a denoising network and an image decoder, the input image of the image generation model is a low-resolution image, and the output image is a high-resolution image after resolution enhancement. The image encoder is configured to perform feature extraction on the input image to obtain image features of the input image. The denoising network is configured to obtain denoising features corresponding to the input image according to the image features of the input image and an image description text. The image decoder is configured to generate an output image according to the denoising features corresponding to the input image. The image description text can be a pre-set description text or a description text obtained by performing text description extraction on the input image by an image description extractor.

[0033] The trained image generation model can be deployed in the model using device 20 for use. The model using device 20 can be an electronic device such as a mobile phone, a tablet computer, a notebook computer, a desktop computer, a smart television, a multimedia playing device, a vehicle terminal, a server, a smart robot, or other electronic devices with strong computing capability. When a high-resolution image needs to be generated, the model using device 20 can achieve the above functions by using the trained image generation model.

[0034] The model training device 10 and the model using device 20 can be two independent devices, or can be the same device. If the model training device 10 and the model using device 20 are the same device, the model training device 10 can be deployed in the model using device 20.

[0035] In the embodiments of the present application, the execution subject of each step can be a computer device, which can be a computer device as shown in Fig. 1. Figure 1The model training device 10 in the server can also be the model using device 20. The server can be a single physical server, a server cluster composed of multiple physical servers, a distributed system, or a cloud server providing cloud computing services.

[0036] Reference can be made to Figure 2 which shows a flowchart of a method for training an image generation model according to an embodiment of the present application. The execution subject of each step of the method can be a computer device. The method can include at least one of the following steps 210-240:

[0037] In step 210, at least one training sample of an image generation model is obtained, each training sample including a first sample image and a second sample image. The first sample image is an image meeting a resolution requirement, and the second sample image is an actually photographed image.

[0038] The resolution requirement refers to a requirement that the resolution reaches a resolution threshold. In the case where an image meets the resolution requirement, the image can be referred to as a super-resolution image, and in the case where an image does not meet the resolution requirement, the image can be referred to as a low-resolution image. That is, the first sample image is an image whose resolution reaches the resolution threshold, and the first sample image is a super-resolution image in the general sense.

[0039] The actually photographed image can be referred to as a real image, and here the actually photographed image refers to an image that has not been artificially processed after being actually photographed. Both the first sample image and the second sample image are actually photographed images.

[0040] It should be noted that the second sample image is an image meeting the resolution requirement, and the second sample image is also a super-resolution image. Alternatively, the resolution of the second sample image can be the same as that of the first sample image, can be greater than that of the first sample image, or can be less than that of the first sample image.

[0041] The image types of the sample images include but are not limited to image types such as a person image, a landscape image, an animal image, a still life image, and a scenery image. The image type of the first sample image and the image type of the second sample image can be the same or different, and the present application does not limit the same. Exemplarily, the first sample image and the second sample image can be person images photographed for different persons.

[0042] In some embodiments, the first sample image and the second sample image can be the same image, and the second sample image can be replaced by the first sample image, and step 210 can be implemented as: obtaining at least one training sample of the image generation model, each training sample comprising a first sample image, the first sample image being an image meeting the resolution requirement and being an actually captured image.

[0043] In step 220, the first sample image is degraded to obtain a degraded first sample image.

[0044] The degradation processing is used to reduce the resolution of the first sample image to obtain a degraded first sample image. Degradation refers to resolution degradation, so the resolution of the degraded first sample image is less than that of the first sample image, and the degraded first sample image is an image that does not meet the resolution requirement. Therefore, the degraded first sample image herein is a low-resolution image.

[0045] In some embodiments, the degradation processing comprises at least one of the following: blur processing, noise adding processing, compression processing.

[0046] The blur processing is used to smooth the image to reduce noise and image details in the image. For example, if the first sample image is a human image, the blur processing can blur the face part in the first sample image. The blur processing comprises at least one of the following processing methods: mean blur processing, Gaussian blur processing, median blur processing. The mean blur processing refers to a processing method in which the pixel average value of the field pixels is used to replace the pixel value of the center pixel. The Gaussian blur processing refers to a processing method in which the pixel value of the field pixels is weighted by a Gaussian function. The median blur processing refers to a processing method in which the pixel median value of the field pixels is used to replace the pixel value of the center pixel.

[0047] The noise adding processing is used to add random noise to the image to simulate noise (such as sensor noise, etc.) in a real shooting scene. The added random noise can be Gaussian noise, so that the pixel value of each pixel in the image after adding Gaussian noise can follow a Gaussian distribution. The added random noise can also be salt and pepper noise, so that each pixel in the image after adding salt and pepper noise becomes pure black or pure white at random.

[0048] The compression processing is used to reduce the size of the image file, including two compression modes of lossy compression and lossless compression. The lossy compression refers to a compression mode in which a part of image information is lost after decompression of the compressed image, but the lost image information has less impact on the overall use of the image. Exemplarily, the lossy compression can be a JPEG (Joint Photographic Experts Group) compression mode. The lossless compression refers to a compression mode in which the compressed image can be completely restored to the original data after decompression, without losing any image information. Exemplarily, the lossless compression can be a PNG (Portable Network Graphics) compression mode.

[0049] When the degradation processing is performed, the application does not limit the processing sequence of the blurring processing, the noise adding processing and the compression processing. Exemplarily, the first sample image is subjected to the blurring processing to obtain a first sample image after blurring processing, the first sample image after blurring processing is subjected to the noise adding processing to obtain a first sample image after noise adding processing, and the first sample image after noise adding processing is subjected to the compression processing to obtain the first sample image after degradation processing.

[0050] In step 230, the first prediction image corresponding to the first sample image after degradation processing is generated by the image generation model according to the first sample image after degradation processing. The resolution of the first prediction image is greater than the resolution of the first sample image after degradation processing.

[0051] The first sample image after degradation processing is input into the image generation model, and the first prediction image is output by the image generation model. The image generation model is used to restore a high-resolution image from a low-resolution image, and therefore, the resolution of the first prediction image is greater than the resolution of the first sample image after degradation processing.

[0052] In some embodiments, the image generation model includes an image encoder, a denoising network and an image decoder, which can be referred to the schematic diagram of the image generation model shown in FIG. 2. Figure 1 Step 230 includes at least one of sub-steps 231-233.

[0053] In sub-step 231, the image feature of the first sample image after degradation processing is obtained by performing feature extraction on the first sample image after degradation processing by the image encoder.

[0054] The image encoder is used to perform feature extraction on the input image to obtain the image feature of the input image. The first sample image after degradation processing is input into the image encoder, and the image feature of the first sample image after degradation processing is output by the image encoder.

[0055] In sub-step 232, the denoising network obtains a denoising feature corresponding to the first sample image after the degradation processing according to the image feature of the first sample image after the degradation processing and the image description text. The image description text is used to describe the image feature of the first predicted image in the form of text.

[0056] The image feature of the first sample image after the degradation processing is in the form of a vector or a matrix, and the image description text is in the form of text.

[0057] In some embodiments, the image description text is a pre-set description text.

[0058] Since the pre-set image description text can be applied to different first sample images, the image description text is a general-purpose image description text, and thus the image description text does not involve the image content of the first sample image, but can be a description text used to describe the image quality of the first predicted image, i.e., the image description text is used to describe the image quality feature of the first predicted image in the form of text. For example, the image description text can be “movie-level picture quality, high contrast, photo-level realism, 8K ultra-high definition, extremely fine detail performance, super-sharpness, and perfect deformation-free.”

[0059] By pre-setting the image description text as a fixed description text, the extraction step of the text description of the image description extractor can be omitted when generating the image, the time for generating the image is saved, the efficiency of generating the image is improved, and since the image description extractor is not included in the image generation model, the model size of the image generation model is greatly reduced, the complexity of the image generation model is reduced, the image generation model is facilitated to be deployed in the computer device, and the deployment space and the deployment cost of the image generation model are saved.

[0060] In some embodiments, the image description text is obtained by performing text description extraction on the first sample image after the degradation processing by using the image description extractor.

[0061] The image description extractor is used to extract the text feature in the first sample image after the degradation processing, and the image description text obtained by using the image description extractor is used to describe the image content feature of the first sample image after the degradation processing in the form of text, so as to indicate the image content feature of the first predicted image.

[0062] By using the image description extractor to extract the image description text, the image description text can contain more image details, so that the generation of the image according to the image description text can improve the generation quality of the first predicted image and enrich the image details in the first predicted image.

[0063] In some embodiments, the denoising network can be a U-net network, which adopts a symmetrical encoder-decoder network structure, including a downsampling network and an upsampling network. The downsampling network is used to realize the feature extraction function of the encoder, gradually extracts the image features of the first sample image after degradation processing and the text features of the image description text through convolution layers and pooling layers, and reduces the resolution of the space. The upsampling network is used to realize the feature restoration function of the decoder, gradually restores the spatial resolution through the deconvolution layer, and combines the low-level detail features of the encoder through the skip connection to splice the features of the corresponding layers of the encoder and the decoder, retain the edge and detail information, improve the segmentation accuracy, and output the denoising features corresponding to the first sample image after degradation processing.

[0064] The denoising features corresponding to the first sample image after degradation processing are image features obtained by removing the noise in the first sample image after degradation processing. The denoising features corresponding to the first sample image after degradation processing can accurately represent the image features of the first prediction image.

[0065] It should be noted that the above denoising network is a U-net network after pruning and fine-tuning. Since the denoising network is pruned and distilled, the parameter amount of the denoising network is small, and it is a lightweight U-net network.

[0066] In sub-step 233, the image decoder generates the first prediction image corresponding to the first sample image after degradation processing according to the denoising features corresponding to the first sample image after degradation processing.

[0067] The image decoder is used for feature restoration of the input features, and restores the input features to the image corresponding to the input features. The denoising features corresponding to the first sample image after degradation processing are input into the image decoder, and the image decoder outputs the first prediction image corresponding to the first sample image after degradation processing.

[0068] By setting a lightweight denoising network in the image generation model, the model size of the image generation model can be reduced, the image generation model can be easily deployed in a computer device, the deployment space and deployment cost of the image generation model can be saved, and only one image denoising is performed during image generation. In addition to ensuring the image generation effect, the image generation time is also saved, and the image generation efficiency is improved.

[0069] In step 240, the parameters of the image generation model are adjusted according to the difference between the first prediction image and the first sample image, and the difference between the first prediction image and the second sample image, to obtain a trained image generation model.

[0070] According to the difference between the first predicted image and the first sample image, the parameters of the image generation model are adjusted, so that the trained image generation model can restore the image content in the input image more accurately, improve the image restoration effect of the trained image generation model, and generate an output image meeting the resolution requirement.

[0071] According to the difference between the first predicted image and the second sample image, the parameters of the image generation model are adjusted, so that the trained image generation model can generate an output image with higher authenticity, and improve the image authenticity of the output image generated by the trained image generation model.

[0072] In some embodiments, step 240 includes at least one of sub-steps 241-243.

[0073] In sub-step 241, a first loss function value is calculated according to the difference between the first predicted image and the first sample image.

[0074] The first loss function value is used to measure the difference between the first predicted image and the first sample image in numerical form. The smaller the first loss function value is, the smaller the difference between the first predicted image and the first sample image is, the better the image restoration effect of the first predicted image is, and the higher the image resolution of the first predicted image is.

[0075] In sub-step 242, a second loss function value is calculated according to the difference between the first predicted image and the second sample image.

[0076] The second loss function value is used to measure the authenticity difference between the first predicted image and the second sample image in numerical form. The smaller the second loss function value is, the smaller the difference between the first predicted image and the second sample image is, and the higher the image authenticity of the first predicted image is.

[0077] If the first sample image is used to replace the second sample image, i.e., only the first sample image is included in the training sample, sub-step 242 can be implemented as follows: the second loss function value is calculated according to the authenticity difference between the first predicted image and the first sample image.

[0078] In sub-step 243, the parameters of the image generation model are adjusted according to the first loss function value and the second loss function value, to obtain a trained image generation model.

[0079] The first loss function value and the second loss function value are weighted to obtain a total loss function value, and the parameters of the image generation model are adjusted according to the total loss function value to obtain a trained image generation model.

[0080] The application does not limit the weights of the first loss function value and the second loss function value. Exemplarily, the first loss function value and the second loss function value are added according to a weight ratio of 1:1 to obtain a total loss function value.

[0081] The specific calculation process of the first loss function value and the second loss function value can refer to the following embodiments, which are not introduced here.

[0082] By respectively calculating the first loss function value corresponding to the difference between the first predicted image and the first sample image and the second loss function value corresponding to the difference between the first predicted image and the second sample image, the image generation model trained according to the first loss function value and the second loss function value can generate images with high resolution and high authenticity, thereby improving the training effect of the image generation model.

[0083] The technical scheme provided in the embodiments of the application sets the first sample image meeting the resolution requirement and the second sample image actually captured in the training sample, so that after the image generation model generates the first predicted image, the parameters of the image generation model can be adjusted based on the difference between the first predicted image and the first sample image, and the trained image generation model can restore a high-resolution image meeting the resolution requirement from a low-resolution image, and the parameters of the image generation model can be adjusted based on the difference between the first predicted image and the second sample image, and the trained image generation model can generate an image with high authenticity. Compared with the way of considering only the authenticity of the generated image in the related art by using the adversarial neural network, the technical scheme provided in the application simultaneously considers the high resolution of the generated image and the authenticity of the generated image, so that the image generated by the trained image generation model has high resolution and high authenticity, improves the generation quality of the generated image, avoids the problem of blurred image details in the related art, improves the training effect of the image generation model, and improves the image resolution and image authenticity of the generated image.

[0084] In some embodiments, the sub-step 241 includes at least one of sub-steps 2411-2413.

[0085] In the sub-step 2411, an average error loss function value is calculated according to the pixel values in the first predicted image and the pixel values in the first sample image.

[0086] The average error loss function used here can be a mean absolute error loss function (MAE) or a mean squared error (MSE) loss function, which is not limited in the application.

[0087] Exemplarily, the mean absolute error loss function can be represented as:

[0088]

[0089] Exemplarily, the mean square error loss function can be represented as:

[0090]

[0091] wherein n represents the number of pixels in the first predicted image, y i represents the pixel value of the i-th pixel in the first sample image, represents the pixel value of the i-th pixel in the first predicted image.

[0092] Sub-step 2412, calculating the perceptual loss function value by the deep neural network according to the first predicted image and the first sample image.

[0093] LPIPS (Learned Perceptual Image Patch Similarity) is an index for measuring the perceptual similarity between images, which extracts high-level features of images through a deep neural network and calculates the distance in the feature space, so as to be closer to the image quality evaluation method of human visual perception. LPIPS loss is a loss function based on LPIPS, which can optimize the perceptual similarity between the first predicted image and the first sample image.

[0094] The image features of the first predicted image and the image features of the first sample image are extracted by the deep neural network, the perceptual distance between the first predicted image and the first sample image is calculated according to the image features of the first predicted image and the image features of the first sample image, and the perceptual loss function value, i.e. LPIPS score, is obtained according to the perceptual distance between the first predicted image and the first sample image.

[0095] Exemplarily, the feature distance between the image features of the first predicted image and the image features of the first sample image can be calculated in the feature space to obtain the perceptual distance between the first predicted image and the first sample image, and the feature distance is taken as the measurement of the perceptual similarity between the first predicted image and the first sample image. Optionally, the Euclidean distance or the cosine distance between the image features of the first predicted image and the image features of the first sample image can be calculated. The feature distances between the image features of the first predicted image and the image features of the first sample image are weighted and combined to obtain the perceptual loss function value.

[0096] Sub-step 2413, obtaining the first loss function value according to the mean error loss function value and the perceptual loss function value.

[0097] The average error loss function value and the perceptual loss function value are weighted to obtain a first loss function value. The application does not limit the weights of the average error loss function value and the perceptual loss function value. Exemplarily, the average error loss function value and the perceptual loss function value are added according to a weight ratio of 1:2 to obtain the first loss function value.

[0098] By adjusting the parameters of the image generation model using the average error loss function, the pixel values of each pixel in the image can be adjusted, improving the generation effect of local details in the image. By adjusting the parameters of the image generation model using the perceptual loss function value, the visual perception effect of humans is considered, making the generated image more perceptually close to the first sample image. When calculating the first loss function value, the average error loss function value and the perceptual loss function value are combined, so that the first loss function value can more comprehensively represent the difference between the first predicted image and the first sample image, thereby improving the image resolution of the generated image when training the image generation model based on the first loss function value. Avoid the incomplete adjustment of model parameters caused by adjusting the parameters of the image generation model using a single loss function.

[0099] In some embodiments, the average error loss function value is calculated according to the pixel values in the first predicted image and the pixel values in the first sample image, and the average error loss function value is taken as the first loss function value.

[0100] In some embodiments, the perceptual loss function value is calculated by a deep neural network according to the first predicted image and the sample image, and the perceptual loss function value is taken as the first loss function value.

[0101] In some embodiments, the sub-step 242 includes at least one of sub-steps 2421-2423.

[0102] Sub-step 2421: obtaining a gradient loss function value according to the first predicted image by a first diffusion model and a second diffusion model.

[0103] The first diffusion model is a pre-trained diffusion model, and the second diffusion model is a diffusion model in a training state. It can be understood that the first diffusion model is an open source diffusion model, and the second diffusion model is a diffusion model that is further trained based on the loss function value on the basis of the first diffusion model.

[0104] Exemplarily, the gradient loss function value can be a KL divergence (Kullback-Leibler Divergence), which is used to measure the difference between the probability distribution obtained by the first diffusion model and the probability distribution obtained by the second diffusion model, and describes the "information loss" or "deviation degree" of a real probability distribution relative to another reference probability distribution. By minimizing the KL divergence, the second diffusion model can learn the model performance of the first diffusion model, thereby improving the image generation effect of the second diffusion model.

[0105] In some embodiments, the noise adding processing is performed on the first predicted image to obtain a first predicted image after noise adding; a real probability distribution corresponding to the first predicted image after noise adding is obtained by the first diffusion model according to the first predicted image after noise adding, the first diffusion model being a pre-trained diffusion model; a false probability distribution corresponding to the first predicted image after noise adding is obtained by the second diffusion model according to the first predicted image after noise adding, the second diffusion model being a diffusion model in a training state; and a gradient loss function value is obtained according to the real probability distribution and the false probability distribution.

[0106] The noise adding processing is used to add random noise to the first predicted image, and the added noise can be Gaussian noise or salt and pepper noise, which is not limited in the present application.

[0107] The first diffusion model predicts the noise added to the loaded first predicted image according to the first predicted image after noise adding, and the obtained real probability distribution is the prediction of the first diffusion model on the probability distribution of the noise. The second diffusion model predicts the noise added to the loaded first predicted image according to the first predicted image after noise adding, and the obtained false probability distribution is the prediction of the second diffusion model on the probability distribution of the noise.

[0108] For example, the first predicted image after noise adding is input into the first diffusion model and the second diffusion model, and the mean and variance of the noise are output by the first diffusion model and the second diffusion model. According to the mean and variance of the noise, a Gaussian distribution of the noise can be constructed. Since the first diffusion model is a pre-trained diffusion model, the Gaussian distribution of the probability constructed based on the output data of the first diffusion model can be referred to as a real probability distribution, which is used as a learned probability distribution. Since the second diffusion model is a diffusion model in a training state, the Gaussian distribution of the probability constructed based on the output data of the second diffusion model can be referred to as a false probability distribution, which is used as a learnable probability distribution.

[0109] The gradient loss function value is calculated according to the real probability distribution and the false probability distribution, and is used to measure the difference between the real probability distribution and the false probability distribution. Exemplarily, the gradient loss function value can be represented as:

[0110]

[0111] wherein P open (∈|x t , t) represents the true probability distribution obtained by the first diffusion model, P train (∈|x t , t) represents the false probability distribution obtained by the second diffusion model.

[0112] If the gradient loss function value is small, it indicates that the prediction of the probability distribution of the noise by the first diffusion model and the second diffusion model is relatively close, the second diffusion model has been trained well, close to the model performance of the first diffusion model, and the authenticity of the first predicted image is high. If the gradient loss function value is large, it indicates that the prediction of the probability distribution of the noise by the first diffusion model and the second diffusion model is quite different, the second diffusion model is still far from being trained, and the authenticity of the first predicted image is relatively lacking.

[0113] By respectively calculating the true probability distribution of the first diffusion model for the first predicted image after adding noise and the false probability distribution of the second diffusion model for the first predicted image after adding noise, the gradient loss function value can be calculated to measure the authenticity of the first predicted image, the learning state of the second diffusion model is supervised by the first diffusion model, and the probability distribution obtained by the second diffusion model and the probability distribution obtained by the first diffusion model are made to converge, and the first predicted model with high authenticity is obtained.

[0114] Sub-step 2422, obtaining an adversarial loss function value by the adversarial neural network according to the first predicted image and the second sample image.

[0115] The adversarial neural network includes a generator and a discriminator, the generator is used to generate an image close to the first predicted image according to the first predicted image, and the discriminator is used to judge the authenticity of the second sample image and the image generated by the generator. The adversarial loss function value includes the loss function value of the generator and the loss function value of the discriminator, the loss function value of the generator is used to measure the similarity between the image generated by the generator and the first predicted image, and the loss function value of the discriminator is used to measure the probability that the second sample image and the image generated by the generator are judged as real images.

[0116] Therefore, the adversarial loss function value can be used to measure the difference between the image generated by the generator and the second sample image, and Nash equilibrium is achieved by the loss function value of the generator and the loss function value of the discriminator in the model training process, so that the generator generates a false image closer to the input image, and the discriminator improves the ability to judge the authenticity of the image, so that the generated image tends to be a real image, and the first predicted image is close to a real image.

[0117] The training target of the adversarial loss function value is to make the generator and the discriminator reach a dynamic balance (Nash equilibrium) in the game, that is, the adversarial loss function value tends to a stable constant value (such as -log4 and log2).

[0118] In substep 2423, the second loss function value is obtained according to the gradient loss function value and the adversarial loss function value.

[0119] The gradient loss function value and the adversarial loss function value are weighted to obtain the second loss function value. The application does not limit the weights of the gradient loss function value and the adversarial loss function value. Exemplarily, the gradient loss function value and the adversarial loss function value are added according to a weight ratio of 1:1 to obtain the second loss function value.

[0120] When the parameters of the image generation model are adjusted according to the second loss function value, the training target is to make the adversarial loss function tend to be stable (such as -log4 and log2), and at the same time, other loss function values are as close to 0 as possible.

[0121] By adjusting the parameters of the image generation model using the gradient loss function, local changes (such as edges and textures) in the image can be optimized, avoiding image blurring or excessive smoothing, improving the ability to preserve local details in the image, and by adjusting the parameters of the image generation model using the adversarial loss function value, it is helpful to generate highly realistic images in terms of texture, detail and global consistency. When calculating the second loss function value, the gradient loss function value and the adversarial loss function value are combined, so that the second loss function value can more comprehensively represent the authenticity difference between the first predicted image and the second sample image, avoiding the use of a single loss function to adjust the parameters of the image generation model, so that the model parameter adjustment is not complete, balancing the authenticity and accuracy of the generated image, improving the quality of image generation, and helping to improve the authenticity of the generated image.

[0122] In some embodiments, the gradient loss function value is obtained from the first predicted image by the first diffusion model and the second diffusion model, and the gradient loss function value is taken as the second loss function value.

[0123] In some embodiments, the adversarial loss function value is obtained from the first predicted image and the second sample image by the adversarial neural network, and the adversarial loss function value is taken as the second loss function value.

[0124] In some embodiments, substep 243 includes at least one of substeps 2431-2432.

[0125] In the sub-step 2431, the parameters of the image encoder, the denoising network, the second diffusion model and the adversarial neural network are adjusted according to the first loss function value and the second loss function value, to obtain an adjusted image encoder and an adjusted denoising network.

[0126] The parameters of the image encoder, the denoising network, the second diffusion model and the adversarial neural network are adjustable parameters, and the parameters of the image decoder, the image description extractor and the first diffusion model are non-adjustable parameters.

[0127] In the adjustment of the parameters of the image encoder, the denoising network, the second diffusion model and the adversarial neural network according to the first loss function value and the second loss function value, the average error loss function value in the first loss function value tends to 0, the perceptual loss function value in the first loss function value tends to 0, the gradient loss function value in the second loss function value tends to 0, and the adversarial loss function value in the second loss function value tends to be stable, to obtain the adjusted image encoder and the adjusted denoising network. The adjusted second diffusion model and the adjusted adversarial neural network are not used in the image generation model.

[0128] In some embodiments, the parameters of the image encoder are adjusted by a weight matrix, and then the parameters of the weight matrix, the denoising network, the second diffusion model and the adversarial neural network are adjusted according to the first loss function value and the second loss function value, to obtain an adjusted weight matrix and an adjusted denoising network; and the adjusted image encoder is obtained according to the image encoder and the adjusted weight matrix.

[0129] The LoRA layer (Low-Rank Adaptation) can be superimposed on the image encoder, the parameters of the image encoder are adjusted by freezing the parameters of the image encoder and introducing the weight matrix of the LoRA layer to match the training task. Therefore, when the parameters of the image encoder are adjusted in the above training, the parameters of the weight matrix can be adjusted.

[0130] The parameters of the image encoder and the adjusted weight matrix are added to obtain an adjusted image encoder. The weight matrix can be composed of two low-rank matrices, and therefore, when the parameters of the weight matrix are adjusted in the above training, the parameters of the two low-rank matrices can be adjusted.

[0131] Exemplarily, the parameters of the adjusted image encoder can be represented as:

[0132] W' = W + A x B

[0133] Wherein, W represents the parameters of the image encoder, W' represents the adjusted parameters of the image encoder, AxB represents the weight matrix, A and B represent the low-rank matrix, B e Rdxr, A e Rrxk, r << min(d, k), r is the low-rank dimension.

[0134] By adjusting the parameters of the weight matrix to adjust the parameters of the image encoder, the amount of parameter adjustment required for adjusting the parameters can be reduced, so that adjusting a small amount of parameters can achieve the adjustment effect of adjusting the parameters of the image encoder, reducing the computing resources required for model training, reducing the computing cost and storage cost required for model training, and improving the training efficiency of the model. Moreover, the parameters of the weight matrix can be easily loaded into the image encoder without modifying the structure and parameters of the image encoder, facilitating deployment and use, and enabling training of the image generation model on resource-prioritized computer devices.

[0135] Sub-step 2432, obtaining the trained image generation model according to the adjusted image encoder, the adjusted denoising network and the image decoder.

[0136] The trained image generation model includes the adjusted image encoder, the adjusted denoising network and the image decoder, and specifically includes the adjusted weight matrix, the image encoder, the adjusted denoising network and the image decoder.

[0137] By adjusting the parameters of the second diffusion model and the adversarial neural network while training the image generation model, the lack of image authenticity generated by the trained image generation model caused by adjusting only the parameters of the image generation model is avoided, so that the high resolution of the generated image and the authenticity of the generated image are considered at the same time. While improving the resolution of the image generated by the trained image generation model, the image authenticity of the generated image is also improved.

[0138] Figure 3A schematic diagram of a training process of an image generation model is shown, which includes an image description extractor, an image encoder, a denoising network and an image decoder, wherein the parameters of the image encoder are adjusted by a weight matrix. The first sample image is degraded to obtain the degraded first sample image. After obtaining the degraded first sample image, the image encoder is used to perform feature extraction on the degraded first sample image to obtain image features of the degraded first sample image. The image description extractor is used to perform text description extraction on the degraded first sample image to obtain an image description text. The denoising network is used to obtain denoising features corresponding to the degraded first sample image according to the image features of the degraded first sample image and the image description text. The image decoder is used to generate a first predicted image corresponding to the degraded first sample image according to the denoising features corresponding to the degraded first sample image. A first loss function value is calculated according to the difference between the first predicted image and the first sample image.

[0139] The first predicted image is subjected to noise addition to obtain a noise-added first predicted image. A first diffusion model is used to obtain a real probability distribution corresponding to the noise-added first predicted image according to the noise-added first predicted image. A second diffusion model is used to obtain a fake probability distribution corresponding to the noise-added first predicted image according to the noise-added first predicted image. A gradient loss function value is obtained according to the real probability distribution and the fake probability distribution. An adversarial neural network is used to obtain an adversarial loss function value according to the first predicted image and a second sample image.

[0140] The parameters of the weight matrix, the denoising network, the second diffusion model and the adversarial neural network are adjusted according to the first loss function value, the gradient loss function value and the adversarial loss function value to obtain an adjusted weight matrix and an adjusted denoising network. A trained image generation model is obtained according to the adjusted image encoder, the adjusted denoising network and the image decoder. An adjusted image encoder is obtained according to the image encoder and the adjusted weight matrix.

[0141] Figure 4 A schematic diagram of a training process of another image generation model is shown, which includes an image encoder, a denoising network and an image decoder, and does not include an image description extractor, wherein the parameters of the image encoder are adjusted by a weight matrix, and the image description text is a pre-set description text, so that the step of performing text description extraction on the degraded first sample image by the image description extractor to obtain the image description text is not needed. The training process of the image generation model is the same as that in the image generation model. Figure 3

[0142] In some embodiments, Figure 4 ​The training process of the image generation model shown can be based on Figure 3 After the trained image generation model shown is obtained Figure 3 After the trained image generation model shown is obtained Figure 4 The training process of the image generation model shown, the image generation model is trained Figure 3 The parameters of the trained image generation model shown are adjusted to obtain the image generation model after retraining. In this way, training based on the training process of the image generation model shown directly can be avoided, and a large loss can be avoided. Figure 4 The training process of the image generation model shown directly.

[0143] Figure 5 The effect comparison chart of QNN and Python quantization inference is shown. The present application adopts QNN (Quantum Neural Networks) tool for model conversion, and performs target machine virtual inference on QNN AI Model cloud. On the original 128x128 test picture, 4 times magnification is adopted, and in the case of no quantization, the average time consumption of quantization inference is 625.6ms. After the image generation model is quantized by W8A16, that is, the weight parameter is quantized by 8bit and the activation parameter is quantized by 16bit, the average time consumption of quantization inference can be reduced to 193.7ms. The inference effect is shown in Figure 5 As shown, the quantization basically does not cause obvious effect loss. Considering that the user may need larger picture generation scale effect, the block multiple inference mode can be adopted, such as generating 1K image, that is, the block size of 512x512 is adopted, and the model inference is executed for 4 times. The inference time consumption can also be maintained within seconds.

[0144] The following is an embodiment of the device of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0145] Please refer to Figure 6 which shows the block diagram of the training device of the image generation model provided by an embodiment of the present application. The device has the function of realizing the training method of the image generation model described above. The function can be realized by hardware, or the corresponding software can be executed by hardware. The device can be the computer device introduced above, or can be arranged in the computer device. As shown in Figure 6 The device 600 can include a sample acquisition module 610, an image degradation module 620, an image generation module 630 and a model training module 640.

[0146] The sample obtaining module 610 is configured to obtain at least one training sample of the image generation model, each training sample comprising a first sample image and a second sample image, the first sample image being an image meeting a resolution requirement, and the second sample image being an actually captured image.

[0147] The image degradation module 620 is configured to perform degradation processing on the first sample image to obtain a degraded first sample image.

[0148] The image generation module 630 is configured to generate, by using the image generation model, a first predicted image corresponding to the degraded first sample image according to the degraded first sample image, the first predicted image having a resolution greater than that of the degraded first sample image.

[0149] The model training module 640 is configured to adjust parameters of the image generation model according to a difference between the first predicted image and the first sample image and a difference between the first predicted image and the second sample image, to obtain a trained image generation model.

[0150] In some embodiments, the model training module 640 is configured to:

[0151] calculate a first loss function value according to the difference between the first predicted image and the first sample image;

[0152] calculate a second loss function value according to the difference between the first predicted image and the second sample image;

[0153] adjust the parameters of the image generation model according to the first loss function value and the second loss function value, to obtain the trained image generation model.

[0154] In some embodiments, the model training module 640 is configured to:

[0155] obtain a gradient loss function value according to the first predicted image by using a first diffusion model and a second diffusion model;

[0156] obtain an adversarial loss function value according to the first predicted image and the second sample image by using an adversarial neural network;

[0157] obtain the second loss function value according to the gradient loss function value and the adversarial loss function value.

[0158] In some embodiments, the model training module 640 is configured to:

[0159] perform noise adding processing on the first predicted image to obtain a first predicted image after noise adding processing;

[0160] obtaining, by the first diffusion model, a real probability distribution corresponding to the first predicted image after adding noise according to the first predicted image after adding noise, the first diffusion model being a pre-trained diffusion model;

[0161] obtaining, by the second diffusion model, a false probability distribution corresponding to the first predicted image after adding noise according to the first predicted image after adding noise, the second diffusion model being a diffusion model in a training state;

[0162] obtaining the gradient loss function value according to the real probability distribution and the false probability distribution.

[0163] In some embodiments, the model training module 640 is configured to:

[0164] calculating an average error loss function value according to each pixel value in the first predicted image and each pixel value in the first sample image;

[0165] calculating a perception loss function value according to the first predicted image and the first sample image by a deep neural network;

[0166] obtaining the first loss function value according to the average error loss function value and the perception loss function value.

[0167] In some embodiments, the image generation model comprises an image encoder, a denoising network and an image decoder; and the model training module 640 is configured to:

[0168] adjusting parameters of the image encoder, the denoising network, the second diffusion model and the adversarial neural network according to the first loss function value and the second loss function value to obtain an adjusted image encoder and an adjusted denoising network;

[0169] obtaining the trained image generation model according to the adjusted image encoder, the adjusted denoising network and the image decoder.

[0170] In some embodiments, the parameters of the image encoder are adjusted by a weight matrix; and the model training module 640 is configured to:

[0171] adjusting parameters of the weight matrix, the denoising network, the second diffusion model and the adversarial neural network according to the first loss function value and the second loss function value to obtain an adjusted weight matrix and the adjusted denoising network;

[0172] obtaining the adjusted image encoder according to the image encoder and the adjusted weight matrix.

[0173] In some embodiments, the image generation model comprises: an image encoder, a denoising network, and an image decoder; and the image generation module 630 is configured to:

[0174] perform feature extraction on the first sample image after the degradation processing by the image encoder to obtain image features of the first sample image after the degradation processing;

[0175] obtain denoising features corresponding to the first sample image after the degradation processing by the denoising network according to the image features of the first sample image after the degradation processing and image description text, the image description text being used to describe the image features of the first predicted image in the form of text;

[0176] generate the first predicted image corresponding to the first sample image after the degradation processing by the image decoder according to the denoising features corresponding to the first sample image after the degradation processing.

[0177] In some embodiments, the image description text is a pre-set description text.

[0178] Alternatively,

[0179] the image description text is a description text obtained by performing text description extraction on the first sample image after the degradation processing by an image description extractor.

[0180] It should be noted that the apparatus provided in the above embodiments, when realizing its functions, is only exemplified by the above division of functional modules. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the content structure of the device is divided into different functional modules to complete all or part of the above-described functions. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0181] For reference Figure 7 which shows a structural block diagram of a computer device 700 provided in an embodiment of the present application. The computer device 700 can be any electronic device with data computing, processing and storage functions. The computer device 700 can be used to implement the training method of the image generation model provided in the above embodiments.

[0182] Generally, the computer device 700 comprises a processor 701 and a memory 702.

[0183] The processor 701 can include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 701 can be implemented in the form of at least one of a DSP (Digital Signal Processing), an FPGA (Field Programmable Gate Array), a PLA (Programmable Logic Array). The processor 701 can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 701 can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content to be displayed by the display screen. In some embodiments, the processor 701 can further include an AI (Artificial Intelligence) processor for processing machine learning-related computing operations.

[0184] The memory 702 can include one or more computer-readable storage media that can be non-transitory. The memory 702 can also include a high-speed random access memory, and a nonvolatile memory such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 702 is configured to store a computer program configured to be executed by one or more processors to implement the above-mentioned image generation model training method.

[0185] Those skilled in the art can understand that, Figure 7 The structure shown in the figure does not constitute a limitation on the computer device 700, and can include more or fewer components than those shown, or combine certain components, or adopt a different arrangement of components.

[0186] In the illustrative embodiment, a computer readable storage medium is also provided, in which a computer program is stored, the computer program realizing the training method of the image generation model described above when executed by a processor of a computer device. Optionally, the computer readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0187] In the illustrative embodiment, a computer program product is also provided, which includes a computer program stored in a computer readable storage medium. The processor of a computer device reads the computer program from the computer readable storage medium, and the processor executes the computer program, so that the computer device executes the training method of the image generation model described above.

[0188] It should be noted that before collecting the relevant data of the user and in the process of collecting the relevant data of the user, a prompt interface, a pop-up window or output voice prompt information can be displayed, which is used to prompt the user that the relevant data of the user is being collected, so that the application only starts to execute the related steps of obtaining the relevant data of the user after obtaining the confirmation operation of the user to the prompt interface or the pop-up window, otherwise (i.e. without obtaining the confirmation operation of the user to the prompt interface or the pop-up window), ending the related steps of obtaining the relevant data of the user, i.e. not obtaining the relevant data of the user. In other words, all the user data collected by the application is collected under the condition that the personal information subject gives informed consent or separate consent according to the requirements of relevant national laws and regulations, and the subsequent data use and processing behavior is carried out within the scope of authorization of the user, and the collection, use and processing of the relevant user data need to comply with the relevant laws and regulations and standards of the country and region.

[0189] It should be understood that "multiple" mentioned herein refers to two or more than two. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship. In addition, the step numbers described herein only exemplarily show a possible execution order between steps, and in some other embodiments, the above steps can also be executed in a non-numbered order, such as two different numbered steps being executed at the same time, or two different numbered steps being executed in an order opposite to that shown in the figure, and the embodiments of the present application are not limited in this regard.

[0190] The above only describes exemplary embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A training method for an image generation model, characterized in that, The method includes: At least one training sample is obtained for the image generation model. Each training sample includes a first sample image and a second sample image. The first sample image is an image that meets the resolution requirements, and the second sample image is an image that is actually captured. The first sample image is subjected to degradation processing to obtain the first sample image after degradation processing; The image generation model generates a first predicted image corresponding to the first degraded sample image based on the first degraded sample image, wherein the resolution of the first predicted image is greater than the resolution of the first degraded sample image. Based on the differences between the first predicted image and the first sample image, and the differences between the first predicted image and the second sample image, the parameters of the image generation model are adjusted to obtain the trained image generation model.

2. The method according to claim 1, characterized in that, The step of adjusting the parameters of the image generation model based on the differences between the first predicted image and the first sample image, and the differences between the first predicted image and the second sample image, to obtain the trained image generation model, includes: Calculate the first loss function value based on the difference between the first predicted image and the first sample image; The second loss function value is calculated based on the difference between the first predicted image and the second sample image; Based on the first loss function value and the second loss function value, the parameters of the image generation model are adjusted to obtain the trained image generation model.

3. The method according to claim 2, characterized in that, The step of calculating the second loss function value based on the difference between the first predicted image and the second sample image includes: Based on the first predicted image, the gradient loss function value is obtained using the first diffusion model and the second diffusion model. The adversarial loss function value is obtained by using the adversarial neural network based on the first predicted image and the second sample image; The second loss function value is obtained based on the gradient loss function value and the adversarial loss function value.

4. The method according to claim 3, characterized in that, The step of obtaining the gradient loss function value based on the first predicted image using the first diffusion model and the second diffusion model includes: The first predicted image is subjected to noise addition processing to obtain the noisy first predicted image; The first diffusion model obtains the true probability distribution corresponding to the first predicted image after adding noise based on the first predicted image after adding noise. The first diffusion model is a diffusion model obtained through pre-training. The second diffusion model obtains the pseudo probability distribution corresponding to the first predicted image after adding noise based on the first predicted image after adding noise using the second diffusion model. The second diffusion model is a diffusion model in the training state. The gradient loss function value is obtained based on the true probability distribution and the false probability distribution.

5. The method according to any one of claims 2 to 4, characterized in that, The step of calculating the first loss function value based on the difference between the first predicted image and the first sample image includes: The average error loss function value is calculated based on each pixel value in the first predicted image and each pixel value in the first sample image; The perceptual loss function value is calculated using a deep neural network based on the first predicted image and the first sample image; The first loss function value is obtained based on the average error loss function value and the perception loss function value.

6. The method according to claim 4, characterized in that, The image generation model includes: an image encoder, a denoising network, and an image decoder; The step of adjusting the parameters of the image generation model based on the first loss function value and the second loss function value to obtain the trained image generation model includes: Based on the first loss function value and the second loss function value, the parameters of the image encoder, the denoising network, the second diffusion model, and the adversarial neural network are adjusted to obtain the adjusted image encoder and the adjusted denoising network. The trained image generation model is obtained based on the adjusted image encoder, the adjusted denoising network, and the image decoder.

7. The method according to claim 6, characterized in that, The parameters of the image encoder are adjusted using a weight matrix; The step of adjusting the parameters of the image encoder, the denoising network, the second diffusion model, and the adversarial neural network based on the first loss function value and the second loss function value to obtain the adjusted image encoder and the adjusted denoising network includes: Based on the first loss function value and the second loss function value, the parameters of the weight matrix, the denoising network, the second diffusion model, and the adversarial neural network are adjusted to obtain the adjusted weight matrix and the adjusted denoising network. The adjusted image encoder is obtained based on the image encoder and the adjusted weight matrix.

8. The method according to any one of claims 1 to 7, characterized in that, The image generation model includes: an image encoder, a denoising network, and an image decoder; The step of generating a first predicted image corresponding to the degraded first sample image using the image generation model includes: The image encoder performs feature extraction on the degraded first sample image to obtain the image features of the degraded first sample image; The denoising network obtains the denoising features corresponding to the first sample image after degradation processing based on the image features and image description text of the first sample image after degradation processing. The image description text is used to describe the image features of the first predicted image in text form. The image decoder generates a first predicted image corresponding to the first sample image after degradation, based on the denoising features corresponding to the first sample image after degradation.

9. The method according to claim 8, characterized in that, The image description text is a pre-set description text; or, The image description text is obtained by performing text description extraction on the degraded first sample image using an image description extractor.

10. A training device for an image generation model, characterized in that, The device includes: The sample acquisition module is used to acquire at least one training sample of the image generation model. Each training sample includes a first sample image and a second sample image. The first sample image is an image that meets the resolution requirements, and the second sample image is an image that was actually captured. An image degradation module is used to perform degradation processing on the first sample image to obtain a degraded first sample image; An image generation module is used to generate a first predicted image corresponding to the first degraded sample image based on the first degraded sample image using the image generation model, wherein the resolution of the first predicted image is greater than the resolution of the first degraded sample image. The model training module is used to adjust the parameters of the image generation model based on the differences between the first predicted image and the first sample image, and the differences between the first predicted image and the second sample image, to obtain the trained image generation model.

11. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the training method for the image generation model as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the training method of the image generation model as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes a computer program that is loaded and executed by a processor to implement the training method of the image generation model as described in any one of claims 1 to 9.

Images