Image generation model training method and device, image generation method and device, equipment and medium

Abstract

The embodiment of the invention provides an image generation model training method and device, an image generation method and device, equipment and a medium. The method comprises the steps that first image description and a first sample image corresponding to the first image description are acquired; obtaining a first generated image through a first image generation model by using the first image description; the first generated image and the first image description are utilized, a prediction weight value is obtained through a target reward model, and the prediction weight value is in positive correlation with the image quality of the first generated image and / or is in positive correlation with the matching degree between the first image description and the first generated image; and adjusting parameters of the first image generation model based on the difference between the first generated image and the first sample image and the predicted weight value.

Description

Technical Field

[0001] The present disclosure relates to the field of computing technology, and in particular to training of an image generation model, an image generation method, an apparatus, a device, and a medium. Background Art

[0002] In the field of image generation, image description (also called text prompt) can generally be used to guide image generation. Specifically, according to the image description, an image corresponding to the image description is obtained through an image generation model based on content generation technology.

[0003] In the above process, in order to ensure the quality of the generated images, the image generation model used needs to have better image generation capabilities. Therefore, how to provide an image generation model that is trained to have better image generation capabilities becomes an urgent problem to be solved. Summary of the invention

[0004] One or more embodiments of the present disclosure provide a method, apparatus, device and medium for training an image generation model, and a method, apparatus, device and medium for generating an image, so as to train an image generation model with better image generation capability.

[0005] According to a first aspect, a method for training an image generation model is provided, comprising:

[0006] Obtaining a first image description and a first sample image corresponding thereto;

[0007] Using the first image description and a first image generation model, a first generated image is obtained;

[0008] Using the first generated image and the first image description, a predicted weight value is obtained through a target reward model, wherein the predicted weight value is positively correlated with an image quality of the first generated image and / or is positively correlated with a degree of match between the first image description and the first generated image;

[0009] Based on the difference between the first generated image and the first sample image and the prediction weight value, the parameters of the first image generation model are adjusted.

[0010] In an optional embodiment, the adjusting the parameters of the first image generation model includes: using a preset loss function to determine an intermediate prediction loss based on the difference between the first generated image and the first sample image; determining a first prediction loss based on the product of the intermediate prediction loss and the prediction weight value; and adjusting the parameters of the first image generation model with the goal of minimizing the first prediction loss. In this embodiment, determining the first prediction loss based on the product of the intermediate prediction loss and the prediction weight value can make it possible to focus more on using image-text pairs with higher corresponding prediction weight values ​​to determine the first prediction loss during the training of the first image generation model, that is, focusing more on using image-text pairs with good quality to train the first image generation model, so as to better improve the image generation capability of the first image generation model and ensure the stability of its image generation capability.

[0011] In an optional embodiment, it also includes: using the first image description input, through the pre-trained second image generation model, to obtain the second generated image; the adjusting the parameters of the first image generation model includes: adjusting the parameters of the first image generation model based on the difference between the first generated image and the first sample image, the predicted weight value, and the difference between the first generated image and the second generated image. In this embodiment, the parameters of the first image generation model are adjusted in combination with the difference between the first generated image and the second generated image generated by the second image generation model, so that while improving the image generation capability of the first image generation model, the image generated by the first image generation model can be prevented from deviating too much from the image generated by the second image generation model, so as to avoid the first image generation model generating a confusing image (an image that does not match the input image description) to deceive the target reward model, causing the target reward model to output a numerically higher predicted weight value for the confusing image generated by the first image generation model.

[0012] In an optional embodiment, the adjusting the parameters of the first image generation model includes: determining an image difference value between the first generated image and the second generated image based on the difference between the first generated image and the second generated image; determining a first prediction loss based on the difference between the first generated image and the first sample image, the prediction weight value and the image difference value; adjusting the parameters of the first image generation model with the goal of minimizing the first prediction loss. In this embodiment, the image difference value between the first generated image and the second generated image is used to punish the situation where the image generated by the first image generation model deviates too far from the image generated by the second image generation model, wherein the larger the image difference value, the greater the penalty, so as to avoid the situation where the first image generation model generates a chaotic image to deceive the target reward model, causing the target reward model to output a numerically higher prediction weight value for the chaotic image generated by the first image generation model.

[0013] In an optional embodiment, the determining of the image difference value between the first generated image and the second generated image includes: using a preset divergence algorithm to determine a divergence value based on the difference between the first generated image and the second generated image; and determining the image difference value based on the divergence value. In this embodiment, the preset divergence algorithm can be used to more accurately determine the image difference between the first generated image and the second generated image, thereby better avoiding the situation where the first image generation model generates a chaotic image to deceive the target reward model, causing the target reward model to output a numerically higher prediction weight value for the chaotic image generated by the first image generation model.

[0014] In an optional implementation, the first sample image is an image generated according to the first image description using the second image generation model. In this implementation, the quality of the image-text pair formed by the image generated according to the first image description using the second image generation model and the corresponding first image description is relatively better, and using such image-text pairs to train the first image generation model is helpful to train a first image generation model that can generate images with better quality.

[0015] In an optional embodiment, it also includes: obtaining a second image description and its corresponding second sample image; using the second image description to obtain a third generated image through the first image generation model; using the difference between the second sample image and the third generated image to determine a second prediction loss; adjusting the parameters of the first image generation model with the goal of minimizing the first prediction loss includes: adjusting the parameters of the first image generation model with the goal of minimizing the first prediction loss and the second prediction loss. In this embodiment, in the process of training the first image generation model, real images and their corresponding image descriptions are also added to train the first image generation model, which increases the diversity of data for training the first image generation model and better improves the image generation capability of the first image generation model.

[0016] In an optional embodiment, it also includes: obtaining the third image description and the third sample image; obtaining the label weight value corresponding to the third image description and the third sample image, the label weight value is positively correlated with the image quality of the third sample image, and / or is positively correlated with the matching degree between the third image description and the third sample image; using the third image description and the third sample image, through the reward model to be trained, to obtain an evaluation weight value; using the evaluation weight value and the label weight value to determine the third prediction loss; with the goal of minimizing the third prediction loss, adjusting the reward model to be trained to obtain a target reward model that meets the first convergence condition. In this embodiment, the reward model to be trained is trained using the third image description and the third sample image, as well as the label weight values ​​corresponding to the two, to obtain the target reward model that meets the first convergence condition, which can achieve supervised training of the determination result of the reward model to be trained, maintain its accuracy, and improve the accuracy of the determination result of the reward model to be trained.

[0017] In an optional embodiment, the label weight value is determined based on at least one of the following information: the clarity score of the third sample image, the aesthetic score of the third sample image, and the matching value between the third sample image and the third image description. In this embodiment, the label weight value is determined by a number of information that can characterize the quality of the image and / or the quality and matching degree of the image and its image description. To a certain extent, the label weight value can better characterize the human quality evaluation of the corresponding image and its image description, thereby improving the accuracy of the predicted weight value output by the target reward model, so that the predicted weight value is more adapted to human feedback.

[0018] According to a second aspect, there is provided an image generation method, comprising:

[0019] Get the description of the image to be processed;

[0020] The target image is obtained by using the image description to be processed and the image generation model obtained by training using the image generation model training method described in the first aspect.

[0021] According to a third aspect, a training device for an image generation model is provided, comprising:

[0022] A first acquisition module, configured to acquire a first image description and a first sample image corresponding thereto;

[0023] A first obtaining module is configured to obtain a first generated image by using the first image description through a first image generation model;

[0024] a second obtaining module configured to obtain a predicted weight value by using the first generated image and the first image description through a target reward model, wherein the predicted weight value is positively correlated with an image quality of the first generated image and / or is positively correlated with a matching degree between the first image description and the first generated image;

[0025] The first adjustment module is configured to adjust the parameters of the first image generation model based on the difference between the first generated image and the first sample image and the prediction weight value.

[0026] According to a fourth aspect, there is provided an image generating device, comprising:

[0027] A second acquisition module is configured to acquire a description of the image to be processed;

[0028] The third obtaining module is configured to obtain a target image by using the image description to be processed and the image generation model obtained by training using the image generation model training device described in the third aspect.

[0029] According to a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method described in the first aspect or the second aspect.

[0030] According to a sixth aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described in the first aspect or the second aspect is implemented.

[0031] According to the training of the image generation model, the image generation method, the apparatus, the device and the medium provided by the embodiments of the present disclosure, after obtaining the first image description and the corresponding first sample image, the first image description is used to obtain the first generated image through the first image generation model, and then the first generated image and the first image description are used to obtain the predicted weight value through the target reward model. The predicted weight value is positively correlated with the image quality of the first generated image and / or is positively correlated with the matching degree between the first image description and the first generated image. Accordingly, the predicted weight value can characterize the image quality of the first generated image and / or the matching degree between the first generated image and the first image description, that is, the strength of the correlation. Then, based on the difference between the first generated image and the first sample image and the predicted weight value, the parameters of the first image generation model are adjusted to realize the combination of the difference between the first generated image and the first sample image and the predicted weight value that can characterize the image quality of the first generated image and / or the matching degree between the first generated image and the first image description, that is, the strength of the correlation, and jointly train the first image generation model, so that in the training process, human feedback information can be better and more timely introduced, and the trained image generation model has better image generation capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0033] Figure 1 A schematic diagram of an implementation framework of an embodiment disclosed in the present disclosure;

[0034] Figure 2 A schematic diagram of a flow chart of a training method for an image generation model provided in an embodiment;

[0035] Figure 3 Another schematic flow chart of a method for training an image generation model provided in an embodiment;

[0036] Figure 4 A schematic diagram of a flow chart of an image generation method provided in an embodiment;

[0037] Figure 5 A schematic block diagram of a training device for an image generation model provided in an embodiment;

[0038] Figure 6 A schematic block diagram of an image generating device provided in an embodiment;

[0039] Figure 7A schematic block diagram of an electronic device provided in an embodiment. DETAILED DESCRIPTION

[0040] The technical solution of the embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0041] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0042] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0043] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0044] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0045] The embodiments of the present disclosure disclose an image generation model training, image generation method, device, equipment and medium. The application scenarios and technical concepts of the method are first introduced as follows:

[0046] As mentioned above, in the field of image generation, in order to ensure the quality of the generated images, it is necessary to train an image generation model with better image generation capabilities. At present, the process of training an image generation model is generally as follows: obtaining a number of sample image-text pairs, each sample image-text pair includes a sample image and its corresponding image description, and then based on the first image description in any first sample image-text pair among the sample image-text pairs, obtaining a generated image corresponding to the first image description through an image generation model, and then using the difference between the generated image corresponding to the first image description and the sample image in the first sample image-text pair to which the first image description belongs to construct a prediction loss, and then adjusting the parameters of the image generation model with the goal of minimizing the prediction loss.

[0047] In the process of training the image generation model, the quality of each image-text pair in the sample image-text pairs (for example, the clarity of the image, the aesthetics of the image and / or the strength of the match between the image and the image description in each image-text pair) is not taken into consideration, resulting in the image generation model obtained by training having an image generation capability that is not good enough and not stable enough.

[0048] In view of this, the embodiments of the present disclosure provide an image generation model training, image generation method, device, equipment and medium to achieve training to obtain an image generation model with better image generation capabilities, better improve the stability of the image generation capabilities of the image generation model, and thereby obtain images of better quality. Figure 1 A schematic diagram of an implementation scenario according to an embodiment disclosed in the present disclosure is shown. In the implementation scenario, an exemplary structure of a training system for fine-tuning an image generation model and an exemplary data processing flow of the training system are shown. The training system may include a first image generation model to be trained, and a target reward model for determining feedback data (i.e., current environmental feedback, i.e., reward score) for fine-tuning the first image generation model.

[0049] In one implementation, the first image generation model may be a model pre-trained based on at least a picture-text dataset (hereinafter referred to as the first picture-text dataset for clarity of description), and has the ability to generate a corresponding image based on an input image description. The first picture-text dataset includes a plurality of picture-text pairs, wherein any picture-text pair (referred to as the first picture-text pair) includes an image and its corresponding image description. The image description may also be referred to as a text prompt, i.e., a text prompt used to generate an image.

[0050] The target reward model is pre-trained based on any image-text pair in the image-text data set (hereinafter referred to as the second image-text data set for clarity) and the label weight value corresponding to the image-text pair. The target reward model is used to output the predicted weight value corresponding to the image-text pair based on the input image-text pair. The label weight value can characterize the quality of the corresponding image-text pair. The quality of the image-text pair can be characterized from the following angles, but not limited to the following angles: the image quality of the image in the image-text pair and / or the matching degree between the image and the image description in the image-text pair. Among them, the image quality of the image can be determined based on the clarity of the image and / or the aesthetics of the image.

[0051] Among them, the higher the clarity of the image in the picture-text pair (i.e., the higher the clarity score), the better the beauty of the image (i.e., the higher the aesthetic score), the better the image quality of the image. The better the image quality of the image in the picture-text pair, and the stronger the match between the image and the image description in the picture-text pair (i.e., the higher the match value), the better the quality of the picture-text pair. It can be understood that the better the quality of the picture-text pair used to train the first image generation model, the better the image generation ability of the first image generation model can be improved. In one implementation, the larger the label weight value, the better the quality of the corresponding picture-text pair.

[0052] In one implementation, the second image-text data set and the subsequent training data set may include the same image-text pairs, or may include different image-text pairs.

[0053] The model training process is introduced below. In the model training process, a training data set for training the first image generation model is obtained. The training data set includes a number of sample image-text pairs. Any sample image-text pair includes an image description and its corresponding sample image. It can be understood that in the training process, the processing process of any pair of sample images and texts is similar. The model training process is described below by taking the processing process of any pair of first sample image-text pairs (including the first image description and its corresponding first sample image) as an example.

[0054] From the training data set, obtain the first image description and its corresponding first sample image, that is, obtain the first sample image-text pair; then, use the first image description to process the first image description through the first image generation model, that is, use the first image generation model to obtain the first generated image; use the first generated image and the first image description, through the target reward model, to obtain the predicted weight value corresponding to the first generated image and the first image description, wherein the predicted weight value can characterize the quality of the image-text pair of the first generated image and the first image description, and the quality of the image-text pair of the first generated image and the first image description can be characterized by the image quality of the first generated image, and / or the matching degree between the first image description and the first generated image. Specifically, the predicted weight value is positively correlated with the image quality of the first generated image, and / or, is positively correlated with the matching degree between the first image description and the first generated image.

[0055] Then, based on the difference between the first generated image and the first sample image and the predicted weight value, the parameters of the first image generation model are adjusted. In one implementation, Figure 1As shown, the difference between the first generated image and the first sample image can be determined based on the first generated image and the first sample image (for example, the Euclidean distance between the first generated image and the first sample image is used to characterize the difference between the first generated image and the first sample image). In another implementation, the probability of the first image generation model obtaining the first sample image for the first image description based on the first image description in the process of generating the first generated image by the first image generation model (i.e., the subsequent p(x1|z1)) can be obtained, and the probability can characterize the difference between the first generated image and the first sample image, wherein the greater the probability of the first image generation model obtaining the first sample image for the first image description, the smaller the difference between the first generated image and the first sample image.

[0056] In the above process, the quality of the first generated image and the first image description is evaluated based on the target reward model to obtain the corresponding prediction weight value, wherein the prediction weight value can characterize the quality of the image-text pair composed of the first generated image and the first image description, and further characterize the ability of the first image generation model to generate images for the first image description. In combination with the prediction weight value and the difference between the first generated image and the first sample image, the parameters of the first image generation model are adjusted, and the first image generation model can be trained based on the online reinforcement learning method in combination with environmental feedback (i.e., the feedback of the environment on the image generation ability of the model, i.e., the prediction weight value). In the process of model training, the first image generation model can be trained by focusing more on using the corresponding prediction weight value to characterize better quality data (e.g., the corresponding generated image with a higher prediction weight value and its corresponding sample image, and increasing its proportion), so that the trained image generation model has better image generation ability and better stability of image generation ability. And to a certain extent, it is guaranteed that the quality of the image generated by the trained first image generation model is more in line with environmental expectations (i.e., human intentions).

[0057] The above-mentioned prediction weight value is determined in real time during the training process of the first image generation model for the first image description and the first generated image generated by the first image generation model according to the first image description. The use of this type of prediction weight value can better improve the learning speed (i.e., convergence speed) of the first image generation model and the image generation quality of the first image generation model.

[0058] In one embodiment, in order to better improve the image generation capability of the first image generation model, Figure 1As shown, the training system of the image generation model may also include a pre-trained second image generation model. Based on the difference between the first generated image and the first sample image and the predicted weight value, before adjusting the parameters of the first image generation model, the first image description may be used to obtain the second generated image through the pre-trained second image generation model; and in the process of adjusting the parameters of the first image generation model, the parameters of the first image generation model may be adjusted in combination with the difference between the first generated image and the first sample image, the predicted weight value, and the difference between the first generated image and the second generated image.

[0059] Among them, Figure 1 As shown, the process of adjusting the parameters of the first image generation model may be: based on the difference between the first generated image and the second generated image, determine the image difference value between the two, based on the difference between the first generated image and the first sample image and the prediction weight value and the image difference value, determine the first prediction loss, and then adjust the parameters of the first image generation model to minimize the first prediction loss. In one implementation, as Figure 1 As shown, the difference between the first generated image and the second generated image can be determined based on the two, that is, the image difference value between the two can be determined (for example, the image difference value between the two can be determined based on the Euclidean distance that characterizes the difference between the first generated image and the second generated image).

[0060] In the above process, it is possible to improve the image generation capability of the first image generation model and improve the stability of its image generation capability while constraining the image generated by the first image generation model to not deviate too much from the image generated by the second image generation model, thereby avoiding the situation where the first image generation model generates a confusing image to deceive the target reward model into outputting a higher prediction weight value, thereby better improving the image generation capability of the first image generation model.

[0061] The following is a detailed description of the training of the image generation model, the image generation method, the apparatus, the device and the medium provided by the present invention in conjunction with specific embodiments. Figure 2 The flowchart of the training method of the image generation model in one embodiment of the present disclosure is shown. The method can be implemented by a target program. The target program can be installed in an electronic device, and the electronic device can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities. Figure 2 As shown, the method includes the following steps S210-S240:

[0062] In step S210, a first image description and its corresponding first sample image are obtained.

[0063] In one implementation, the preset data storage space corresponding to the target program may pre-store a training data set for training the first image generation model, and the training data set may include a plurality of image-text pairs, referred to as first image-text pairs, and any first image-text pair includes an image description and its corresponding sample image. In this step, the target program may obtain any first image description and its corresponding first sample image from the training data set stored in the preset data storage space, that is, obtain any pair of first image-text pairs in the training data set.

[0064] It can be understood that in the embodiments of the present disclosure, a pair of first image-text pairs in the training data set (for example, a first image description and its corresponding first sample image) is taken as an example to illustrate the processing process of each pair of first image-text pairs in the training data set, that is, the training process of the first image generation model. Using each pair of first image-text pairs in the training data set, the first image generation model can be trained based on a similar process.

[0065] In one implementation, the first sample image may be an image generated by a trained second image generation model using a first image description. The quality of the image-text pairs formed by this type of sample image and image description is relatively good. Using this type of image-text pairs with relatively good quality to train the first image generation model facilitates training to obtain a first image generation model with relatively good image generation capabilities.

[0066] In one implementation, the target program can also construct the aforementioned training data set, wherein the construction process includes: obtaining a number of image descriptions A (for example, obtaining image descriptions from a public data set that can provide a large number of image-text pairs), and then using each image description A respectively, through the trained second image generation model, to obtain an image corresponding to each image description A, i.e., a sample image; for each image description A, the image description A and its corresponding sample image are determined as a pair of first image-text pairs to construct the training data set.

[0067] In one implementation, the initialization parameters of the first image generation model may be the same as the parameters of the trained second image generation model, that is, the initial values ​​of the parameters of the first image generation model are the same as the values ​​of the parameters of the trained second image generation model.

[0068] For a clear layout, the following first introduces the pre-training process of the second image generation model (or the process of training the first image generation model).

[0069] The process includes: obtaining an image generation model that needs to be pre-trained (referred to as the image generation model to be trained); obtaining a picture-text dataset S (such as the aforementioned first picture-text dataset) used to train the image generation model to be trained, and the picture-text dataset S may include multiple picture-text pairs, and one picture-text pair includes an image description X and its corresponding image T. In one case, the picture-text dataset S may be obtained from a public dataset that can provide a large number of picture-text pairs or a designated artificially created dataset. The artificially created dataset includes a number of collected images and the image descriptions corresponding to each image that are artificially added to the image.

[0070] In one case, the image description may also be referred to as a text prompt, that is, a text prompt used to generate an image.

[0071] Then, according to the image description X in any image-text pair M in the image-text dataset S, the corresponding generated image T' is obtained through the image generation model to be trained, that is, the image description X is input into the image generation model to be trained, and the image description X is processed by the image generation model to be trained to generate the corresponding generated image T'; the difference between the image T in the aforementioned image-text pair M and the generated image T' is used to determine the loss corresponding to the image generation model to be trained, which is called the first generation loss. In one case, a preset loss function (such as a cross entropy loss function, an MSE (Mean Square Error) loss function, etc.) can be used to determine the first generation loss based on the difference between the image T and the generated image T'. The greater the difference between the image T and the generated image T', the greater the first generation loss.

[0072] Afterwards, with the goal of minimizing the first generation loss, that is, reducing the difference between the image in the image-text pair and the corresponding generated image, the parameters of the image generation model to be trained are adjusted until the image generation model reaches a preset convergence condition, and a pre-trained image generation model is obtained, that is, the pre-trained second image generation model is obtained, that is, the initial first image generation model in the training system is obtained. The preset convergence condition may include, but is not limited to, the number of parameter adjustments is not less than the first adjustment threshold, the training time exceeds the preset time, or the calculated first generation loss is lower than the first loss threshold.

[0073] Let's review the above process. The above embodiment is explained by taking a pair of image-text pairs (i.e., a pair of image descriptions X and their corresponding images T) as a sample. In another embodiment, the above process can also be performed on a batch of samples, i.e., multiple pairs of image-text pairs, to obtain the generated images corresponding to the image descriptions in each pair of image-text pairs, and then based on the difference between the generated images corresponding to the image descriptions in each pair of image-text pairs and the images in the pair of image-text pairs, determine the first generation loss, and adjust the parameters of the image generation model to be trained with the goal of minimizing the first generation loss. In this embodiment, the first generation loss is determined for a batch of samples, and then the parameters of the image generation model to be trained are adjusted. This can reduce the number of parameter adjustments for the image generation model to be trained, making it easier to implement the above process.

[0074] In one implementation, the first image generation model and the second image generation model may be image generation models based on a Diffusion Model (diffusion model).

[0075] In one implementation, the aforementioned training data set may also be used as training data for the target reward model, that is, the training data set and the aforementioned second image-text data set may include the same image-text pairs, or may include different image-text pairs.

[0076] After the target program obtains the first image description and the first sample image corresponding thereto, in step S220, the first image description is used to generate a first generated image through a first image generation model.

[0077] It can be understood that, when the current training process is the first training process based on the training system (i.e., the first training of the first image generation model based on the training system), the first image generation model is the image generation model obtained by the aforementioned pre-training, and its parameters are the same as the parameters of the aforementioned second image generation model. When the current training process is a non-first training process based on the training system for the image generation model (i.e., the Nth training of the first image generation model based on the training system, where N is greater than 1), the parameters of the first image generation model are the parameters adjusted by the previous training process of the current training process (i.e., the N-1th training process of the first image generation model based on the training system).

[0078] In this step, the target program may input the acquired first image description into the first image generation model, and process the first image description using the first image generation model to obtain a first generated image corresponding to the first image description.

[0079] Then, in step S230, the first generated image and the first image description are used to obtain a predicted weight value through the target reward model, and the predicted weight value is positively correlated with the image quality of the first generated image and / or is positively correlated with the degree of match between the first image description and the first generated image. In one implementation, the target reward model can be pre-stored in a preset data storage space corresponding to the target program, and accordingly, the target program can obtain the target reward model from the preset data storage space.

[0080] In one embodiment, the target reward model may be a model pre-trained based on any image-text pair in an image-text dataset (i.e., the aforementioned second image-text dataset) and the label weight value corresponding to the image-text pair. Wherein, any image-text pair in the second image-text dataset includes an image description and its corresponding image. For the sake of clarity, any image-text pair in the second image-text dataset is referred to as a second image-text pair, any image description in the second image-text pair is referred to as a third image description, and any image in the second image-text pair in the second image-text dataset is referred to as a third sample image.

[0081] In one implementation, the third sample image may be an image generated by the trained second image generation model according to the corresponding third image description.

[0082] The label weight value y can be the weight value annotated by the annotator or a specific program for each second image-text pair in the second image-text data set. The label weight value y can characterize the quality of the corresponding second image-text pair, wherein the better the quality of the second image-text pair, the larger the corresponding label weight value y. In one case, the value range of the label weight value y can be [0, 1] (i.e., the value range is 0 to 1). Correspondingly, the value range of the prediction weight value can be [0, 1] (i.e., the value range is 0 to 1).

[0083] The quality of each second image-text pair can be determined by the image quality of the third sample image contained therein, and / or the degree of match between the third image description contained therein and the third sample image. The better the image quality of the third sample image contained in the second image-text pair, the higher the degree of match between the third image description contained in the second image-text pair and the third sample image, which can indicate that the quality of the second image-text pair is better. Accordingly, the label weight value corresponding to the second image-text pair is positively correlated with the image quality of the third sample image contained in the second image-text pair, and / or is positively correlated with the degree of match between the third image description contained in the second image-text pair and the third sample image.

[0084] In one implementation, the image quality may be determined by the clarity of the image and / or the aesthetics of the image, wherein the higher the clarity of the image and the better the aesthetics of the image, the better the image quality of the image.

[0085] Accordingly, the label weight value y can be determined based on at least one of the following information: the clarity score of the third sample image (which can characterize the clarity of the third sample image, wherein the larger the clarity score, the higher the clarity), the aesthetic score of the third sample image (which can characterize the aesthetics of the image, wherein the larger the aesthetic score, the better the aesthetics of the image), and the matching value between the third sample image and its corresponding third image description (which can characterize the matching degree, i.e., the correlation degree, between the third sample image and its corresponding third image description, wherein the larger the matching value, the stronger the matching degree). It can be understood that the above information is only an exemplary example of reference information for determining the label weight value, and the disclosed embodiment can also use other information that can characterize the image quality of the image to determine the label weight value corresponding to the image-text pair.

[0086] By using the above information to determine the label weight value, to a certain extent, the label weight value can better represent the human quality evaluation of the corresponding image and its image description, thereby improving the accuracy of the predicted weight value output by the target reward model and making the predicted weight value more adapted to human intentions.

[0087] In another implementation, the disclosed embodiment may also provide a training process of a target reward model. For a clear layout, the training process of the target reward model is introduced below, wherein the training process of the target reward model includes the following steps 11-15: In step 11, a third image description and its corresponding third image description are obtained. In this step, the target program may obtain the third image description and its corresponding third image description from the aforementioned second image and text data set (or the aforementioned training data set); or, the target program may obtain an image description from the aforementioned public data set or an artificially created data set as the third image description, and then obtain a third sample image corresponding to the third image description through the aforementioned trained second image generation model according to the third image description.

[0088] Then, in step 12, the label weight value corresponding to the third image description and the third sample image is obtained. As mentioned above, the label weight value may be a weight value annotated by an annotator or a specific program for the third image description and its corresponding third sample image, which may characterize the quality of the image-text pair composed of the corresponding third image description and its corresponding third sample image. Specifically, the label weight value is positively correlated with the image quality of the third sample image, and / or is positively correlated with the matching degree between the third image description and the third sample image.

[0089] Then in step 13, the third image description and the third sample image are used to obtain an evaluation weight value through the reward model to be trained. In this step, the target program can input the third image description and the third sample image into the reward model to be trained, and use the reward model to be trained to encode the third image description and the third sample image respectively to obtain the text representation corresponding to the third image description and the image representation corresponding to the third sample image. After that, the reward model to be trained is used to determine the evaluation weight value corresponding to the third image description and the third sample image based on the text representation corresponding to the third image description and the image representation corresponding to the third sample image. Among them, the evaluation weight value can represent the quality evaluated by the reward model to be trained for the third image description and the third sample image.

[0090] In one implementation, the reward model to be trained may include an image encoder, a text encoder and an MLP (Multilayer Perceptron) layer. Accordingly, the target program may input the third sample image into the image encoder to obtain an image representation corresponding to the third sample image, input the third image description into the text encoder to obtain a text representation corresponding to the third image description, and then input the image representation corresponding to the third sample image and the text representation corresponding to the third image description into the MLP layer to obtain an evaluation weight value corresponding to the third image description and the third sample image.

[0091] In one case, the target program may also input the third sample image and its corresponding third image description into a trained CLIP (Contrastive Language-Image Pre-Training) model to obtain an image representation corresponding to the third sample image and a text representation corresponding to the third image description, and then input the image representation corresponding to the third sample image and the text representation corresponding to the third image description into the aforementioned MLP layer to obtain an evaluation weight value corresponding to the third image description and the third sample image.

[0092] After obtaining the evaluation weight value, in step 14, the third prediction loss is determined using the evaluation weight value and the label weight value. In this step, the third prediction loss can be determined using the evaluation weight value and the label weight value according to the specified loss function. The greater the difference between the evaluation weight value and the label weight value, the greater the third prediction loss. In one implementation, the specified loss function can be an MSE (mean square error) loss function, wherein the third prediction loss It can be expressed by the following formula (1): in, represents the parameters to be adjusted of the reward model to be trained, a represents the third sample image, b represents the third image description, y represents the label weight value corresponding to the third sample image and the third image description, It represents the evaluation weight value determined by the reward model to be trained for the input third sample image and the third image description, and E can represent the calculated expected value.

[0093] Then in step 15, with the goal of minimizing the third prediction loss, the reward model to be trained is adjusted to obtain a target reward model that meets the first convergence condition. In this step, the target program determines the model parameter gradient of the reward model to be trained based on the third prediction loss using the back propagation algorithm; determines the updated model parameters of the reward model to be trained (i.e., the updated values ​​of the model parameters of the reward model to be trained) using the determined model parameter gradient and the current model parameters of the reward model to be trained (i.e., the current values ​​of the model parameters of the reward model to be trained); and then adjusts the model parameters of the reward model to be trained based on the updated model parameters. Among them, the determination of the model parameter gradient of the reward model to be trained is obtained with the goal of minimizing the third prediction loss. Minimizing the third prediction loss can achieve supervised training of the determination results of the reward model to be trained, maintain its accuracy, and improve the accuracy of the determination results of the reward model to be trained.

[0094] The above steps 11 to 15 are a model iterative training process for the reward model to be trained. In order to train a better reward model to be trained, the above process can be iterated multiple times. That is, after step 15, based on the updated model parameters of the reward model to be trained, return to step 11.

[0095] The stopping condition of the above-mentioned model iterative training process, i.e., the first convergence condition, may include: the number of iterative training times reaches a preset threshold, or the iterative training duration reaches a preset duration, or the third prediction loss is less than a set loss threshold, etc.

[0096] After the target program obtains the target reward model, it uses the first generated image and the first image description to obtain the predicted weight value corresponding to the first generated image and the first image description through the target reward model, that is, the first generated image and the first image description are input into the target reward model, and the first generated image and the first image description are processed by the target reward model to obtain the predicted weight value. Among them, the predicted weight value can characterize the quality predicted by the target reward model for the first generated image and the first image description. Specifically, the predicted weight value is positively correlated with the image quality of the first generated image, and / or, is positively correlated with the matching degree between the first image description and the first generated image. Among them, the predicted weight value can also characterize the quality of the image generated by the first image generation model for the first image description, and can also characterize the ability of the first image generation model to generate images for the first image description.

[0097] After obtaining the first generated image and the prediction weight value corresponding to the first generated image and the first image description, in step S240, the parameters of the first image generation model are adjusted based on the difference between the first generated image and the first sample image and the prediction weight value. In this step, the target program can determine the first prediction loss based on the difference between the first generated image and the first sample image and the prediction weight value, and then adjust the parameters of the first image generation model with the goal of minimizing the first prediction loss.

[0098] Specifically, in one embodiment, step S240 may include the following steps 21-23:

[0099] In step 21, an intermediate prediction loss is determined according to the difference between the first generated image and the first sample image using a preset loss function. In one implementation, the preset loss function may be a cross entropy loss function. In this step, the target program may determine the cross entropy loss between the first generated image and the first sample image based on the cross entropy loss function according to the difference between the first generated image and the first sample image as the intermediate prediction loss. The greater the difference between the first generated image and the first sample image, the greater the intermediate prediction loss.

[0100] The intermediate prediction loss can be expressed by the following formula (2): l(θ) = E (x1，z1)～D1[-log p(x1|z1)](2); wherein l(θ) represents the intermediate prediction loss, θ represents the parameters to be adjusted involved in the first image generation model to be trained, x1 represents the first sample image, z1 represents the first image description, p(x1|z1) represents the probability that the first generated image obtained by inputting the first image description into the first image generation model is the same as the first sample image, that is, the probability that the first image generation model obtains the first sample image according to the first image description, and the probability can be obtained in the process of the first image generation model generating the first generated image based on the first image description. The p(x1|z1) can represent the difference between the first generated image and the first sample image, wherein the larger the p(x1|z1), the smaller the difference between the first generated image and the first sample image, D1 represents the aforementioned training data set, and E represents the calculated expected value.

[0101] In another implementation, the preset loss function may also be an MSE (mean square error) loss function, and the MSE loss function may be used to determine the difference between the first generated image and the first sample image, and then determine the intermediate prediction loss based on the difference. The intermediate prediction loss is positively correlated with the difference between the first generated image and the first sample image.

[0102] Then, in step 22, a first prediction loss is determined based on the product of the intermediate prediction loss and the prediction weight value. In this step, the target program may first calculate the product of the intermediate prediction loss and the prediction weight value, and then determine the first prediction loss based on the product. In one implementation, the first prediction loss may be represented by the following formula (3):

[0103] Among them, L1(θ) represents the first prediction loss, represents the aforementioned predicted weight value, that is, the predicted weight value determined by the target reward model for the first generated image and the first image description.

[0104] It can be understood that in the process of training the first image generation model, the greater the probability that the first generated image obtained by inputting the first image description into the first image generation model is the same as the first sample image, the smaller the image difference between the first generated image and the first sample image, and the smaller the first prediction loss. In the process of training the first image generation model, the larger the p(x1|z1), the better the image generation capability of the first image generation model. Accordingly, in step 23, the parameters of the first image generation model are adjusted with the goal of minimizing the first prediction loss.

[0105] In this step, the target program determines the adjustment gradient of the parameters of the first image generation model based on the first prediction loss using the back propagation algorithm; then determines the updated parameters of the first image generation model (i.e., the updated values ​​of the parameters) using the determined adjustment gradient and the current parameters of the first image generation model (i.e., the current values ​​of the parameters); and then adjusts the parameters of the first image generation model based on the updated parameters. The determination of the parameter gradient of the first image generation model is obtained with the goal of minimizing the first prediction loss. Minimizing the first prediction loss can achieve supervision of the generation results of the first image generation model and improve the quality of the images it generates.

[0106] In this embodiment, based on the online reinforcement learning method, combined with the difference between the first generated image and the first sample image, and the prediction weight value that can characterize the quality of the image-text pair composed of the first generated image and the first image description (for example, characterizing the image quality of the first generated image and / or the matching degree between the first generated image and the first image description), the first image generation model is trained, and the first image generation model is trained based on the online reinforcement learning method and combined with environmental feedback (i.e., feedback on the image generation ability of the environment pair). In the process of model training, it can be achieved that more emphasis is placed on using better data with better quality (for example, generated images with higher corresponding prediction weight values ​​and their corresponding sample images, which increase their proportion in determining the first prediction loss) to train the first image generation model, so as to better improve the image generation ability of the first image generation model and ensure the stability of its image generation ability. And to a certain extent, it is ensured that the quality of the image generated by the trained first image generation model is more in line with environmental expectations (i.e., human intentions).

[0107] Moreover, the above-mentioned prediction weight value is determined in real time during the training process of the first image generation model for the first image description and the first generated image generated by the first image generation model according to the first image description, which can better improve the learning speed (i.e., convergence speed) of the first image generation model and the image generation quality.

[0108] The above steps S210 to S240 are a model iterative training process for the first image generation model. In order to train a first image generation model with better image generation capability, the above process can be iterated multiple times. That is, after step S240, based on the updated parameters of the first image generation model, return to step S210 until the stop condition of the model iterative training process is reached, and the trained first image generation model is obtained.

[0109] The stopping conditions of the model iterative training process of the first image generation model may include: the number of iterative training times reaches a preset number threshold, or the iterative training duration reaches a preset duration, or the first prediction loss is less than a set threshold, etc.

[0110] Let's review the training process of the aforementioned first image generation model. The above embodiment is explained by taking a pair of image-text pairs (i.e., a pair of first image descriptions and their corresponding first sample images) as an example. In another embodiment, the above training process can also be performed for a batch of samples, i.e., multiple pairs of image-text pairs, to obtain the generated images corresponding to the image descriptions in each pair of image-text pairs, and obtain the prediction weight values ​​corresponding to the image descriptions in each pair of image-text pairs and the corresponding generated images, and then based on the difference between the generated images corresponding to the image descriptions in each pair of image-text pairs and the sample images in the pair of image-text pairs, and the prediction weight values ​​corresponding to the image descriptions in the pair of image-text pairs and the corresponding generated images, determine the first prediction loss, and adjust the parameters of the first image generation model to be trained with the goal of minimizing the first prediction loss. In this embodiment, the first prediction loss is determined for a batch of samples, and then the parameters of the first image generation model are adjusted, so that the number of parameter adjustments to the first image generation model can be reduced, making it easier to implement the training process.

[0111] In one embodiment, in order to better improve the image generation capability of the first image generation model, Figure 1 As shown, the training system of the image generation model may also include a pre-trained second image generation model. Figure 3 As shown, the method may include the following steps S310-S350:

[0112] In step S310, a first image description and a first sample image corresponding thereto are obtained. Figure 2 The step S210 shown in FIG. 1 is not described in detail here.

[0113] In step S320, the first image description is used to generate a first generated image through the first image generation model. Figure 2 The step S220 shown in FIG. 1 is not described in detail here.

[0114] In step S330, the first image description is used to obtain a second generated image through a pre-trained second image generation model. In this step, the target program can input the first image description into the second image generation model to process the first image description using the second image generation model to obtain a second generated image corresponding to the first image description.

[0115] Then, in step S340, the first generated image and the first image description are used to obtain a predicted weight value through the target reward model, and the predicted weight value is positively correlated with the image quality of the first generated image and / or is positively correlated with the matching degree between the first image description and the first generated image. Figure 2The step S230 shown in FIG. 1 is not described in detail here.

[0116] Understandably, Figure 3 A flow chart of the training method of the image generation model is exemplarily shown, which does not limit the execution order of the steps in the training method of the image generation model, wherein step S340 can be executed after step S330 or before step S330. In another case, step S330 can be executed after step S320 or before step S320, or in parallel with step S320.

[0117] After obtaining a generated image, a second generated image and a predicted weight value, in step S350, the parameters of the first image generation model are adjusted based on the difference between the first generated image and the first sample image, the predicted weight value, and the difference between the first generated image and the second generated image.

[0118] In this implementation, in order to better improve the image generation capability of the first image generation model and ensure the stability of its image generation capability, the training system of the image generation model may also include a pre-trained second image generation model, the first image description is input into the second image generation model, and a second generated image is obtained. Then, in step S350, the difference between the second generated image and the first generated image is used to punish the situation that the image generated by the first image generation model deviates too far from the image generated by the second image generation model, wherein the greater the difference between the second generated image and the first generated image, the greater the penalty, so as to avoid the first image generation model generating a chaotic image (an image that does not match the input image description) to deceive the target reward model, causing the target reward model to output a numerically higher prediction weight value for the chaotic image generated by the first image generation model. Accordingly, in step S350, based on the difference between the first generated image and the second generated image, and the difference and prediction weight value between the first generated image and the first sample image, the first prediction loss is jointly determined, and then the parameters of the first image generation model are adjusted with the goal of minimizing the first prediction loss.

[0119] Specifically, in one embodiment, step S350 may include the following steps 31-33:

[0120] In step 31 , an image difference value between the first generated image and the second generated image is determined based on the difference between the first generated image and the second generated image.

[0121] In one implementation, step 31 may specifically include: using a preset divergence algorithm to determine a divergence value based on the difference between the first generated image and the second generated image; and determining an image difference value based on the divergence value. The preset divergence algorithm may be a KL (Kullback-Leibler) divergence algorithm, using the KL divergence algorithm to determine a KL divergence value based on the difference between the first generated image and the second generated image, and then determining an image difference value between the first generated image and the second generated image based on the KL divergence value (for example, determining the KL divergence value as the image difference value), wherein the smaller the KL divergence value, that is, the smaller the image difference value, the smaller the difference between the first generated image and the second generated image, that is, the smaller the deviation between the image generated by the first image generation model and the image generated by the second image generation model.

[0122] In one implementation, the image difference value, i.e., the KL divergence value, can be expressed by the following formula (4):

[0123] l KL =log[π SFT (x1|z1) / π RL (x1|z1)](4), where l KL Represents the KL divergence value, i.e., the image difference value, π SFT (x1|z1) represents the probability that the second image generation model obtains the first sample image x1 for the first image description z1, that is, the probability that the second generated image obtained by the second image generation model for the first image description z1 is the same as the first sample image x1, which can be obtained in the process of the second image generation model generating the second generated image based on the first image description; π RL (x1, z1) represents the probability that the first image generation model obtains the first sample image x1 for the first image description z1, which is equal to the aforementioned p(x1|z1).

[0124] In another implementation, the target program may also calculate the Euclidean distance or mean square error between the first generated image and the second generated image to determine the image difference value between the first generated image and the second generated image, wherein the image difference value is positively correlated with the Euclidean distance or mean square error between the first generated image and the second generated image.

[0125] Then, in step 32, a first prediction loss is determined based on the difference between the first generated image and the first sample image, the prediction weight value, and the image difference value. In this step, the target program may first determine an intermediate prediction loss based on the difference between the first generated image and the first sample image, then calculate the product of the intermediate prediction loss and the prediction weight value, and then jointly determine the first prediction loss based on the product and the image difference value, wherein the first prediction loss is positively correlated with the product and the image difference value, respectively, that is, the smaller the product and the image difference value, the smaller the first prediction loss.

[0126] The first loss prediction is determined by combining the image difference value (i.e., the aforementioned KL divergence value), which can better implement the constraints on the first image generation model. The greater the difference between the first generated image and the second generated image, the more the output (generated image) of the first image generation model deviates from the output (generated image) of the second image generation model. Accordingly, a greater penalty needs to be given to the first image generation model to avoid the first image generation model generating confusing images (i.e., images that do not match the input image description) to deceive the target reward model, causing the target reward model to output a numerically higher prediction weight value for the confusing images generated by the first image generation model, so as to better improve the image generation capability of the first image generation model and ensure that the quality of the images generated by the first image generation model is better.

[0127] Accordingly, in one implementation, the first prediction loss L1(θ) can be expressed by the following formula (5):

[0128]

[0129] Wherein, φ is a preset parameter weight value, as mentioned above, θ represents the parameters involved in the first image generation model to be trained, x1 represents the first sample image, z1 represents the first image description, and p(x1|z1) represents the probability that the first image generation model obtains the first sample image for the first image description; represents the aforementioned predicted weight value, that is, the predicted weight value determined by the target reward model for the first generated image and the first image description, and D1 represents the aforementioned training data set.

[0130] Next, in step 33, the parameters of the first image generation model are adjusted with the goal of minimizing the first prediction loss.

[0131] In this implementation, the first prediction loss is determined by combining the image difference value and the difference between the first generated image and the first sample image, and the prediction weight value, and then the parameters of the first image generation model are adjusted with the goal of minimizing the first prediction loss. This can improve the image generation capability of the first image generation model while preventing the image generated by the first image generation model from deviating too much from the image generated by the second image generation model, better ensure the stability of the image generation capability of the first image generation model, and better improve the quality of the image generated by the first image generation model.

[0132] In another embodiment, in order to better improve the image generation capability of the first image generation model, the method may further include the following steps 41-43: In step 41, a second image description and a corresponding second sample image are obtained. In one implementation, the second image description and the corresponding second sample image may be obtained from the aforementioned first image and text data set. Compared with the aforementioned generated image (i.e., the first sample image) generated by the input image description based on the second image generation model, the second sample image corresponding to the second image description is a real image, which can carry more diverse image data.

[0133] Then, in step 42, the third generated image is obtained by using the first image generation model using the second image description. And in step 43, the second prediction loss is determined using the difference between the second sample image and the third generated image. After the target program obtains the third generated image, it can determine the second prediction loss based on a preset loss function using the difference between the second sample image and the third generated image, wherein the greater the difference between the second sample image and the third generated image, the greater the second prediction loss. In one implementation, the preset loss function can be a cross entropy loss function. Among them, the second prediction loss L2(E) can be expressed by the following formula (6): L2(θ)=E (x2， z 2)～D2 [-lpg p(x2|z2)](6); D2 represents the data set to which the second image description and its corresponding second sample image belong (for example, the aforementioned first image and text data set); x2 represents the aforementioned second sample image, z2 represents the aforementioned second image description, E represents the calculated expected value, and p(x2|z2) represents the probability that the third generated image obtained by the first image generation model for the second image description is the same as the second sample image, that is, the probability that the first image generation model obtains the second sample image for the second image description, which can be obtained in the process of the first image generation model generating the third generated image for the second image description. The larger the p(x2|z2), the smaller the difference between the second sample image and the third generated image, and the smaller the second prediction loss.

[0134] In another implementation, the preset loss function can also be an MSE (mean square error) loss function. The MSE loss function is used to determine the difference between the second sample image and the third generated image, and then the second prediction loss is determined based on the difference, wherein the second prediction loss is positively correlated with the difference between the second sample image and the third generated image.

[0135] Accordingly, the aforementioned process of adjusting the parameters of the first image generation model with the goal of minimizing the first prediction loss may include: adjusting the parameters of the first image generation model with the goal of minimizing the first prediction loss and the second prediction loss. In this step, the target program may first determine the total prediction loss based on the first prediction loss and the second prediction loss, and then adjust the parameters of the first image generation model with the goal of minimizing the total prediction loss, wherein the total prediction loss is positively correlated with the first prediction loss and the second prediction loss, respectively. In one case, the total prediction loss L(θ) can be expressed by the following formula (7): L(θ)=L1(θ)+βL2(θ)(7); wherein β is the weight value corresponding to the preset second prediction loss.

[0136] It can be understood that the images in the image-text pairs in the aforementioned training data set are generated images using the aforementioned second image generation model for the corresponding input image descriptions. Relative to this type of generated images, the second sample image corresponding to the second image description is a real image. In this implementation, in the process of training the first image generation model, real images and their corresponding image descriptions are also added to train the first image generation model, which increases the diversity of data for training the first image generation model and better improves the image generation capability of the first image generation model.

[0137] Corresponding to the above method embodiment, the present disclosure provides an image generation method, which can be implemented by a first program. The first program can be the same physical program as the aforementioned target program, or can be a different physical program from the aforementioned target program, such as Figure 4 As shown, the method may include the following steps S410-S420:

[0138] In step S410, the image description to be processed is obtained. In one implementation, the image description to be processed may be any image description input by the user when there is a need to generate an image using the image description (for example, when generating corresponding illustrations for novels, news, etc.), and accordingly, the first program may obtain the image description input by the user as the image description to be processed.

[0139] In step S420, the target image is obtained by using the image description to be processed and training the image generation model obtained by the training method of the image generation model, that is, the first image generation model trained as described above. The process of obtaining the target image can refer to the process of obtaining the first generated image as described above, and will not be described in detail here.

[0140] In the above process, based on the description of the image to be processed, a target image with better quality can be obtained by training the first image generation model to improve the user experience.

[0141] The foregoing describes specific embodiments of the present disclosure, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims may be performed in an order different from that in the embodiments, and the desired results may still be achieved. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible, or may be advantageous.

[0142] Corresponding to the above method embodiment, the present disclosure provides an image generation model training device 500, whose schematic block diagram is as follows: Figure 5 As shown, it is used to execute Figure 2 or Figure 3 The method shown, the apparatus 500 may include:

[0143] A first acquisition module 510, configured to acquire a first image description and a first sample image corresponding thereto;

[0144] A first obtaining module 520 is configured to obtain a first generated image by using the first image description through a first image generation model;

[0145] a second obtaining module 530, configured to obtain a predicted weight value by using the first generated image and the first image description through a target reward model, wherein the predicted weight value is positively correlated with an image quality of the first generated image and / or is positively correlated with a matching degree between the first image description and the first generated image;

[0146] The first adjustment module 540 is configured to adjust the parameters of the first image generation model based on the difference between the first generated image and the first sample image and the prediction weight value.

[0147] In an optional implementation, the first adjustment module 540 is specifically configured to determine an intermediate prediction loss according to a difference between the first generated image and the first sample image using a preset loss function;

[0148] Determining a first prediction loss based on a product of the intermediate prediction loss and the prediction weight value;

[0149] The parameters of the first image generation model are adjusted with the goal of minimizing the first prediction loss.

[0150] In an optional implementation, it also includes:

[0151] a fourth obtaining module (not shown in the figure), configured to obtain a second generated image by using the first image description through a pre-trained second image generation model;

[0152] The first adjustment module 540 is specifically configured to adjust the parameters of the first image generation model based on the difference between the first generated image and the first sample image, the prediction weight value, and the difference between the first generated image and the second generated image.

[0153] In an optional implementation, the first adjustment module 540 includes:

[0154] a first determining unit (not shown in the figure), configured to determine an image difference value between the first generated image and the second generated image based on a difference between the first generated image and the second generated image;

[0155] A second determining unit (not shown in the figure) is configured to determine a first prediction loss based on the difference between the first generated image and the first sample image, the prediction weight value, and the image difference value;

[0156] An adjustment unit (not shown in the figure) is configured to adjust the parameters of the first image generation model with the goal of minimizing the first prediction loss.

[0157] In an optional implementation, the first determining unit is specifically configured to determine a divergence value between the first generated image and the second generated image based on a difference between the first generated image and the second generated image by using a preset divergence algorithm;

[0158] Based on the divergence value, the image difference value is determined.

[0159] In an optional implementation, the first sample image is an image generated by using the second image generation model according to the first image description.

[0160] In an optional implementation, it also includes:

[0161] A third acquisition module (not shown in the figure), configured to acquire a second image description and a second sample image corresponding thereto;

[0162] a fifth obtaining module (not shown in the figure), configured to obtain a third generated image by using the second image description and the first image generation model;

[0163] A first determination module (not shown in the figure), configured to determine a second prediction loss by using a difference between the second sample image and the third generated image;

[0164] The first adjustment module 540 is specifically configured to adjust the parameters of the first image generation model with the goal of minimizing the first prediction loss and the second prediction loss.

[0165] In an optional implementation, it also includes:

[0166] a fourth acquisition module (not shown in the figure), configured to acquire the third image description and the third sample image;

[0167] a fifth acquisition module (not shown in the figure), configured to acquire the label weight value corresponding to the third image description and the third sample image, wherein the label weight value is positively correlated with the image quality of the third sample image and / or is positively correlated with the matching degree between the third image description and the third sample image;

[0168] a sixth obtaining module (not shown in the figure), configured to obtain an evaluation weight value by using the third image description and the third sample image through a reward model to be trained;

[0169] A second determination module (not shown in the figure) is configured to determine a third prediction loss using the evaluation weight value and the label weight value;

[0170] The second adjustment module (not shown in the figure) is configured to adjust the reward model to be trained with the goal of minimizing the third prediction loss to obtain a target reward model that meets the first convergence condition.

[0171] In an optional implementation, the tag weight value is determined based on at least one of the following information: a clarity score of the third sample image, an aesthetic score of the third sample image, and a matching value between the third sample image and the third image description.

[0172] Corresponding to the above method embodiment, the present disclosure embodiment provides an image generating device 600, whose schematic block diagram is as follows: Figure 6 As shown, it is used to execute Figure 4 The method shown, the apparatus 600 may include:

[0173] A second acquisition module 610 is configured to acquire a description of an image to be processed;

[0174] The third obtaining module 620 is configured to obtain a target image by using the image description to be processed and the image generation model obtained by training using the aforementioned image generation model training device.

[0175] The above device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.

[0176] The present disclosure provides an electronic device, including a memory and a processor, wherein the memory stores an executable code, and when the processor executes the executable code, the following is implemented: Figure 2 , Figure 3 or Figure 4 The method shown below. Figure 7 , which shows a schematic structural diagram of an electronic device suitable for implementing an embodiment of the present disclosure. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0177] like Figure 7 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In RAM 703, various programs and data required for the operation of the electronic device are also stored. The processing device 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0178] Typically, the following devices may be connected to the I / O interface 705: input devices 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, etc.; output devices 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 708 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 709. The communication devices 709 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Figure 7 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 7 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0179] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0180] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute the following steps provided in the present disclosure: Figure 2 or Figure 3 The training method of the image generation model shown in Figure 4 The image generation method shown.

[0181] It should be noted that the computer-readable medium described in the embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0182] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains a first image description and a first sample image corresponding to the first image description; obtains a first generated image by using the first image description through a first image generation model; obtains a predicted weight value by using the first generated image and the first image description through a target reward model, wherein the predicted weight value is positively correlated with the image quality of the first generated image and / or is positively correlated with the matching degree between the first image description and the first generated image; adjusts the parameters of the first image generation model based on the difference between the first generated image and the first sample image and the predicted weight value. Alternatively, the electronic device: obtains a description of the image to be processed; obtains a target image by using the image generation model trained by using the image generation model training method of the aforementioned claim using the image description to be processed.

[0183] Computer program code for performing the operations of the disclosed embodiments may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0184] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the storage medium and computing device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0185] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present invention may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0186] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the embodiments of the present invention in detail. It should be understood that the above description is only a specific implementation method of the embodiments of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A training method for an image generation model, comprising: Obtaining a first image description and a first sample image corresponding thereto; Using the first image description and a first image generation model, a first generated image is obtained; Using the first generated image and the first image description, a predicted weight value is obtained through a target reward model, wherein the predicted weight value is positively correlated with an image quality of the first generated image and / or is positively correlated with a degree of match between the first image description and the first generated image; Based on the difference between the first generated image and the first sample image and the prediction weight value, the parameters of the first image generation model are adjusted.

2. The method of claim 1, wherein: The adjusting the parameters of the first image generation model comprises: Determining an intermediate prediction loss according to a difference between the first generated image and the first sample image using a preset loss function; Determining a first prediction loss based on a product of the intermediate prediction loss and the prediction weight value; The parameters of the first image generation model are adjusted with the goal of minimizing the first prediction loss.

3. The method of claim 1, further comprising: Using the first image description, a second generated image is obtained through a pre-trained second image generation model; The adjusting the parameters of the first image generation model comprises: The parameters of the first image generation model are adjusted based on the difference between the first generated image and the first sample image, the prediction weight value, and the difference between the first generated image and the second generated image.

4. The method of claim 3, wherein: The adjusting the parameters of the first image generation model comprises: determining an image difference value between the first generated image and the second generated image based on a difference between the first generated image and the second generated image; determining a first prediction loss based on a difference between the first generated image and the first sample image, the prediction weight value, and the image difference value; The parameters of the first image generation model are adjusted with the goal of minimizing the first prediction loss.

5. The method of claim 4, wherein: The determining an image difference value between the first generated image and the second generated image comprises: Determining a divergence value based on a difference between the first generated image and the second generated image using a preset divergence algorithm; Based on the divergence value, the image difference value is determined.

6. The method of claim 3, wherein: The first sample image is an image generated by using the second image generation model according to the first image description.

7. The method of claim 2, further comprising: Obtaining a second image description and a second sample image corresponding to the second image description; Using the second image description and the first image generation model, a third generated image is obtained; determining a second prediction loss using a difference between the second sample image and the third generated image; The step of adjusting the parameters of the first image generation model with the goal of minimizing the first prediction loss includes: The parameters of the first image generation model are adjusted with the goal of minimizing the first prediction loss and the second prediction loss.

8. The method according to any one of claims 1 to 7, further comprising: Obtaining the third image description and its corresponding third sample image; Acquire the label weight value corresponding to the third image description and the third sample image, wherein the label weight value is positively correlated with the image quality of the third sample image and / or is positively correlated with the matching degree between the third image description and the third sample image; Using the third image description and the third sample image, through the reward model to be trained, to obtain an evaluation weight value; Determining a third prediction loss using the evaluation weight value and the label weight value; With the goal of minimizing the third prediction loss, the reward model to be trained is adjusted to obtain a target reward model that meets the first convergence condition.

9. The method according to any one of claims 1 to 7, wherein: The tag weight value is determined based on at least one of the following information: a clarity score of the third sample image, an aesthetic score of the third sample image, and a matching value between the third sample image and the third image description.

10. A method for generating an image, comprising: Get the image description to be processed; The target image is obtained by using the image description to be processed and training the image generation model obtained by using the image generation model training method described in any one of claims 1 to 9.

11. A training device for an image generation model, comprising: A first acquisition module, configured to acquire a first image description and a first sample image corresponding thereto; A first obtaining module is configured to obtain a first generated image by using the first image description through a first image generation model; a second obtaining module configured to obtain a predicted weight value by using the first generated image and the first image description through a target reward model, wherein the predicted weight value is positively correlated with an image quality of the first generated image and / or is positively correlated with a matching degree between the first image description and the first generated image; The first adjustment module is configured to adjust the parameters of the first image generation model based on the difference between the first generated image and the first sample image and the prediction weight value.

12. An image generating device, comprising: A second acquisition module is configured to acquire a description of the image to be processed; The third obtaining module is configured to obtain a target image by using the image description to be processed and the image generation model obtained by training using the image generation model training device according to claim 11.

13. An electronic device comprising a memory and a processor, wherein: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 10 is implemented.

14. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 10.

Images