Text generation model training method and device, image generation method and device, equipment and medium

Abstract

The embodiment of the invention provides a text generation model training method and device, an image generation method and device, equipment and a medium. The method comprises the steps that first image description is acquired; determining a first generated image through a target image generation model based on the first image description; based on the first image description, through a text generation model, obtaining corresponding prediction forward description and prediction reverse description; determining a second generated image through a target image generation model according to the predicted forward description and the predicted reverse description; obtaining a first quality score for the first generated image and a second quality score for the second generated image; and on the basis of comparison of the first quality score and the second quality score, parameters of the text generation model are adjusted to realize training to obtain a model capable of generating image description with better quality, so that the image description with better quality can be obtained.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a text generation model training, image generation method, device, equipment and medium. Background Art

[0002] In the field of image generation, image description (also called text prompt) can be used to guide image generation. Specifically, the image description can be input into an image generation model based on content generation technology, and accordingly, the image generation model can output an image corresponding to the image description.

[0003] In the above process, the quality of image description is crucial to improving the quality of generated images. Summary of the invention

[0004] One or more embodiments of the present disclosure provide a text generation model training, image generation method, apparatus, device and medium to achieve training to obtain a model that can generate better quality image descriptions, so as to obtain better quality image descriptions, and then optimize the quality of images generated based on the image descriptions.

[0005] According to a first aspect, a method for training a text generation model is provided, comprising:

[0006] Obtaining a first image description;

[0007] Based on the first image description, determining a first generated image through a target image generation model;

[0008] Based on the first image description, obtaining a corresponding predicted positive description and a predicted negative description through a text generation model;

[0009] Determining a second generated image through the target image generation model according to the predicted forward description and the predicted reverse description;

[0010] Obtaining a first quality score for the first generated image and a second quality score for the second generated image;

[0011] Based on the comparison of the first quality score and the second quality score, a parameter of the text generation model is adjusted.

[0012] In an optional embodiment, it also includes: obtaining a first positive description label and a first reverse description label corresponding to the first image description; the adjusting the parameters of the text generation model includes: determining a target loss based on a comparison of the first quality score and the second quality score, and the first output related to the predicted positive description and the second output related to the predicted reverse description, the first positive description label and the first reverse description label obtained by the text generation model; adjusting the parameters of the text generation model with the goal of minimizing the target loss. In this implementation, a target loss is constructed in combination with a comparison of the first quality score and the second quality score, and the first output related to the predicted positive description and the second output related to the predicted reverse description, the first positive description label and the first reverse description label obtained by the text generation model, so as to obtain a target loss that can better feedback the state of the text generation model, and then based on the target loss, the parameters of the text generation model can be better adjusted, so that the adjusted text generation model can obtain higher quality image descriptions that are more in line with user expectations and image descriptions that do not meet user expectations, and better improve its image description optimization capability.

[0013] In an optional embodiment, the determining the target loss includes: determining an image optimization reward parameter based on the difference between the second quality score and the first quality score; determining a model prediction loss based on the first output and the first forward description label, and the second output and the first reverse description label; and determining the target loss based on the product of the image optimization reward parameter and the model prediction loss. In this implementation, the target loss is determined based on the difference between the second quality score and the first quality score, i.e., the current environment reward, and the model prediction loss, to obtain a target loss that can better adjust the parameters of the text generation model so that the text generation model can be aligned with the user's preferences and with the target image generation model.

[0014] In an optional implementation, the determining of the second generated image includes: obtaining image features corresponding to the predicted forward description and the predicted reverse description respectively through the target image generation model based on the predicted forward description and the predicted reverse description respectively: determining the second generated image using the image features corresponding to the predicted forward description and the image features corresponding to the predicted reverse description, so as to obtain a generated image that can better characterize the quality of the predicted forward description and the predicted reverse description, and provide a basis for better adjusting the text generation model.

[0015] In an optional embodiment, the first quality score includes at least one of the following: a first aesthetic score of the first generated image, a first clarity score of the first generated image, and a first correlation value between the first generated image and the first image description; the second quality score includes at least one of the following: a second aesthetic score of the second generated image, a second clarity score of the second generated image, and a second correlation value between the second generated image and the first image description. The quality of the generated image can be determined in a variety of ways to provide more accurate environmental feedback, so as to better improve the image description generation optimization capability of the text generation model, so that the text generation model is more aligned with the target image generation model.

[0016] In an optional implementation, the first aesthetic score of the first generated image is determined based on the aesthetic prediction model for the first generated image; the first clarity score of the first generated image is calculated based on the preset gradient algorithm for the first generated image; the first correlation value between the first generated image and the first image description is determined based on the image-text correlation model for the first generated image and the first image description. In this implementation, the corresponding score of the generated image is determined by the corresponding trained model to ensure accurate evaluation of the quality of the generated image, thereby ensuring a better performance text generation model obtained through training.

[0017] In an optional embodiment, before obtaining the corresponding predicted positive description and predicted reverse description through the text generation model based on the first image description, it also includes: obtaining the second image description and its corresponding second positive description label and second reverse description label; obtaining the third output and fourth output of the text generation model to be trained based on the second image description through the text generation model to be trained; constructing the description prediction loss based on the third output and the second positive description label, and the fourth output and the second reverse description label; adjusting the parameters of the text generation model to be trained with the goal of minimizing the description prediction loss until the text generation model to be trained reaches a preset convergence condition, thereby obtaining the text generation model. The text generation model to be trained is pre-trained so that the text generation model has the initial ability to generate image descriptions, and can generate positive descriptions that meet the user's expectations and reverse descriptions that do not meet the user's expectations, so that the text generation model is aligned with the user's preferences, and it is more convenient to fine-tune (train) the parameters of the text generation model later.

[0018] According to a second aspect, there is provided an image generation method, comprising:

[0019] Get the description of the image to be processed;

[0020] Based on the description of the image to be processed, a target forward description and a target reverse description are obtained through a text generation model;

[0021] Based on the target forward description and the target reverse description, a target image is generated through an image generation model.

[0022] In an optional embodiment, the text generation model is trained using any of the text generation model training methods described in the first aspect, so that images can be generated based on the text to obtain better quality target positive descriptions that meet user expectations and target reverse descriptions that do not meet user expectations.

[0023] In an optional embodiment, generating the target image includes: obtaining image features corresponding to the target forward description and the target reverse description respectively based on the target forward description and the target reverse description through the image generation model; and determining the target image using the image features corresponding to the target forward description and the image features corresponding to the target reverse description to obtain a target image with better quality.

[0024] According to a third aspect, a training device for a text generation model is provided, comprising:

[0025] A first acquisition module, configured to acquire a first image description;

[0026] A first determination module is configured to determine a first generated image based on the first image description by using a target image generation model;

[0027] A first obtaining module is configured to obtain a corresponding predicted positive description and a predicted reverse description based on the first image description through a text generation model;

[0028] A second determination module is configured to determine a second generated image through the target image generation model according to the predicted forward description and the predicted reverse description;

[0029] a second acquisition module, configured to acquire a first quality score for the first generated image and a second quality score for the second generated image;

[0030] The first adjustment module is configured to adjust the parameters of the text generation model based on a comparison of the first quality score and the second quality score.

[0031] According to a fourth aspect, there is provided an image generating device, comprising:

[0032] A third acquisition module is configured to acquire a description of the image to be processed;

[0033] A second obtaining module is configured to obtain a target forward description and a target reverse description through a text generation model based on the description of the image to be processed;

[0034] The generation module is configured to generate a target image through an image generation model based on the target forward description and the target reverse description.

[0035] According to a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method described in the first aspect or the second aspect.

[0036] According to a sixth aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described in the first aspect or the second aspect is implemented.

[0037] According to the training of the text generation model, the image generation method, apparatus, device and medium provided by the embodiments of the present disclosure, after obtaining the first image description, based on the first image description, the first generated image is determined through the target image generation model, then based on the first image description, the corresponding predicted forward description and predicted reverse description are obtained through the text generation model, and according to the predicted forward description and the predicted reverse description, the second generated image is determined through the target image generation model; then, a first quality score for the first generated image and a second quality score for the second generated image are obtained, and based on the comparison of the first quality score and the second quality score, that is, based on the comparison of the quality of the first generated image and the second generated image, the parameters of the text generation model are adjusted to achieve a better optimization of the text generation optimization capability of the text generation model based on the reinforcement learning method and combined with the environmental feedback (that is, the feedback on the quality of the image), so that the text generation model can optimize the output of a higher quality image description that is more in line with the user's expectations and an image description that is not in line with the user's expectations according to the input image description, and then based on the aforementioned image description, an image that is more in line with the user's expectations is obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1A A schematic diagram of an implementation framework of an embodiment disclosed in the present disclosure;

[0040] Figure 1BA schematic diagram of a flow chart of a pre-training process of a text generation model provided in an embodiment;

[0041] Figure 2 A flowchart of a method for training a text generation model provided in an embodiment;

[0042] Figure 3 A schematic diagram of another flow chart of a method for training a text generation model provided in an embodiment;

[0043] Figure 4 A schematic diagram of a flow chart of an image generation method provided in an embodiment;

[0044] Figure 5 A schematic block diagram of a training device for a text generation model provided in an embodiment;

[0045] Figure 6 A schematic block diagram of an image generating device provided in an embodiment;

[0046] Figure 7 A schematic block diagram of an electronic device provided in an embodiment. DETAILED DESCRIPTION

[0047] The technical solution of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0048] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0049] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0050] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0051] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0052] The embodiments of the present disclosure disclose a training method, device, equipment and medium for a text generation model, and an image generation method, device, equipment and medium. The application scenarios and technical concepts of the corresponding methods are first introduced as follows:

[0053] As mentioned above, in the field of image generation, image description (also called text prompt) can be used to guide image generation. Specifically, the image description can be input into the image generation model based on content generation technology, and accordingly, the image generation model can output the image corresponding to the image description. In the above process, the quality of the image description is crucial to improving the quality of the generated image.

[0054] In view of this, the embodiments of the present disclosure provide a training method, apparatus, device and medium for a text generation model, as well as an image generation method, apparatus, equipment and medium. By using the training method of the text generation model provided by the present disclosure, it is possible to train a text generation model with better image description generation expansion capabilities (i.e., text expansion). Figure 1A A schematic diagram of an implementation scenario according to an embodiment disclosed in the present disclosure is shown. In the implementation scenario, an exemplary structure of a training system for fine-tuning (training) a text generation model based on reinforcement learning and an exemplary data processing flow of the training system are shown. Among them, the training system may include a text generation model that needs to be trained (i.e., needs to be fine-tuned), a target image generation model that can generate an image based on an image description, and a reward model for determining feedback data (i.e., current environmental feedback, i.e., reward scores) for adjusting the text generation model. The text generation model is used to expand the input text (i.e., the input image description), wherein the expansion includes forward expansion and reverse expansion.

[0055] The text generation model may be a pre-trained text generation model. Specifically, the text generation model may be pre-trained based on the image description and its corresponding forward description label and reverse description label as label data. In one case, the image description may also be referred to as a text prompt, i.e., a prompt for generating an image.

[0056] The target image generation model may be a pre-trained image generation model. Specifically, the target image generation model may be pre-trained based on any image-text pair (i.e., image description and its corresponding image) in the image-text dataset. In one implementation, the target image generation model may be a generation model based on a Diffusion Model in the related art. The target image generation model may process the input image description to generate a corresponding image.

[0057] It can be understood that when the target image generation model is used to generate an image, when the target image generation model determines that there is no "negative prompt" (such as the reverse description in the present disclosure) input, it can generate an image corresponding to a normal prompt based on the input prompt (that is, the image description used to guide image generation, which can be called a normal prompt or a normal image description). When the target image generation model determines that there is a normal prompt input and there is a "negative prompt" input, the corresponding image can be generated based on the input normal prompt and the input "negative prompt".

[0058] The reward model is used to evaluate the quality of the input image and determine the quality score corresponding to the input image, and the quality score can characterize the quality of the corresponding image. In one implementation, the reward model can evaluate the quality of the image from the following angles, but not limited to the following angles: the clarity of the image (the clarity score of the image), the beauty of the image (i.e., the aesthetic score), and the correlation between the image and its corresponding image description (i.e., the degree of match between the image and its corresponding image description). Among them, the higher the image clarity (the larger the image clarity score), the better the image beauty (i.e., the higher the aesthetic score), and the stronger the correlation between the image and its corresponding image description, the higher the quality score corresponding to the image.

[0059] In the present disclosure, the quality score determined for the input image by the reward model is used to determine the feedback data (i.e., the current environment reward) used to train (i.e., fine-tune) the text generation model, so as to better improve the image description generation capability of the text generation model, and make the text generation model more aligned with human preferences and the target image generation model. The trained text generation model can output better quality forward descriptions that meet user expectations, and reverse descriptions that do not meet user expectations, thereby obtaining images with better quality. Among them, for a forward description obtained by forward extension of a trained text generation model based on a certain image description, the quality score of the image obtained based on the forward description is higher than the quality score of the image obtained based on the image description; for a reverse description obtained by reverse extension of a trained text generation model based on a certain image description, the quality score of the image obtained based on the reverse description is lower than the quality score of the image obtained based on the image description.

[0060] The following is an introduction to the fine-tuning process of the text generation model. Figure 1AAs shown, during the fine-tuning process, the target program can obtain a first image description from a sample data set constructed for training the text generation model, wherein the sample data set includes a plurality of image descriptions. Then, the target program determines a first generated image corresponding to the first image description through a target image generation model based on the first image description; obtains a predicted forward description and a predicted reverse description corresponding to the first image description through a text generation model based on the first image description; then determines to obtain a second generated image through the target image generation model based on the predicted forward description and the predicted reverse description; obtains a first quality score for the first generated image and a second quality score for the second generated image using a reward model; and then adjusts the parameters of the text generation model based on a comparison of the first quality score and the second quality score (which can reflect the current environmental feedback).

[0061] In the above process, the reinforcement learning idea is adopted, and the current environmental feedback is obtained based on the comparison of the first quality score and the second quality score, that is, based on the comparison of the quality of the first generated image corresponding to the first image description and the second generated image corresponding to the image description obtained by expansion of the text generation model (predicted forward description and predicted reverse description), and then the parameters of the text generation model are adjusted using the environmental feedback to achieve better improvement of the image description generation expansion capability of the text generation model based on the reinforcement learning method and combined with the environmental feedback (which can reflect the current image description generation expansion capability of the text generation model), so that the text generation model can output higher quality image descriptions that are more in line with user expectations and image descriptions that do not meet user expectations according to the input image description, thereby improving the quality of the image generated based on the image description, so that the image is more in line with user expectations.

[0062] In the following, in conjunction with specific embodiments, the training method, apparatus, device and medium of the text generation model, and the image generation method, apparatus, device and medium provided by the present disclosure are described in detail.

[0063] Figure 2 A flow chart of a method for training a text generation model in one embodiment of the present disclosure is shown. The method can be implemented by a target program. The target program can be installed in an electronic device, and the electronic device can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities.

[0064] For a clear layout, before introducing the training (i.e., fine-tuning) process of the text generation model based on reinforcement learning, we first introduce the pre-training process of the target image generation model and the pre-training process of the text generation model.

[0065] The pre-training process of the target image generation model may include: obtaining the image generation model to be trained, and obtaining a picture and text data set for training the image generation model, referred to as a first picture and text data set S, the first picture and text data set S may include multiple picture and text pairs, and a picture and text pair may include an image description X and its corresponding image Z. In one case, the first picture and text data set S may be obtained from any specified public data set that can provide picture and text pairs (images and their corresponding image descriptions) or a specified artificially created data set. The artificially created data set includes: each collected image and the image descriptions manually annotated for each image.

[0066] In one case, the image description may also be referred to as a text prompt, that is, a prompt for generating an image.

[0067] Then, for any image-text pair M in the first image-text dataset S, based on the image description X in the image-text pair M, the generated image Z' corresponding to the image description X is obtained through the image generation model to be trained; the image difference between the image Z in the image-text pair M and the generated image Z' is used to determine the loss corresponding to the image generation model, which is called the first generation loss. The greater the image difference between the image Z and the generated image Z', the greater the first generation loss.

[0068] Afterwards, the parameters of the image generation model are adjusted to minimize the first generation loss, that is, to reduce the image difference between the image in the image-text pair and the corresponding generated image, until the image generation model reaches a preset convergence condition, thereby obtaining a pre-trained image generation model, that is, obtaining a target image generation model. The preset convergence condition may include, but is not limited to, the number of parameter adjustments is not less than the first adjustment threshold, the training time exceeds the preset time, or the calculated first generation loss is less than the first loss threshold.

[0069] Let's review the aforementioned pre-training process of the image generation model. The above embodiment is explained by taking a pair of image-text pairs (i.e., a pair of image descriptions X and their corresponding images Z) as an example. In another embodiment, the above pre-training process can also be performed on a batch of samples, i.e., multiple pairs of image-text pairs, to obtain the generated images corresponding to the image descriptions in each pair of image-text pairs, and then based on the generated images corresponding to the image descriptions in each pair of image-text pairs and the images in the pair of image-text pairs, determine the first generation loss, and adjust the parameters of the image generation model with the goal of minimizing the first generation loss. In this embodiment, the first generation loss is determined for a batch of samples, and then the parameters of the image generation model are adjusted with the goal of minimizing the first generation loss. This can reduce the number of parameter adjustments to the image generation model and make it easier to implement the pre-training process.

[0070] In one implementation, the target image generation model may be an image generation model based on a Diffusion Model in the related art.

[0071] The following is an introduction to the pre-training process of the text generation model. Figure 1B As shown, the process may include the following steps S110-S140:

[0072] In step S110, a second image description and its corresponding second forward description label and second reverse description label are obtained. In this step, the target program can first construct a training data set required for the pre-trained text generation model, wherein the training data set includes a plurality of image description groups, wherein each image description group includes: an original image description and a forward description and a reverse description corresponding to the original image description.

[0073] In one implementation, the process of constructing a training data set may include: obtaining an image description i (i.e., an original image description that has not been processed by a specified generation model) from a specified data set, and then adding a first guiding content to the image description i, wherein the first guiding content is used to instruct the specified generation model (also referred to as a language model) to output a result that the user expects to output, i.e., used to guide the specified generation model to generate an image description that is richer in content, richer in details, and that makes the quality of the generated image better than the original image description (e.g., image description i), which is called a forward description (i.e., the quality of the image generated based on the forward description is better than the quality of the image generated based on the original image description); adding a second guiding content to the image description i, wherein the second guiding content is used to instruct the specified generation model to output a result that the user does not expect to output, i.e., used to guide the specified generation model to generate an image description that is inappropriate in content, lacks details, and that makes the quality of the generated image worse than the original image description, which is called a reverse description (i.e., the quality of the image generated based on the reverse description is inferior to the quality of the image generated based on the original image description).

[0074] In one case, the first guiding content includes, for example, the content for guiding the generation of a better quality description, such as the desire for richer details, more beautiful and clear images, and the desire for the animal's body shape to conform to biological common sense. The second guiding content includes, for example, the image is dim and unclear, and the generated animal's body shape does not conform to biological common sense (for example, the original image description is "cat", and the second guiding content includes, for example, the content of having 6 legs, etc.), and the content for guiding the generation of a poor quality description. The poor quality description makes the image quality generated based on it poor, for example, the image is unclear and / or the aesthetic score is low.

[0075] Afterwards, the target program obtains the forward description corresponding to the image description i (i.e., the result expected to be output by the user) through the specified generation model based on the image description i with the first guide content added. Specifically, the image description i with the first guide content added is input into the specified generation model, so that the specified generation model generates the forward description corresponding to the image description i according to the first guide content and based on the image description i; the target program obtains the reverse description corresponding to the image description i (i.e., the result not expected to be output by the user) through the specified generation model based on the image description i with the second guide content added. That is, the image description i with the second guide content added is input into the specified generation model, so that the specified generation model generates the reverse description corresponding to the image description i according to the second guide content and based on the image description i, thereby obtaining a group of image descriptions consisting of the image description i and its corresponding forward description and reverse description, and the forward description and reverse description corresponding to the image description i are used as label data for training the text generation model.

[0076] According to the above process, for several original image descriptions in the specified data set, several groups of image description groups corresponding to the several original image descriptions are obtained, that is, the training data set is constructed. It can be understood that the specified generation model can be any Transformer-based generative pre-training model in the relevant technology that can optimize the input image description.

[0077] The target program can obtain any image description group from the training data set constructed above, that is, obtain the image description (for the sake of clarity, called the second image description) and its corresponding forward description (for the sake of clarity, called the second forward description label) and reverse description (for the sake of clarity, called the second reverse description label). The second forward description label and the second reverse description label corresponding to the second image description serve as the label data required for pre-training the text generation model.

[0078] Then, in step S120, based on the second image description, the text generation model to be trained is used to obtain the third output and the fourth output of the text generation model to be trained. And in step S130, based on the third output and the second forward description label, and the fourth output and the second reverse description label, a description prediction loss is constructed.

[0079] It can be understood that the text generation model to be trained can correspond to a preset word set, which can be constructed according to the image description group in the training data set constructed above, and the preset word set includes a number of words (tokens), including at least the words involved in the image description group in the training data set. Both the forward description label and the reverse description label can include one or more words in the preset word set.

[0080] In this step, the target process obtains a third output and a fourth output based on the second image description using the text generation model to be trained. The third output may include a first word probability set corresponding to each of the several positions determined by the text generation model to be trained based on the second image description, wherein the first word probability set corresponding to each position may include the first probability of each word in the preset word set appearing at the position (i.e., the first probability of each word in the preset word set appearing at the position). The first word probability set may represent the probability corresponding to the result output by the text generation model for the second image description that meets the user's expectations, and based on the first word probability set, a positive description (which may be called an estimated positive description) corresponding to the second image description that meets the user's expectations (i.e., what the user wants to obtain) may be determined. The specific number of the several positions may be determined based on the specified parameters set for the text generation model to be trained, and may refer to the longest length of the model output.

[0081] The fourth output may include a second word probability set corresponding to each of the plurality of positions determined by the text generation model to be trained for the second image description, wherein the second word probability set corresponding to each position may include the second probability of each word in the preset word set appearing at the position (i.e., the second probability of each word in the preset word set appearing at the position). The second probability set may represent the probability corresponding to the result output by the text generation model for the second image description that does not meet the user's expectations, and based on the second probability set, a reverse description (which may be called an estimated reverse description) corresponding to the second image description that does not meet the user's expectations (i.e., what the user does not want to obtain) may be determined.

[0082] Afterwards, in one implementation, the target program may employ a cross entropy loss function to construct a description prediction loss based on the third output and the second forward description label, and the fourth output and the second reverse description label.

[0083] Specifically, the target program can determine the current probability corresponding to the second forward description label and the current probability of the second reverse description label based on the third output and the fourth output, respectively. In one case, if the second forward description label includes multiple words (all belonging to a preset word set), the current probability corresponding to the second forward description label can be determined by calculating (for example, calculating an average value or a multiplication result) the first probabilities of the words corresponding to the first positions in the second forward description label at each first position specified in the first probability set.

[0084] If the second reverse description tag includes multiple words (all belonging to a preset word set), the current probability corresponding to the second reverse description tag can be determined by calculating (for example, calculating the average value or multiplication result) the second probability of the words appearing at the corresponding positions in the second reverse description tag at each second position specified in the second probability set.

[0085] Then, based on the current probability corresponding to the second positive description label and the current probability of the second negative description label, a description prediction loss is constructed. In one case, the description prediction loss can be expressed by the following formula (1):

[0086]

[0087] Among them, L 1 Describes the prediction loss, x 1 Represents the second image description, y p1 Indicates the second positive description label corresponding to the second image description, y n1 represents the second reverse description label corresponding to the second image description, D represents the aforementioned training data set to which the second image description and its corresponding second forward description label and second reverse description label belong, p(y p ,y n |x 1 ) represents the description of x according to the second image 1 , through the text generation model to be trained, the obtained y p1 The current probability and get y n1 The sum or average of the current probabilities, It means to calculate logp(y p1 ,y n1 |x 1 )’s expected value.

[0088] In one implementation, the text generation model to be trained may be a transformer-based generative pre-trained model.

[0089] In one implementation, the text generation model to be trained may include at least one text encoder (referred to as the first text encoder for the sake of clarity) and two text decoders arranged in parallel. Accordingly, the target program encodes the second image description based on the second image description through the first text encoder of the text generation model to be trained to obtain the text encoding of the second image description, and then decodes the text encoding of the second image description through the two text decoders arranged in parallel of the text generation model to be trained to obtain the third output and the fourth output.

[0090] In another implementation, the target program may determine an estimated forward description corresponding to the third output based on the third output and the specified word determination rule, and determine an estimated reverse description corresponding to the fourth output based on the fourth output and the specified word determination rule; and determine the description prediction loss based on the difference between the estimated forward description and the second forward description label, and the difference between the estimated reverse description and the second reverse description label. The specified word determination rule may be any rule in the related art that can determine the corresponding description based on the output of the text generation model.

[0091] In another implementation, after the target program obtains the third output and the fourth output, a first loss can be constructed based on the difference between the first word probability set corresponding to each of the several positions in the third output and the label probability set corresponding to the second forward description label (for example, the Euclidean distance between the two can be constructed to represent the difference between the two), and a second loss can be constructed based on the difference between the second word probability set corresponding to each of the several positions in the fourth output and the label probability set corresponding to the second reverse description label (for example, the Euclidean distance between the two can be constructed to represent the difference between the two), and a description prediction loss can be determined based on the first loss and the second loss, wherein the description prediction loss is positively correlated with the first loss and the second loss, respectively.

[0092] Among them, the label probability set corresponding to the second positive description label may include the label probability of each word in the preset word set appearing in each position in a number of positions. In the label probability set corresponding to the second positive description label, the label probability corresponding to the word involved in each position of the second positive description label is 1, and the label probability corresponding to the word not involved in each position in the second positive description label is 0. For example, the word "X" is involved at position d of the second positive description label, and accordingly, the label probability corresponding to the word "X" at position d in the label probability set corresponding to the second positive description label is 1, and the label probability corresponding to the word other than the word "X" in the preset word set at position d is 0.

[0093] Similarly, the label probability set corresponding to the second reverse description label may include the label probability of each word in the preset word set appearing in each of several positions, wherein, in the label probability set corresponding to the second reverse description label, the label probability corresponding to the word involved in each position of the second reverse description label is 1, and the label probability corresponding to the word not involved in each position of the second reverse description label is 0. In one case, the label probability sets corresponding to the third output, the fourth output, the second forward description label and the second reverse description label involved in the present disclosure may exist in the form of a matrix.

[0094] It can be understood that the smaller the description prediction loss is, the stronger the image description generation expansion capability of the text generation model is. In view of this, in step S140, the parameters of the text generation model to be trained are adjusted with the goal of minimizing the description prediction loss until the text generation model to be trained reaches the preset convergence condition to obtain the text generation model. In this step, the parameters of the text generation model to be trained are adjusted with the goal of minimizing the description prediction loss, that is, with the goal of maximizing the current probability corresponding to the aforementioned second forward description label and the current probability of the second reverse description label, that is, with the goal of minimizing the difference between the aforementioned estimated forward description and the second forward description label and the difference between the estimated reverse description and the second reverse description label, so that the text generation model to be trained learns to obtain the distribution of the forward description corresponding to the image description in the training data set and the distribution of the reverse description corresponding to the image description, so that the pre-trained text generation model can generate a forward description that meets the user's expectations and a reverse description that does not meet the user's expectations.

[0095] Let's review the pre-training process of the aforementioned text generation model, i.e., steps S110-S140. The above embodiment is described by taking a group of image description groups (i.e., a group of second image descriptions and their corresponding second forward description labels and second reverse description labels) as an example. In another embodiment, the above pre-training process can also be performed on a batch of samples, i.e., multiple groups of image description groups, to obtain the third output and fourth output corresponding to the original image description in each group of image description groups, respectively, and then determine the description prediction loss based on the third output and fourth output corresponding to the original image description in each group of image description groups and the second forward description label and the second reverse description label; with the goal of minimizing the description prediction loss, adjust the parameters of the text generation model to be trained. In this embodiment, the description prediction loss is determined for a batch of samples, and then the parameters of the text generation model to be trained are adjusted. This can reduce the number of parameter adjustments for the text generation model to be trained, making it easier to implement the pre-training process.

[0096] In the above process, the text generation model to be trained is pre-trained through comparative learning, so that the text generation model has the initial ability to generate image descriptions, and can generate positive descriptions that meet user expectations and negative descriptions that do not meet user expectations, thereby aligning the text generation model with user preferences and making it easier to fine-tune (train) the parameters of the text generation model later.

[0097] Through the above method, the target image generation model and the pre-trained text generation model are obtained. Afterwards, based on the reinforcement learning method, the pre-trained text generation model is further trained (i.e., fine-tuned). The following is an introduction to the training process. Figure 2 As shown, the method includes the following steps S210-S260:

[0098] In step S210, the first image description is obtained. In this step, the target program can obtain the first image description from a pre-constructed sample data set for continuing to train the text generation model. The sample data set may include a plurality of image descriptions, and the first image description may be any image description in the sample data set. In one implementation, the plurality of image descriptions in the sample data set may be extracted from a specified public data set containing a large number of image descriptions. It is understandable that the sample data set and the aforementioned training data set may include the same image descriptions or different image descriptions.

[0099] After the target program obtains the first image description, in step S220, based on the first image description, a first generated image is determined by a target image generation model. The target image generation model is the aforementioned pre-trained target image generation model. In this step, the target program can input the first image description into the target image generation model, and use the target image generation model to process the first image description to determine a generated image corresponding to the first image description, which is called a first generated image.

[0100] Next, in step S230, based on the first image description, a corresponding predicted positive description and predicted negative description are obtained through a text generation model, wherein the text generation model is pre-trained based on the second image description and its corresponding positive description label and negative description label.

[0101] In this step, the target program may input the first image description into the text generation model, so as to use the text generation model to process the first image description and obtain the first output and the second output of the text generation model. The first output may include a third word probability set corresponding to each of the plurality of positions; the third word probability set corresponding to each of the plurality of positions may include the third probability of each word in the aforementioned preset word set appearing at the position determined by the text generation model based on the first image description; the second output may include a fourth word probability set corresponding to each of the plurality of positions; the fourth word probability set corresponding to each of the plurality of positions may include the fourth probability of each word in the aforementioned preset word set appearing at the position determined by the text generation model based on the first image description. The target program may determine a predicted forward description based on the first output and the specified word determination rule, and may determine a predicted reverse description based on the second output and the specified word determination rule.

[0102] In one implementation, the target program can encode the first image description through the first text encoder of the text generation model to obtain the text encoding corresponding to the first image description, and decode the text encoding corresponding to the first image description through two text decoders set in parallel of the text generation model to obtain the first output and the second output. Then, the specified word determination rule is used to determine the predicted forward description and the predicted reverse description corresponding to the first image description based on the first output and the second output.

[0103] Then, in step S240, a second generated image is determined through a target image generation model according to the predicted forward description and the predicted reverse description.

[0104] Specifically, in one embodiment, step S240 may include the following steps 11-12:

[0105] In step 11, based on the predicted forward description and the predicted reverse description, the target image generation model is used to obtain the image features corresponding to the predicted forward description and the predicted reverse description. In this step, the target program can input the predicted forward description and the predicted reverse description into the target image generation model, and use the text encoder of the target image generation model (referred to as the second text encoder for clarity) and the denoising network of the target image generation model to sequentially process the predicted forward description and the predicted reverse description to obtain the image features corresponding to the predicted forward description and the image features corresponding to the predicted reverse description.

[0106] Then, in step 12, the second generated image is determined by using the image features corresponding to the predicted forward description and the image features corresponding to the predicted reverse description. In one implementation, the target program can subtract the image features corresponding to the predicted forward description from the image features corresponding to the predicted reverse description to obtain the corresponding difference features. It can be understood that the predicted forward description is a description that meets the user's expectations, and its corresponding image features contain more features of images that meet the user's expectations and are considered to have good quality; the predicted reverse description is a description that does not meet the user's expectations, and its corresponding image features contain more features of images that do not meet the user's expectations and are considered to have poor quality. By subtracting the image features corresponding to the predicted reverse description from the image features corresponding to the predicted forward description, it can be considered that the features of images that do not meet the user's expectations and have good quality (i.e., the image features corresponding to the predicted reverse description) are subtracted from the features of images that meet the user's expectations and have good quality (i.e., the image features corresponding to the predicted forward description), thereby obtaining features of images that have better quality and are more in line with the user's expectations, i.e., the difference features. The difference feature is then input into the image decoder of the target image generation model, and the difference feature is decoded by the image decoder to obtain a second generated image, which is an image that better meets user expectations and has good quality. Accordingly, the second quality score of the second generated image obtained subsequently is a score obtained by combining the features of the generated image corresponding to the predicted forward description and the features of the generated image corresponding to the predicted reverse description.

[0107] After the target program obtains the first generated image and the second generated image, in step S250, a first quality score for the first generated image and a second quality score for the second generated image are obtained. In this step, the target program may obtain the first quality score for the first generated image and the second quality score for the second generated image based on the reward model.

[0108] It is understandable that the quality score of the generated image can reflect the quality of the corresponding generated image. In one case, the quality of the generated image can be measured from multiple perspectives, for example, the perspectives for measuring the quality of the generated image can include but are not limited to: image clarity (which can be represented by the clarity score of the image, for example, the higher the clarity score of the image, the higher the image clarity of the image), the aesthetic score of the image, and the correlation between the image and its corresponding image description (that is, the matching degree between the image and its corresponding image description).

[0109] Accordingly, in one embodiment, the first quality score may include at least one of the following: a first aesthetic score of the first generated image (which may represent the aesthetics of the image), a first clarity score of the first generated image, and a first correlation value between the first generated image and the first image description (which may represent the correlation between the image and the corresponding image description); and the second quality score may include at least one of the following: a second aesthetic score of the second generated image, a second clarity score of the second generated image, and a second correlation value between the second generated image and the first image description. Accordingly, the aforementioned reward model may include at least one of the following models: an aesthetic prediction model F for determining the aesthetic score of an image B , a preset gradient algorithm F for determining the clarity score of an image T and an image-text relevance model F for determining a relevance value between an image and its corresponding image description G .

[0110] In one implementation, the first aesthetic score of the first generated image may be based on the aesthetic prediction model F B Specifically, the target program may input the first generated image into the aesthetic prediction model F B , the first generated image is processed by the aesthetic prediction model to obtain a first aesthetic score of the first generated image, and then the first aesthetic score of the first generated image is included in the first quality score of the first generated image. Similarly, the second aesthetic score of the second generated image can be, based on the aesthetic prediction model F B Specifically, the target program may input the second generated image into the aesthetic prediction model F B , processing the second generated image through the aesthetic prediction model to obtain a second aesthetic score of the second generated image, and then incorporating the second aesthetic score of the second generated image into a second quality score of the second generated image. In one case, the aesthetic score can take a value between [0, 1].

[0111] The aesthetic prediction model F B It can be any model in the related art that can evaluate the aesthetics of an image, i.e., the aesthetic score. In one implementation, the aesthetic prediction model can be a neural network model pre-trained based on images and their corresponding aesthetic score labels, and the aesthetic prediction model can output the aesthetic score of the image for an input image. The images and their corresponding aesthetic score labels used to train the aesthetic prediction model can be selected from any data set that includes a large number of images and their corresponding aesthetic score labels that characterize the aesthetics of the images.

[0112] In one case, the higher the first aesthetic score (second aesthetic score) of the first generated image (second generated image), the more beautiful the first generated image (second generated image) is, and the better the image quality is.

[0113] In one implementation, the first definition score of the first generated image may be based on a preset gradient algorithm F T Calculated for the first generated image. In one case, the preset gradient algorithm may be a Tenengrad gradient algorithm. Specifically, the target program may use the Sobel operator to calculate the horizontal gradient (which may be called the horizontal gradient) and the vertical gradient (which may be called the vertical gradient) of each pixel point (referred to as the first pixel point) of the first generated image, and then use the horizontal gradient and the vertical gradient of each first pixel point to determine the total gradient of each first pixel point; then calculate the average value of the sum of the total gradients of each first pixel point as the first gradient of the first generated image. Then use the first gradient of the first generated image to determine the first clarity score of the first generated image. For example, the first gradient of the first generated image may be directly determined as the first clarity score of the first generated image; or the first clarity score of the first generated image may be determined by using the first gradient of the first generated image and the corresponding relationship between the preset gradient and the clarity score; then the first clarity score of the first generated image may be included in the first quality score of the first generated image. Similarly, the second clarity score of the second generated image may be determined based on the preset gradient algorithm, and then it may be included in the second quality score of the second generated image.

[0114] The larger the gradient of the first generated image (or the second generated image), the higher its clarity, and the larger its clarity score, that is, the better the image quality. In one case, the clarity score can take a value between [0, 1].

[0115] In one case, the total gradient of each first pixel point (for example, the jth first pixel point) can be determined by the following formula (2):

[0116]

[0117] Among them, T j represents the total gradient of the jth first pixel, t j1 represents the horizontal gradient of the jth first pixel, t j2 Represents the vertical gradient of the j-th first pixel.

[0118] In one implementation, the correlation value between the first generated image and the first image description may be based on the image-text correlation model F GDetermined for the first generated image and the first image description. The image-text correlation model is a model pre-trained based on a number of sample image-text pairs (each sample image-text pair includes an image and an image description) and their corresponding labels (including sample correlation values ​​representing the correlation of the corresponding sample image-text pairs).

[0119] The target program may input the first generated image and the first image description into the image-text correlation model, so as to use the image-text correlation model to process the first generated image and the first image description, determine a first correlation value therebetween, and then include it in the first quality score for the first generated image. Similarly, the target program inputs the second generated image and the first image description into the image-text correlation model, so as to use the image-text correlation model to process the second generated image and the first image description, determine a second correlation value therebetween, and then include it in the second quality score for the second generated image.

[0120] The larger the correlation value between the first generated image and the first image description (the second generated image and the first image description), the better the quality of the first generated image and the first image description (the second generated image and the first image description). In one case, the correlation value can take a value between [0, 1].

[0121] After the target program obtains the first quality score for the first generated image and the second quality score for the second generated image, in step S260 , the parameters of the text generation model are adjusted based on a comparison of the first quality score and the second quality score.

[0122] In one implementation, the target program can adjust the parameters of the text generation model based on a comparison of the first quality score and the second quality score, i.e., a calculated difference between the second quality score and the first quality score, such as a differential value (i.e., a subsequent image optimization reward parameter), as the current environmental reward, i.e., a reward score.

[0123] Wherein, in the case where the first quality score includes a first aesthetic score of the first generated image, a first clarity score of the first generated image, and a first correlation value between the first generated image and the first image description, and the second quality score includes a second aesthetic score of the second generated image, a second clarity score of the second generated image, and a second correlation value between the second generated image and the first image description, the difference between the second quality score and the first quality score can be expressed as: R=[F B (I P )-F B (I O )]+[F T (I P )-F T (I O)]+[F G (I P , x 2 )-F G (I O , x 2 )](3), where R represents the difference between the second quality score and the first quality score, and F B (I P ) represents the second aesthetic score of the second generated image, F T (I P ) represents the second clarity score of the second generated image, F G (I P , x 2 ) represents the second generated image and the first image description x 2 The second correlation value between B (I O ) represents the first aesthetic score of the first generated image, F T (I O ) represents the first clarity score of the first generated image, F G (I O , x 2 ) represents the first correlation value between the first generated image and the first image description, I P represents the second generated image, I O represents the first generated image, x 2 Represents the first image description.

[0124] Among them, the second correlation value between the second generated image and the first image description is calculated, and the difference between the second correlation value and the first correlation value between the first generated image and the first image description is used to construct the current environmental reward, so that the semantics of the predicted forward description and the predicted reverse description obtained after the first image description is expanded by the text generation model do not deviate from the semantics of the first image description. For example: the first image description indicates the generation of two cats, and it is necessary to ensure that the semantics of the predicted forward description and the predicted reverse description obtained after the expansion of the text generation model are still instructions to generate two cats, rather than instructions to generate one cat. In the case where the semantics of the predicted forward description and the predicted reverse description deviate from the generation of two cats, the reward model needs to give a penalty to the predicted forward description and the predicted reverse description, that is, the determined correlation value between the second generated image and the first image description will be relatively small.

[0125] It can be understood that the greater the difference between the second quality score and the first quality score (i.e., the current environmental reward), the better the quality of the predicted forward description and the predicted reverse description generated by the text generation model, and accordingly, the better the description generation ability of the text generation model.

[0126] In view of this, the aforementioned process of adjusting the parameters of the text generation model based on the difference (i.e., the difference) between the second quality score and the first quality score may include: constructing a first prediction loss based on the difference, wherein the first prediction loss is negatively correlated with the difference; and adjusting the parameters of the text generation model with the goal of minimizing the first prediction loss. The process of adjusting the parameters of the text generation model may specifically include: determining the adjustment gradient of the parameters of the text generation model based on the first prediction loss using a back propagation algorithm; determining the updated value of the parameters of the text generation model using the determined adjustment gradient and the current value of the parameters of the text generation model; and adjusting the parameters of the text generation model based on the updated value of the parameters determined above. The adjustment gradient of the parameters of the text generation model is determined with the goal of minimizing the first prediction loss. Minimize the first prediction loss to improve the ability of the text generation model to generate high-quality image descriptions.

[0127] In one embodiment, Figure 3 As shown, the method may further include the following steps S310-S380:

[0128] In step S310, a first image description is obtained. The implementation of step S310 can be found in Figure 2 The implementation method of step S210 shown is not described in detail here.

[0129] In step S320, based on the first image description, a first generated image is determined by using a target image generation model. The implementation of step S320 can refer to the following. Figure 2 The implementation method of step S220 shown is not described in detail here.

[0130] In step S330, based on the first image description, a corresponding predicted positive description and predicted negative description are obtained through a text generation model. The text generation model is pre-trained based on the second image description and its corresponding positive description and negative description. The implementation of step S330 can be found in FIG. Figure 2 The implementation method of step S230 shown is not described in detail here.

[0131] In step S340, a second generated image is determined by generating a target image model according to the predicted forward description and the predicted reverse description. Figure 2 The implementation method of step S240 shown is not described in detail here.

[0132] In step S350, a first quality score for the first generated image and a second quality score for the second generated image are obtained. The implementation of step S350 can be seen in FIG. Figure 2The implementation method of step S250 shown is not described in detail here.

[0133] In step S360, a first forward description label and a first reverse description label corresponding to the first image description are obtained. In one implementation, the aforementioned sample data set may also include forward description labels and reverse description labels corresponding to each image description, and the forward description labels and reverse description labels corresponding to each image description may be generated according to each image description using the aforementioned specified generation model. The specific generation process may refer to the generation process of the forward description and reverse description corresponding to the aforementioned image description i, which will not be described in detail here.

[0134] In this step, the target program can obtain the forward description (referred to as the first forward description label) and the reverse description (referred to as the first reverse description label) corresponding to the aforementioned first image description from the sample data set. Then, in step S370, based on the comparison of the first quality score and the second quality score, and the first output related to the predicted forward description and the second output related to the predicted reverse description obtained by the text generation model, the first forward description label and the first reverse description label, the target loss is determined.

[0135] Specifically, in one embodiment, step S370 may include the following steps 21-24:

[0136] In step 21, an image optimization reward parameter is determined based on the difference between the second quality score and the first quality score. In this step, the target program may determine the difference R between the second quality score and the first quality score determined above as the image optimization reward parameter.

[0137] Next, at step 22, a model prediction loss is determined based on the first output and the first forward description label, and the second output and the first reverse description label.

[0138] As described above, the first output may include a third word probability set corresponding to each of the plurality of positions, and the third word probability set corresponding to each position may include the third probability of each word in the aforementioned preset word set appearing at the position determined by the text generation model based on the first image description. The second output may include a fourth word probability set corresponding to each of the plurality of positions, and the fourth word probability set corresponding to each position may include the fourth probability of each word in the aforementioned preset word set appearing at the position determined by the text generation model based on the first image description.

[0139] In one implementation, the target program can determine the predicted probability corresponding to the first positive description label based on the third probability of the words involved in the first positive description label appearing at the positions corresponding to the positions of the first positive description label in the first output. Specifically, the process of determining the predicted probability corresponding to the first positive description label can refer to the aforementioned process of determining the current probability corresponding to the second positive description label, which will not be repeated here.

[0140] Next, the target program may determine the predicted probability corresponding to the first reverse description label based on the fourth probability that the words involved in the first reverse description label appear at positions corresponding to the positions of the first forward description label in the second output.

[0141] Afterwards, the target program can adopt a preset loss function (such as a cross entropy loss function) to determine the model prediction loss based on the prediction probability corresponding to the first forward description label and the prediction probability corresponding to the first reverse description label, wherein the larger the prediction probability corresponding to the first forward description label and the prediction probability corresponding to the first reverse description label, the better the image description generation and expansion capability of the text generation model, and accordingly, the model prediction loss is negatively correlated with the prediction probability corresponding to the positive description label and the prediction probability corresponding to the reverse description label, respectively.

[0142] In another implementation, the target program may also determine the model prediction loss based on the difference (e.g., Euclidean distance) between the predicted forward description (the description determined based on the first output) and the first forward description label, and the difference (e.g., Euclidean distance) between the predicted reverse description (the description determined based on the second output) and the first reverse description label, wherein the model prediction loss is positively correlated with the difference between the predicted forward description and the first forward description label, and the difference between the predicted reverse description and the first reverse description label, respectively.

[0143] Next, in step 23, the target loss is determined based on the product of the image optimization reward parameter and the model prediction loss. In this step, the target loss can be expressed by the following formula (4):

[0144]

[0145] Among them, L represents the target loss, x 2 represents the first image description, y p2 Indicates the first positive description label corresponding to the first image description, y n2 represents the first reverse description label corresponding to the first image description, Q represents the aforementioned sample data set to which the first image description and its corresponding first forward description label and first reverse description label belong, p(y p2 ,y n2 |x 2 ) represents the first image description x2 Through the text generation model, we get y p2 The predicted probability and y n2 The sum or average of the predicted probabilities, logp(y p2 ,y n2 |x 2 ) represents the model prediction loss, Represents the expected value of the product of the reward parameter for computing image optimization and the model prediction loss.

[0146] After the target loss is obtained in the above manner, in step S380, the parameters of the text generation model are adjusted with the goal of minimizing the target loss. In this step, the target program can determine the adjustment gradient of the parameters of the text generation model based on the target loss using the back propagation algorithm, and determine the updated value of the parameters of the text generation model using the determined adjustment gradient and the current value of the parameters of the text generation model. Then, based on the updated value of the parameter, the parameters of the text generation model are adjusted. Among them, the adjustment gradient of the parameters of the text generation model is determined with the goal of minimizing the target loss. Minimize the target loss to improve the ability of the text generation model to generate high-quality image descriptions, so that the text generation model has the ability to output a forward description with better quality that meets the user's expectations and a reverse description with better quality that does not meet the user's expectations.

[0147] Reviewing the execution process of the aforementioned steps S310-S380, the above embodiment is described by taking a first image description and its corresponding first positive description label and first reverse description label (hereinafter referred to as a description group) as an example. In another embodiment, the above steps S310-S380 can also be performed for a batch of samples, i.e., multiple groups of description groups, to obtain the first quality score of the first generated image, the second quality score of the second generated image, and the first output and the second output of the text generation model corresponding to each group of description groups, and then in step S370, based on the comparison of the first quality score and the second quality score corresponding to each group of description groups, and the first output and the second output of the text generation model, the first positive description label and the first reverse description label therein, determine the target loss, and adjust the parameters of the text generation model with the goal of minimizing the target loss. In this embodiment, the target loss is determined for a batch of samples, and then the parameters of the text generation model are adjusted with the goal of minimizing the target loss, so that the number of parameter adjustments of the text generation model can be reduced, and the implementation of the training process is easier.

[0148] The parameters of the text generation model are adjusted until the text generation model reaches a specified convergence condition, thereby obtaining a trained text generation model. The specified convergence condition may include but is not limited to the number of parameter adjustments of the text generation model exceeding a preset number threshold, or the target loss is lower than a preset loss threshold, or the training time reaches a preset time.

[0149] In this embodiment, the reinforcement learning concept is adopted, and based on the comparison of the first quality score and the second quality score, that is, based on the comparison of the qualities of the first generated image corresponding to the first image description and the second generated image corresponding to the image description obtained by expansion of the text generation model (predicted forward description and predicted reverse description), the current environmental feedback is obtained, and then the parameters of the text generation model are adjusted using the environmental feedback to achieve better improvement of the image description generation expansion capability of the text generation model based on the reinforcement learning method and combined with the environmental feedback (which can reflect the current image description generation expansion capability of the text generation model), so that the text generation model can output higher quality image descriptions that are more in line with user expectations and image descriptions that do not meet user expectations according to the input image description, thereby improving the quality of the image generated based on the image description, so that the image is more in line with user expectations.

[0150] Furthermore, by comparing the first quality score with the second quality score as current environmental feedback, the trained text generation model can be more aligned with user preferences (i.e., the output positive description is more in line with user expectations, and the output negative description is less in line with user expectations), and the trained text generation model can be more aligned with the target image generation model.

[0151] Corresponding to the above method embodiment, the present disclosure embodiment further provides an image generation method, which can be applied to a first program, and the first program can be the same physical program as the aforementioned target program, or can be a different physical program from the aforementioned target program. Figure 4 As shown, the method may include the following steps S410-S430:

[0152] In step S410, the image description to be processed is obtained. In one implementation, the image description to be processed may be any image description input by the user when there is a need to generate an image using the image description (for example, when generating corresponding illustrations for novels, news, etc.), and accordingly, the target program may obtain the image description input by the user as the image description to be processed.

[0153] Then in step S420, based on the description of the image to be processed, the target forward description and the target reverse description are obtained through the text generation model. In this step, the process of obtaining the target forward description and the target reverse description corresponding to the description of the image to be processed can refer to the aforementioned process of obtaining the predicted forward description and the predicted reverse description corresponding to the first image description, which will not be repeated here. In one implementation, the text generation model is a text generation model trained based on the aforementioned text generation model.

[0154] Then, in step S430, based on the target forward description and the target reverse description, the target image is generated through the image generation model. Among them, the image generation model can be the target image generation model in the aforementioned embodiment. In this step, the first program can generate a target image with relatively better quality through the image generation model based on the target forward description and the target reverse description. Specifically, in one implementation, in step S430, the following steps 31-32 may be included: in step 31, based on the target forward description and the target reverse description, the image features corresponding to the target forward description and the target reverse description are obtained through the image generation model; in step 32, the target image is determined using the image features corresponding to the target forward description and the image features corresponding to the target reverse description. To obtain a target image with better quality. Among them, the process of determining the image features corresponding to the target forward description and the target reverse description, and the process of determining the target image, can refer to the aforementioned process of determining the image features corresponding to the predicted forward description and the predicted reverse description, and the process of determining the second generated image, which will not be repeated here.

[0155] In the above process, based on the description of the image to be processed, the text generation model can be used to obtain a target positive description that meets the user's expectations and a target reverse description that does not meet the user's expectations. Then, based on the target positive description and the target reverse description, the image generation model can be used to generate a target image with relatively better quality to improve the user experience.

[0156] In the disclosed embodiment, after obtaining the target forward description and the target reverse description through the text generation model based on the description of the image to be processed, the corresponding images can also be obtained through the image generation model based on the target forward description or the target reverse description.

[0157] The foregoing describes specific embodiments of the present disclosure, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims may be performed in an order different from that in the embodiments, and the desired results may still be achieved. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible, or may be advantageous.

[0158] Corresponding to the above method embodiment, the present disclosure embodiment provides a training device 500 for a text generation model, and its schematic block diagram is as follows: Figure 5 As shown, it is used to execute Figure 2 or Figure 3 The method shown, the device 500 includes:

[0159] A first acquisition module 510, configured to acquire a first image description;

[0160] A first determination module 520 is configured to determine a first generated image based on the first image description by using a target image generation model;

[0161] A first obtaining module 530 is configured to obtain a corresponding predicted positive description and a predicted reverse description based on the first image description through a text generation model;

[0162] A second determination module 540 is configured to determine a second generated image through the target image generation model according to the predicted forward description and the predicted reverse description;

[0163] A second acquisition module 550, configured to acquire a first quality score for the first generated image and a second quality score for the second generated image;

[0164] The first adjustment module 560 is configured to adjust the parameters of the text generation model based on the comparison between the first quality score and the second quality score.

[0165] In an optional implementation, it also includes:

[0166] A third acquisition module (not shown in the figure), configured to acquire a first positive description tag and a first negative description tag corresponding to the first image description;

[0167] The first adjustment module 560 includes:

[0168] a first determining unit (not shown in the figure), configured to determine a target loss based on a comparison of the first quality score and the second quality score, a first output related to the predicted forward description and a second output related to the predicted reverse description obtained by the text generation model, the first forward description label, and the first reverse description label;

[0169] An adjustment unit (not shown in the figure) is configured to adjust the parameters of the text generation model with the goal of minimizing the target loss.

[0170] In an optional implementation, the first determining unit is specifically configured to determine an image optimization reward parameter based on a difference between the second quality score and the first quality score;

[0171] Determining a model prediction loss based on the first output and the first forward description label, and the second output and the first reverse description label;

[0172] The target loss is determined based on the product of the image optimization reward parameter and the model prediction loss.

[0173] In an optional implementation, the second determination module 540 is specifically configured to obtain image features corresponding to the predicted forward description and the predicted reverse description respectively through the target image generation model based on the predicted forward description and the predicted reverse description;

[0174] The second generated image is determined using the image features corresponding to the predicted forward description and the image features corresponding to the predicted reverse description.

[0175] In an optional embodiment, the first quality score includes at least one of the following: the first quality score includes at least one of the following: a first aesthetic score of the first generated image, a first clarity score of the first generated image, and a first correlation value between the first generated image and the first image description;

[0176] The second quality score comprises at least one of: a second aesthetic score of the second generated image, a second sharpness score of the second generated image, and a second relevance value between the second generated image and the first image description.

[0177] In an optional implementation, the first aesthetic score of the first generated image is determined for the first generated image based on an aesthetic prediction model;

[0178] A first clarity score of the first generated image, calculated for the first generated image based on a preset gradient algorithm;

[0179] The first correlation value between the first generated image and the first image description is determined based on a picture-text correlation model for the first generated image and the first image description.

[0180] In an optional implementation, the method further includes: a third acquisition module (not shown in the figure), configured to acquire the second image description and its corresponding second forward description label and second reverse description label before obtaining the corresponding predicted forward description and predicted reverse description through the text generation model based on the first image description;

[0181] A second processing module (not shown in the figure) is configured to obtain a third output and a fourth output of the text generation model to be trained based on the second image description through the text generation model to be trained;

[0182] a construction module (not shown in the figure), configured to construct a description prediction loss based on the third output and the second forward description label, and the fourth output and the second reverse description label;

[0183] The second adjustment module is configured to adjust the parameters of the text generation model to be trained with the goal of minimizing the description prediction loss until the text generation model to be trained reaches a preset convergence condition, thereby obtaining the text generation model.

[0184] Corresponding to the above method embodiment, the present disclosure embodiment provides an image generating device 600, whose schematic block diagram is as follows: Figure 6 As shown, it is used to execute Figure 4 The method shown, the device 600 includes:

[0185] A third acquisition module 610 is configured to acquire a description of an image to be processed;

[0186] A second obtaining module 620 is configured to obtain a target forward description and a target reverse description based on the description of the image to be processed by using a text generation model;

[0187] The generation module 630 is configured to generate a target image based on the target forward description and the target reverse description through an image generation model.

[0188] In an optional implementation, the text generation model is trained using any of the aforementioned text generation model training devices.

[0189] In an optional implementation, the generation module 630 is specifically configured to obtain image features corresponding to the target forward description and the target reverse description respectively through the image generation model based on the target forward description and the target reverse description; and determine the target image using the image features corresponding to the target forward description and the image features corresponding to the target reverse description, so as to obtain a target image with better quality.

[0190] The above device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.

[0191] The present disclosure provides an electronic device, including a memory and a processor, wherein the memory stores an executable code, and when the processor executes the executable code, the following is implemented: Figure 2 or Figure 3 or Figure 4 The method shown below. Figure 7 , which shows a schematic structural diagram of an electronic device suitable for implementing an embodiment of the present disclosure. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0192] like Figure 7 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In RAM 703, various programs and data required for the operation of the electronic device are also stored. The processing device 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0193] Typically, the following devices may be connected to the I / O interface 705: input devices 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, etc.; output devices 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 708 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 709. The communication devices 709 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Figure 7 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 7 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0194] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0195] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute the following steps provided in the present disclosure: Figure 2 or Figure 3 or Figure 4 The method shown.

[0196] It should be noted that the computer-readable medium described in the embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0197] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains a first image description; determines a first generated image based on the first image description through a target image generation model; obtains a corresponding predicted forward description and predicted reverse description based on the first image description through a text generation model; determines a second generated image based on the predicted forward description and the predicted reverse description through the target image generation model; obtains a first quality score for the first generated image and a second quality score for the second generated image; and adjusts the parameters of the text generation model based on a comparison of the first quality score and the second quality score. Alternatively, the electronic device: obtains a description of an image to be processed; obtains a target forward description and a target reverse description based on the image to be processed through a text generation model; generates a target image based on the target forward description and the target reverse description through an image generation model.

[0198] Computer program code for performing the operations of the disclosed embodiments may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0199] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the storage medium and computing device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0200] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present invention may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0201] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the embodiments of the present invention in detail. It should be understood that the above description is only a specific implementation method of the embodiments of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A training method for a text generation model, comprising: Obtaining a first image description; Based on the first image description, determining a first generated image through a target image generation model; Based on the first image description, obtaining a corresponding predicted positive description and a predicted negative description through a text generation model; Determining a second generated image through the target image generation model according to the predicted forward description and the predicted reverse description; Obtaining a first quality score for the first generated image and a second quality score for the second generated image; Based on the comparison of the first quality score and the second quality score, a parameter of the text generation model is adjusted.

2. The method of claim 1, further comprising: Obtaining a first forward description tag and a first reverse description tag corresponding to the first image description; The adjusting the parameters of the text generation model includes: Determining a target loss based on a comparison of the first quality score and the second quality score, a first output associated with the predicted forward description and a second output associated with the predicted reverse description obtained by the text generation model, the first forward description label, and the first reverse description label; The parameters of the text generation model are adjusted with the goal of minimizing the target loss.

3. The method of claim 2, wherein: The determining of the target loss comprises: determining an image optimization reward parameter based on a difference between the second quality score and the first quality score; Determining a model prediction loss based on the first output and the first forward description label, and the second output and the first reverse description label; The target loss is determined based on the product of the image optimization reward parameter and the model prediction loss.

4. The method of claim 1, wherein: The determining of the second generated image comprises: Based on the predicted forward description and the predicted reverse description, respectively, the image features corresponding to the predicted forward description and the predicted reverse description are obtained through the target image generation model; The second generated image is determined using the image features corresponding to the predicted forward description and the image features corresponding to the predicted reverse description.

5. The method of claim 1, wherein: The first quality score comprises at least one of: a first aesthetic score of the first generated image, a first sharpness score of the first generated image, and a first correlation value between the first generated image and the first image description; The second quality score comprises at least one of: a second aesthetic score of the second generated image, a second sharpness score of the second generated image, and a second relevance value between the second generated image and the first image description.

6. The method of claim 5, wherein: a first aesthetic score of the first generated image, determined for the first generated image based on an aesthetic prediction model; A first clarity score of the first generated image, calculated for the first generated image based on a preset gradient algorithm; The first correlation value between the first generated image and the first image description is determined based on a picture-text correlation model for the first generated image and the first image description.

7. The method according to any one of claims 1 to 6, before obtaining the corresponding predicted positive description and predicted negative description by using a text generation model based on the first image description, further comprising: Obtaining the second image description and its corresponding second forward description tag and second reverse description tag; Based on the second image description, obtaining a third output and a fourth output of the text generation model to be trained through the text generation model to be trained; constructing a description prediction loss based on the third output and the second forward description label, and the fourth output and the second reverse description label; With the goal of minimizing the description prediction loss, the parameters of the text generation model to be trained are adjusted until the text generation model to be trained reaches a preset convergence condition, thereby obtaining the text generation model.

8. A method for generating an image, comprising: Get the description of the image to be processed; Based on the description of the image to be processed, a target forward description and a target reverse description are obtained through a text generation model; Based on the target forward description and the target reverse description, a target image is generated through an image generation model.

9. The method of claim 8, wherein: The text generation model is trained using the method described in any one of claims 1 to 7.

10. The method of claim 8, wherein: The generating of the target image comprises: Based on the target forward description and the target reverse description, respectively, the image features corresponding to the target forward description and the target reverse description are obtained through the image generation model; The target image is determined by using the image features corresponding to the target forward description and the image features corresponding to the target reverse description.

11. A training device for a text generation model, comprising: A first acquisition module, configured to acquire a first image description; A first determination module is configured to determine a first generated image based on the first image description by using a target image generation model; A first obtaining module is configured to obtain a corresponding predicted positive description and a predicted reverse description based on the first image description through a text generation model; A second determination module is configured to determine a second generated image through the target image generation model according to the predicted forward description and the predicted reverse description; a second acquisition module, configured to acquire a first quality score for the first generated image and a second quality score for the second generated image; The first adjustment module is configured to adjust the parameters of the text generation model based on a comparison of the first quality score and the second quality score.

12. An image generating device, comprising: A third acquisition module is configured to acquire a description of the image to be processed; A second obtaining module is configured to obtain a target forward description and a target reverse description through a text generation model based on the description of the image to be processed; The generation module is configured to generate a target image through an image generation model based on the target forward description and the target reverse description.

13. An electronic device comprising a memory and a processor, wherein: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 10 is implemented.

14. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 10.

Images