Training method and device of image generation model and computer equipment

Abstract

The invention relates to an image generation model training method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: inputting a forward guide text and a reverse guide text into a language encoder to obtain text features respectively corresponding to the forward guide text and the reverse guide text; inputting an image hidden variable corresponding to the image training sample and a text feature corresponding to the forward guidance text into a hidden variable encoder to obtain an image hidden variable after semantic editing; inputting the semantically edited image hidden variables and text features corresponding to the reverse guide text into a hidden variable decoder to obtain semantically restored image hidden variables; and training a hidden variable encoder and a hidden variable decoder in the image generation model according to loss information determined by the image hidden variable corresponding to the image training sample and the image hidden variable after semantic reduction. By adopting the method, the specific attribute editing efficiency of the image can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a training method, apparatus, computer equipment, storage medium and computer program product for an image generation model. Background Art

[0002] Image generation technology is widely used in various industries, such as fashion design, art creation, digital media, virtual reality and other fields.

[0003] At present, when editing certain specific attributes of the original image, such as modifying and editing specific attributes of parts of the human body or objects, it is often implemented based on generative models, such as generative adversarial networks and diffusion models. However, the network structures of generative adversarial networks and diffusion models are relatively complex, and there are problems such as difficulty in convergence and low generation efficiency during the model training process.

[0004] Therefore, there is a problem in that the conventional technology is inefficient in editing specific attributes of an image. Summary of the invention

[0005] Based on this, it is necessary to provide a training method, device, computer equipment, computer-readable storage medium and computer program product for an image generation model that can improve the efficiency of editing specific attributes of images in response to the above-mentioned technical problems.

[0006] A training method for an image generation model, comprising:

[0007] Obtain an image training sample set, and determine a forward guiding text and a reverse guiding text for the image training sample set; the forward guiding text is used to guide the image training sample set to perform image transformation according to the target image attribute characteristics; the reverse guiding text describes the original image attribute characteristics of the image training sample set;

[0008] Use any image training sample in the image training sample set to perform the following training process to train the image generation model:

[0009] Inputting the forward guided text and the reverse guided text into the language encoder of the image generation model to obtain text features corresponding to the forward guided text and the reverse guided text respectively;

[0010] Obtain image latent variables corresponding to image training samples, and input the image latent variables and text features corresponding to the forward-guided text into the latent variable encoder of the image generation model to obtain semantically edited image latent variables;

[0011] Input the semantically edited image latent variables and the text features corresponding to the reverse guided text into the latent variable decoder of the image generation model to obtain the semantically restored image latent variables;

[0012] According to the loss information determined by the image latent variables corresponding to the image training samples and the image latent variables after semantic restoration, the latent variable encoder and latent variable decoder in the image generation model are trained.

[0013] In one embodiment, the image generation model further includes a visual encoder, which obtains image latent variables corresponding to the image training samples, including:

[0014] The image training samples are input into the visual encoder in the image generation model, and the image training samples are encoded by the visual encoder to obtain the encoding features corresponding to the image training samples;

[0015] The encoded features are input into the codebook to discretize the encoded features and obtain the image latent variables corresponding to the image training samples.

[0016] In one embodiment, the image generation model further includes a visual decoder, the visual encoder and the visual decoder are pre-trained, and the training process of the visual encoder and the visual decoder is:

[0017] Input the image training samples into the visual encoder to be trained, encode the image training samples through the visual encoder to be trained, and obtain the encoding features corresponding to the image training samples;

[0018] Inputting the coding features corresponding to the image training samples into the codebook to discretize the coding features corresponding to the image training samples to obtain the image latent variables corresponding to the image training samples;

[0019] The image latent variables corresponding to the image training samples are input into the visual decoder to be trained, and the image latent variables corresponding to the image training samples are decoded by the visual decoder to be trained to obtain the restored images corresponding to the image training samples;

[0020] According to the difference between the image training sample and the restored image, the visual encoder to be trained and the visual decoder to be trained are trained.

[0021] In one embodiment, the image latent variables and text features corresponding to the forward-guided text are input into the latent variable encoder of the image generation model to obtain the semantically edited image latent variables, including:

[0022] Inputting the text features corresponding to the image latent variables and the positive guidance text into the latent variable encoder of the image generation model, and fusing the text features corresponding to the image latent variables and the positive guidance text through the cross attention mechanism of the latent variable encoder to obtain a first fusion result;

[0023] The first fusion result is used as the semantically edited image latent variable.

[0024] In one embodiment, the semantically edited image latent variables and text features corresponding to the reverse guided text are input into the latent variable decoder of the image generation model to obtain the semantically restored image latent variables, including:

[0025] Inputting the text features corresponding to the semantically edited image latent variables and the reverse guided text into the latent variable decoder of the image generation model, and fusing the text features of the semantically edited image latent variables and the reverse guided text through the cross attention mechanism in the latent variable decoder to obtain a second fusion result;

[0026] The second fusion result is used as the image latent variable after semantic restoration.

[0027] A method for generating an image, comprising:

[0028] Acquire an image to be processed, and acquire a positive guidance text for the image to be processed; the positive guidance text is used to guide the image to be processed to perform image transformation according to the attribute characteristics of the target image;

[0029] Inputting the image to be processed into the visual encoder in the pre-trained image generation model to obtain the image latent variable corresponding to the image to be processed, and inputting the forward guided text into the language encoder in the pre-trained image generation model to obtain the text feature corresponding to the forward guided text;

[0030] Input the image latent variables and text features into the latent variable encoder in the pre-trained image generation model to obtain the semantically edited image latent variables;

[0031] The semantically edited image latent variables are input into the visual decoder in the pre-trained image generation model to obtain a transformed image corresponding to the image to be processed; the transformed image has the attribute characteristics of the target image.

[0032] A training device for an image generation model, comprising:

[0033] An acquisition module is used to acquire an image training sample set and determine a forward guiding text and a reverse guiding text for the image training sample set; the forward guiding text is used to guide the image training sample set to perform image transformation according to the target image attribute characteristics; the reverse guiding text describes the image training sample attribute characteristics of the image training sample set;

[0034] The training module is used to perform the following training process using any image training sample in the image training sample set to train the image generation model:

[0035] Inputting the forward guided text and the reverse guided text into the language encoder of the image generation model to obtain text features corresponding to the forward guided text and the reverse guided text respectively;

[0036] Obtain image latent variables corresponding to image training samples, and input the image latent variables and text features corresponding to the forward-guided text into the latent variable encoder of the image generation model to obtain semantically edited image latent variables;

[0037] Input the semantically edited image latent variables and the text features corresponding to the reverse guided text into the latent variable decoder of the image generation model to obtain the semantically restored image latent variables;

[0038] According to the loss information determined by the image latent variables corresponding to the image training samples and the image latent variables after semantic restoration, the latent variable encoder and latent variable decoder in the image generation model are trained.

[0039] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0040] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0041] A computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0042] The training method, apparatus, computer device, storage medium and computer program product of the above-mentioned image generation model obtain an image training sample set and determine a forward guiding text and a reverse guiding text for the image training sample set; the forward guiding text is used to guide the image training sample set to perform image transformation according to the target image attribute features; the reverse guiding text describes the original image attribute features of the image training sample set; any image training sample in the image training sample set is used to perform the following training process to train the image generation model: the forward guiding text and the reverse guiding text are input into the language encoder of the image generation model to obtain the text features corresponding to the forward guiding text and the reverse guiding text respectively; the image latent variables corresponding to the image training sample are obtained, and the image latent variables and the text features corresponding to the forward guiding text are input into the latent variable encoder of the image generation model to obtain the semantically edited image latent variables; the semantically edited image latent variables and the text features corresponding to the reverse guiding text are input into the latent variable decoder of the image generation model to obtain the semantically restored image latent variables; according to the image latent variables and The loss information of the image hidden variables after semantic restoration is determined, and the hidden variable encoder and the hidden variable decoder in the image generation model are trained. In this way, when adding editing attributes for the image, the original image attributes and the target image attributes that need to be transformed can be determined to construct forward guiding text and reverse guiding text, and the forward guiding text and reverse guiding text with opposite semantics are used to guide the image generation model for training. An image generation model that can edit the image attributes of an original image with the original image attribute characteristics to obtain a target image with the target image attribute characteristics is obtained through training. The problems of difficult model convergence and low image generation efficiency in the training method of the traditional image generation model are solved. At the same time, the features in the image latent space can be located more accurately and stably without modifying irrelevant features, and the non-editing area of ​​the original image is retained with high fidelity. At the same time, since no other annotation data is required in the process of guiding training, only two guiding texts with opposite semantics are required, which improves the quality of visual features aligned with text, and reduces the workload of data preparation while improving the image attribute editing capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A diagram of an application environment of a training method for an image generation model in one embodiment;

[0045] Figure 2 A schematic diagram of a flow chart of a training method for an image generation model in an embodiment;

[0046] Figure 3 is a schematic diagram of an image reconstruction model in one embodiment;

[0047] Figure 4 is a schematic diagram of a semantic encoding reconstruction model in one embodiment;

[0048] Figure 5 is a schematic diagram of an image generation model in one embodiment;

[0049] Figure 6 is a schematic flow chart of an image generating method in an embodiment;

[0050] Figure 7 A schematic diagram of a flow chart of a training method for an image generation model in another embodiment;

[0051] Figure 8 A structural block diagram of a training device for an image generation model in one embodiment;

[0052] Fig. 9 is a structural block diagram of an image generating device in one embodiment;

[0053] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] The image generation model of this application can perform semantic editing on attributes that are not related to the category, and generate images in a targeted manner through semantic information, such as the modification and editing of specific attributes such as faces, bodies, and objects. The image generation model provided by this application is based on text-conditional image semantic editing, which aims to enhance visual representation learning based on the text information provided by the user as a guide, thereby supporting fine-grained attribute editing and obtaining high-quality generated images. Among them, image semantic editing includes changes to image content and narrative, affecting the story, background, or theme elements of the depicted scene. Tasks within this category include object addition, object removal, object replacement, background change, and emotional expression modification.

[0056] It should be noted that the training method of the image generation model proposed in this application is essentially an image semantic editing method based on conditional text. It is guided by given conditional text and uses unsupervised learning methods to edit the potential semantic information of the image to achieve precise control of related attributes. The image generation model adopts a two-stage training strategy during training, including training an image reconstruction model in the first stage to construct the potential semantic space of the data set and obtain image feature coding. The first stage is the unsupervised learning stage; the second stage is training a semantic coding reconstruction model to fuse text descriptions and obtain feature coding after operating attributes. The second stage is the conditional learning stage. During model reasoning, the image generation model reorganizes some of the modules that have completed the training above, and works together to complete the editing and synthesis of the image. By decoupling the model training process into two-stage division of labor training, the resources consumed by the model calculation can be reduced. At the same time, the two are connected in series in an autoregressive manner, so that the model has the ability to generate higher quality images. At the same time, after decoupling, each module will gain greater flexibility, providing the possibility of migrating to richer data types and model paradigms. For example, when adding editing attributes to the same batch of data, only the second-stage module needs to be fine-tuned; or when adding the same type of data, only the first-stage module needs to be fine-tuned without retraining the entire model framework. Furthermore, if the prompt method is not text but appears in other ways, such as 3D perception or structure guidance, the first-stage model can still be retained; or when changing other generation modes, the second-stage method can still be applied.

[0057] The training method of the image generation model provided in the embodiment of the present application can be applied to Figure 1In the application environment shown, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. The server 104 obtains an image training sample set, and determines a forward guiding text and a reverse guiding text for the image training sample set; the forward guiding text is used to guide the image training sample set to perform image transformation according to the target image attribute features; the reverse guiding text describes the original image attribute features of the image training sample set; the server 104 uses any image training sample in the image training sample set to perform the following training process to train the image generation model: the server 104 inputs the forward guiding text and the reverse guiding text into the language encoder of the image generation model to obtain the text features corresponding to the forward guiding text and the reverse guiding text respectively; the server 104 obtains the image latent variables corresponding to the image training sample, and inputs the image latent variables and the text features corresponding to the forward guiding text into the latent variable encoder of the image generation model to obtain the semantically edited image latent variables; the server 104 inputs the semantically edited image latent variables and the text features corresponding to the reverse guiding text into the latent variable decoder of the image generation model to obtain the semantically restored image latent variables; the server 104 trains the latent variable encoder and the latent variable decoder in the image generation model according to the loss information determined by the image latent variables corresponding to the image training sample and the semantically restored image latent variables. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0058] In an exemplary embodiment, Figure 2 As shown, a training method for an image generation model is provided, which corresponds to the training method of the second stage of the above model training. Figure 1 Taking the server 104 in the example as an example, the following steps are included:

[0059] Step S210, obtain an image training sample set, and determine a forward guiding text and a reverse guiding text for the image training sample set; the forward guiding text is used to guide the image training sample set to perform image transformation according to the target image attribute characteristics; the reverse guiding text describes the original image attribute characteristics of the image training sample set.

[0060] The image training sample set includes multiple image training samples, and the image training samples in the image training sample set have the same original image attribute characteristics. For example, the image training samples in the image training sample set are all middle-aged face images.

[0061] Among them, the positive guidance text can describe the direction of image transformation, which is used to guide the image to transform image attributes. The positive guidance text can be recorded as .

[0062] Among them, the reverse guidance text can describe the direction of image restoration, that is, describe the original attributes of the image. The reverse guidance text can be recorded as .

[0063] In practical applications, the forward guidance text and the reverse guidance text need to be determined according to the training objectives. For example, if the training goal is to train an image generation model that can transform the age of a face from an original young person to an old person, then the forward guidance text is "old person" and the reverse guidance text is "young person".

[0064] The target image attribute feature may refer to the attribute feature of the image to be generated, and the original image attribute feature may refer to the attribute feature of the original image. For example, if the training goal is to train an image generation model that can transform the age of a face from a young person to an old person, then the target image attribute feature may refer to the face feature of an old person, and the original image attribute feature may refer to the face feature of a young person.

[0065] Optionally, after obtaining the image training sample set, it is necessary to determine the training target for the image generation model, that is, to determine the image editing function that the trained image generation model should have. If the training goal is to train an image generation model that can transform a face from a "young person" to an "old person", that is, to determine that the trained image generation model has the image editing function of face aging, then the forward guidance text is determined to be "old person" and the reverse guidance text is determined to be "young person".

[0066] Step S220, using any image training sample in the image training sample set to perform the following training process to train the image generation model:

[0067] Step S222, input the forward guidance text and the reverse guidance text into the language encoder of the image generation model to obtain the text features corresponding to the forward guidance text and the reverse guidance text respectively.

[0068] Among them, the language encoder is used to encode the text.

[0069] Among them, the text feature corresponding to the positive guidance text can be the text feature vector corresponding to the positive guidance text, which is recorded as , the text feature corresponding to the reverse guidance text can be the text feature vector corresponding to the reverse guidance text, recorded as .

[0070] The traditional method of discovering the direction vector of image attributes is to calculate the average value of positive vectors and negative vectors. If there are many attributes to be edited, it takes a lot of time to distinguish between positive and negative samples and enumerate and calculate them one by one; or when multiple attributes are involved in editing, it is necessary to consider the weight distribution. The text-guided image attribute editing used in this application can more accurately and stably locate the features in the image latent space without modifying irrelevant features, thereby retaining the non-editing area of ​​the original image with high fidelity, and no other annotation data is required during the guidance process. Only two semantically opposite text descriptions are required to improve the quality of visual features aligned with the text. Therefore, this method reduces the workload of data preparation while improving editing capabilities.

[0071] Optionally, the forward guided text and the reverse guided text are input into the language encoder of the image generation model, and the forward guided text and the reverse guided text are encoded by the language encoder of the image generation model to obtain text encoding results corresponding to the forward guided text and text encoding results corresponding to the reverse guided text, that is, text features corresponding to the forward guided text and the reverse guided text are obtained respectively.

[0072] Step S224, obtaining the image latent variables corresponding to the image training samples, and inputting the image latent variables and the text features corresponding to the forward guided text into the latent variable encoder of the image generation model to obtain the semantically edited image latent variables.

[0073] Among them, the image latent variable is obtained by discretizing the image training samples after encoding them using the pre-trained video encoder. The image latent variable can be recorded as It should be noted that the image latent variable here is encoded by the video encoder in the pre-trained image reconstruction model obtained by the training method of the first stage.

[0074] Among them, the latent variable encoder is used to encode the latent variables.

[0075] Among them, the semantically edited image latent variables carry text prompt information.

[0076] Optionally, the image training sample can be input into a pre-trained visual encoder to obtain a continuous feature vector of the image training sample, and then the continuous feature vector of the image training sample can be input into a codebook to obtain a discrete image latent variable , the image hidden variable Text features corresponding to the forward guidance text Input to the latent variable encoder of the image generation model, output the image latent variable carrying the prompt text, that is, the semantically edited image latent variable .

[0077] Step S226, inputting the semantically edited image latent variables and the text features corresponding to the reverse guided text into the latent variable decoder of the image generation model to obtain the semantically restored image latent variables.

[0078] Among them, the latent variable decoder is used to decode the latent variables.

[0079] Optionally, the semantically edited image latent variable Text features corresponding to the reverse guidance text After being input into the latent variable decoder of the image generation model, the output is the semantically restored image latent variable .

[0080] Step S228, training the latent variable encoder and the latent variable decoder in the image generation model according to the image latent variables corresponding to the image training samples and the loss information determined by the image latent variables after semantic restoration.

[0081] The loss information can be composed of two parts: reconstruction error and codebook alignment error. The reconstruction error is the difference between the original image training sample I and the reconstructed image The codebook alignment error uses L2 loss to measure the error between the codebook encoding and the visual encoding vector, hoping to build a stable and effective semantic discrete space.

[0082] Optionally, according to the image latent variables corresponding to the image training samples And the image latent variables after semantic restoration The determined reconstruction error and codebook alignment error are used to adjust the parameters of the latent variable encoder and latent variable decoder in the image generation model to achieve model iteration.

[0083] The above steps S222 to S228 are to construct positive guiding text and negative guiding text on the pre-trained image reconstruction model to fine-tune the semantic encoding reconstruction model, thereby obtaining an image generation model. The image reconstruction model can be found in Figure 3 The image reconstruction model includes a visual encoder and a visual decoder. The semantic encoding reconstruction model can be found in Figure 4The semantic coding reconstruction model includes a latent variable encoder Latent Code Encoder, a latent variable decoder Latent Code Decoder, and a language encoder LanguageEncoder with fixed weights for encoding text; the image generation model can be Figure 5 , the image generation model includes a visual encoder Visual Encoder, a latent variable encoder Latent Code Encoder, a visual decoder Visual Decoder, and a language encoder Language Encoder. This application uses VQ-VAE as the model framework. The image generation method of the image generation model follows the VAE model. Compared with other generation models, this method has certain advantages in generating diversity, handling missing data and noise, interpretability, and learning high-dimensional data representation: sufficient learning of data distribution enables it to have the ability to fill and remove noise, which can reduce the cost of data cleaning preparation; good interpretability makes it easier to find problems in the model during training, which is conducive to iterative optimization; it supports high-dimensional feature representation, and the model can handle more complex data types or richer encoding types.

[0084] In the training method of the above-mentioned image generation model, an image training sample set is obtained, and a forward guiding text and a reverse guiding text for the image training sample set are determined; the forward guiding text is used to guide the image training sample set to perform image transformation according to the target image attribute features; the reverse guiding text describes the original image attribute features of the image training sample set; any image training sample in the image training sample set is used to perform the following training process to train the image generation model: the forward guiding text and the reverse guiding text are input into the language encoder of the image generation model to obtain the text features corresponding to the forward guiding text and the reverse guiding text respectively; the image latent variables corresponding to the image training sample are obtained, and the image latent variables and the text features corresponding to the forward guiding text are input into the latent variable encoder of the image generation model to obtain the semantically edited image latent variables; the semantically edited image latent variables and the text features corresponding to the reverse guiding text are input into the latent variable decoder of the image generation model to obtain the semantically restored image latent variables; the loss signal is determined according to the image latent variables corresponding to the image training sample and the semantically restored image latent variables. The latent variable encoder and latent variable decoder in the image generation model are trained based on the information; in this way, when adding editing attributes for the image, the original image attributes of the image and the target image attributes that need to be transformed can be determined to construct forward guiding text and reverse guiding text, and the forward guiding text and reverse guiding text with opposite semantics are used to guide the image generation model for training, and an image generation model that can edit the image attributes of the original image with the original image attribute characteristics to obtain a target image with the target image attribute characteristics is obtained through training, which solves the problems of difficult model convergence and low image generation efficiency in the training method of the traditional image generation model. At the same time, it can more accurately and stably locate the features in the latent space of the image without modifying irrelevant features, and retain the non-editing area of ​​the original image with high fidelity, ensuring the fidelity of the generated image. At the same time, since no other annotation data is required in the process of guiding training, only two guiding texts with opposite semantics are required, which improves the quality of visual features aligned with text, while improving the image attribute editing ability and reducing the workload of data preparation.

[0085] In an exemplary embodiment, the image generation model also includes a visual encoder to obtain image latent variables corresponding to image training samples, including: inputting the image training samples into the visual encoder in the image generation model, encoding the image training samples through the visual encoder to obtain encoding features corresponding to the image training samples; inputting the encoding features into a codebook to discretize the encoding features to obtain image latent variables corresponding to the image training samples.

[0086] The codebook refers to the codebook under the VQ-VAE model architecture, in which K vectors are pre-stored. The K vectors pre-stored on the codebook are used to match the encoding features corresponding to the image training samples to determine the closest vector.

[0087] Optionally, in practical applications, after inputting the image training sample into the visual encoder, a Feature Map Then, the feature map In The D-dimensional vectors are calculated similarity with the K vectors in the codebook respectively, and the closest vector is found , expressed as index, we get The characteristic matrix , then, for the feature map Each index in the codebook uses the hidden variable corresponding to the codebook Represents that we get a new feature map . Later, the new feature map can be As the input of the visual decoder, the reconstructed image is finally obtained through the visual decoder.

[0088] In this embodiment, the image training samples are input into the visual encoder in the image generation model, and the image training samples are encoded by the visual encoder to obtain the encoding features corresponding to the image training samples; the encoding features are input into the code book to discretize the encoding features to obtain the image latent variables corresponding to the image training samples; in this way, the continuous encoding features corresponding to the image training samples can be input into the code book, so as to quickly and accurately obtain discrete image latent variables.

[0089] In an exemplary embodiment, the image generation model also includes a visual decoder, and the visual encoder and the visual decoder are pre-trained. The training process of the visual encoder and the visual decoder is: inputting the image training samples into the visual encoder to be trained, and encoding the image training samples through the visual encoder to be trained to obtain the encoding features corresponding to the image training samples; inputting the encoding features corresponding to the image training samples into the code book to discretize the encoding features corresponding to the image training samples to obtain the image latent variables corresponding to the image training samples; inputting the image latent variables corresponding to the image training samples into the visual decoder to be trained, and decoding the image latent variables corresponding to the image training samples through the visual decoder to be trained to obtain the restored image corresponding to the image training samples; and training the visual encoder to be trained and the visual decoder to be trained according to the difference between the image training samples and the restored image.

[0090] Among them, the visual encoder and visual decoder are in the image reconstruction model, see Figure 3 The image reconstruction model includes a visual encoder Visual Encoder and a visual decoder Visual Decoder. By training the image reconstruction model, a pre-trained image reconstruction model is obtained, and the pre-trained visual encoder and visual decoder in the pre-trained image reconstruction model are used as the visual encoder and visual decoder of the image generation model.

[0091] Optionally, the image training sample I is input into the visual encoder to be trained to obtain a Feature Map Then, the feature map In The D-dimensional vectors are calculated similarity with the K vectors in the codebook respectively, and the closest vector is found , expressed as index, we get The characteristic matrix , then, for the feature map Each index in the codebook uses the hidden variable corresponding to the codebook Represents that we get a new feature map Then, the new feature map Input to the visual decoder to be trained, and convert the new feature map Decode to restore image Then, the loss function is constructed, including the reconstruction error and the codebook alignment error. The reconstruction error is the difference between the image training sample I and the restored image The mean square error of the two is expected to have a high similarity. The codebook alignment error uses L2 loss to measure the error between the codebook encoding and the visual encoding vector. It is expected to build a stable and effective semantic discrete space. Based on the loss information determined by the loss function, the image reconstruction model is trained, that is, the visual encoder and the visual decoder are trained to obtain the pre-trained visual encoder and the pre-trained visual decoder.

[0092] The above-mentioned codebook converts continuous feature vectors into discrete latent variables, uses vectorization technology to discretize semantic features, and encodes image features through discrete representation methods. Since the prior distribution obtained by model training will also be discrete encoding, and the image generation process of this application focuses on and focuses on editing perceptible attributes represented by discrete information, vectorization technology makes attribute information easier to distinguish and quantify, and filters the noise of non-discrete information, thereby completing meaningful image compression and obtaining a semantic space containing richer features, which is conducive to the modeling of visual decoders. At the same time, both high-frequency and low-frequency signals of the image can be effectively processed, which contains more details that the user can perceive, and improves editing controllability. The constructed semantic feature space used to describe the data set can also be applied to other downstream tasks, such as targeted feature differences and quantitative modifications of the same type of data.

[0093] The technical solution of this embodiment corresponds to the training method of the first stage of the above-mentioned model training, which corresponds to the unsupervised learning stage. Unsupervised learning is a machine learning method that uses machine learning algorithms to analyze unlabeled data sets and perform clustering or dimensionality reduction. The inherent structure and relationship in the data can be discovered without human intervention. Specifically, by training Figure 3 The image reconstruction model shown uses VQ-VAE as the model architecture, which contains two Transformer-based trainable modules: Visual Encoder and Visual Decoder.

[0094] In this embodiment, the image training samples are input into the visual encoder to be trained, and the image training samples are encoded by the visual encoder to be trained to obtain the encoding features corresponding to the image training samples; the encoding features corresponding to the image training samples are input into the code book to discretize the encoding features corresponding to the image training samples to obtain the image latent variables corresponding to the image training samples; the image latent variables corresponding to the image training samples are input into the visual decoder to be trained, and the image latent variables corresponding to the image training samples are decoded by the visual decoder to be trained to obtain the restored images corresponding to the image training samples; the visual encoder to be trained and the visual decoder to be trained are trained according to the difference between the image training samples and the restored images; in this way, the image reconstruction model can be firstly trained unsupervisedly, so as to obtain the pre-trained visual encoder and visual decoder, and complete the first stage of training in the model training. If it is necessary to add editing attributes for the image later, it is only necessary to fine-tune the second stage of the model training.

[0095] In an exemplary embodiment, the image latent variables and the text features corresponding to the positive guidance text are input into the latent variable encoder of the image generation model to obtain the semantically edited image latent variables, including: inputting the image latent variables and the text features corresponding to the positive guidance text into the latent variable encoder of the image generation model, fusing the image latent variables and the text features corresponding to the positive guidance text through the cross-attention mechanism of the latent variable encoder to obtain a first fusion result; and using the first fusion result as the semantically edited image latent variables.

[0096] Optionally, the image latent variable Text features corresponding to the forward guidance text The latent variable encoder input into the image generation model, through the cross attention mechanism of the latent variable encoder, fuses the text features of the image training sample and the forward guidance text, and obtains the first fusion result, that is, the semantically edited image latent variable .

[0097] In this embodiment, the image latent variables and the text features corresponding to the positive guidance text are input into the latent variable encoder of the image generation model, and the text features corresponding to the image latent variables and the positive guidance text are fused through the cross-attention mechanism of the latent variable encoder to obtain a first fusion result; the first fusion result is used as the image latent variable after semantic editing; in this way, the image latent variables and the image latent variables of the positive guidance text can be fused to obtain accurate image latent variables carrying the prompt text.

[0098] In an exemplary embodiment, the semantically edited image latent variables and the text features corresponding to the reverse guided text are input into the latent variable decoder of the image generation model to obtain the semantically restored image latent variables, including: inputting the semantically edited image latent variables and the text features corresponding to the reverse guided text into the latent variable decoder of the image generation model, fusing the semantically edited image latent variables and the text features of the reverse guided text through the cross attention mechanism in the latent variable decoder to obtain a second fusion result; and using the second fusion result as the semantically restored image latent variables.

[0099] Optionally, the semantically edited image latent variable Text features corresponding to the reverse guidance text The latent variable decoder input to the image generation model uses the cross attention mechanism in the latent variable decoder to analyze the semantically edited image latent variables. and text features of reverse-guided text Fusion is performed to obtain the second fusion result, that is, the image latent variable after semantic restoration .

[0100] In this embodiment, the semantically edited image latent variables and the text features corresponding to the reverse guided text are input into the latent variable decoder of the image generation model, and the text features of the semantically edited image latent variables and the reverse guided text are fused through the cross-attention mechanism in the latent variable decoder to obtain a second fusion result; the second fusion result is used as the semantically restored image latent variables; in this way, the semantically edited image latent variables and the reverse guided text image latent variables can be fused to obtain accurate semantically restored image latent variables.

[0101] In an exemplary embodiment, Figure 6 As shown, an image generation method is provided, corresponding to the model reasoning stage of the pre-trained image generation model. The model architecture of the pre-trained image generation model is as follows Figure 5 As shown in FIG. 1 , the method is composed of a trained visual encoder Visual Encoder, a latent variable encoder Latent Code Encoder, a visual decoder VisualDecoder and a language encoder Language Encoder. The latent variable decoder Latent Code Decoder involved in the above embodiment is mainly used to supervise the training of the latent variable encoder in the second stage, i.e., the conditional learning stage, and is not needed in the inference process. Figure 1 Taking the server 104 in the example as an example, the following steps are included:

[0102] Step S602, obtaining an image to be processed, and obtaining a positive guidance text for the image to be processed; the positive guidance text is used to guide the image to be processed to perform image transformation according to the attribute characteristics of the target image.

[0103] Optionally, obtain the image to be processed , and construct the forward guidance text .

[0104] Step S604: input the image to be processed into the visual encoder in the pre-trained image generation model to obtain the image latent variables corresponding to the image to be processed, and input the forward guided text into the language encoder in the pre-trained image generation model to obtain the text features corresponding to the forward guided text.

[0105] Optionally, the image to be processed Input to the visual encoder to obtain the hidden variables of the image to be processed , and input the forward-guided text into the language encoder to obtain the text feature vector ,by Figure 4 For example, if we want to transform the target face in the image to be processed into an elderly person, then For “elderly people”.

[0106] Step S606: input the image latent variables and text features into the latent variable encoder in the pre-trained image generation model to obtain the semantically edited image latent variables.

[0107] Optionally, the image latent variable of the image to be processed With text feature vector Input the latent variable encoder and output the latent variable after modifying the age attribute in the feature space .

[0108] Step S608, inputting the semantically edited image latent variables into the visual decoder in the pre-trained image generation model to obtain a transformed image corresponding to the image to be processed; the transformed image has the attribute characteristics of the target image.

[0109] Optionally, the semantically edited latent variables Input the visual decoder to generate the target face image edited for age .

[0110] The above-mentioned image generation method obtains an image to be processed, and obtains a positive guidance text for the image to be processed; the positive guidance text is used to guide the image to be processed to perform image transformation according to the target image attribute characteristics; the image to be processed is input into the visual encoder in the pre-trained image generation model to obtain the image latent variables corresponding to the image to be processed, and the positive guidance text is input into the language encoder in the pre-trained image generation model to obtain the text features corresponding to the positive guidance text; the image latent variables and the text features are input into the latent variable encoder in the pre-trained image generation model to obtain the semantically edited image latent variables; the semantically edited image latent variables are input into the visual decoder in the pre-trained image generation model to obtain the transformed image corresponding to the image to be processed; the transformed image has the target image attribute characteristics; in this way, under the guidance of the positive guidance text, each module in the pre-trained image generation model can be used to efficiently process the image to be processed, transform the image to be processed into an image with the target image attribute characteristics, and improve the image generation quality.

[0111] The beneficial effects of the image generation model of this application include:

[0112] 1) Generate high-fidelity attribute editing images. The discretization technology supports the model to build a rich semantic space, filter out noise information that may interfere, and fully learn prior knowledge; by introducing text modality, the positioning and modification of the features to be edited will be more accurate and reliable, and other semantically irrelevant information will be retained as much as possible to prevent image distortion; high-quality images and strong editability make the image generation model of this application particularly suitable for data enhancement tasks. By preparing labels for missing types of data, samples can be supplemented in a targeted manner, thereby improving the generalization ability of the model and preventing overfitting that may occur during training.

[0113] 2) It has high practical value, reduces computing overhead, and saves training and inference costs. Other generation paradigms may face problems such as training instability and model collapse during training, which require fine-tuning of parameters and training strategies; some generation paradigms have too many complicated steps in the inference process and require more computing resources. This application uses a simpler and more intuitive model structure, which combines stability and computing speed without losing image generation quality; during model training, it has good tolerance for noisy data, saving time and manpower costs for data cleaning; the two-stage training model can reduce computational complexity and improve the flexibility of migrating to downstream tasks; the use of a self-supervised learning paradigm improves data utilization, overcomes the pain point of limited annotation in actual application scenarios, and lowers the threshold for model training; during model inference, a non-autoregressive method is used to generate high-quality images in one go, reducing delays and waiting time.

[0114] In another embodiment, Figure 7 As shown, a training method for an image generation model is provided, and the method is applied to Figure 1 Taking the server 104 in the example as an example, the following steps are included:

[0115] Step S7100, obtain an image training sample set, and determine a forward guiding text and a reverse guiding text for the image training sample set; the forward guiding text is used to guide the image training sample set to perform image transformation according to the target image attribute characteristics; the reverse guiding text describes the original image attribute characteristics of the image training sample set.

[0116] Step S7200: Use any image training sample in the image training sample set to perform the following training process to train the image generation model:

[0117] Step S7210, input the forward guidance text and the reverse guidance text into the language encoder of the image generation model to obtain the text features corresponding to the forward guidance text and the reverse guidance text respectively.

[0118] Step S7220: input the image training sample into the visual encoder in the image generation model, encode the image training sample through the visual encoder, and obtain the encoding features corresponding to the image training sample.

[0119] Step S7230: input the coding features into a codebook to discretize the coding features to obtain image latent variables corresponding to the image training samples.

[0120] Step S7240, input the image latent variables and the text features corresponding to the forward guided text into the latent variable encoder of the image generation model to obtain the semantically edited image latent variables.

[0121] Step S7250, inputting the semantically edited image latent variables and the text features corresponding to the reverse guided text into the latent variable decoder of the image generation model to obtain the semantically restored image latent variables.

[0122] Step S7260, training the latent variable encoder and the latent variable decoder in the image generation model according to the image latent variables corresponding to the image training samples and the loss information determined by the image latent variables after semantic restoration.

[0123] It should be noted that the specific limitations of the above steps can refer to the specific limitations of the training method for an image generation model mentioned above.

[0124] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0125] Based on the same inventive concept, the embodiment of the present application also provides a training device for an image generation model for implementing the training method for the image generation model involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the training device for one or more image generation models provided below can refer to the limitations of the training method for the image generation model above, and will not be repeated here.

[0126] In an exemplary embodiment, Figure 8As shown, a training device for an image generation model is provided, including: an acquisition module 802 and a training module 804, wherein:

[0127] The acquisition module 802 is used to acquire an image training sample set and determine a forward guiding text and a reverse guiding text for the image training sample set; the forward guiding text is used to guide the image training sample set to perform image transformation according to the target image attribute characteristics; the reverse guiding text describes the image training sample attribute characteristics of the image training sample set;

[0128] The training module 804 is used to perform the following training process using any image training sample in the image training sample set to train the image generation model:

[0129] Inputting the forward guided text and the reverse guided text into the language encoder of the image generation model to obtain text features corresponding to the forward guided text and the reverse guided text respectively;

[0130] Obtain image latent variables corresponding to image training samples, and input the image latent variables and text features corresponding to the forward-guided text into the latent variable encoder of the image generation model to obtain semantically edited image latent variables;

[0131] Input the semantically edited image latent variables and the text features corresponding to the reverse guided text into the latent variable decoder of the image generation model to obtain the semantically restored image latent variables;

[0132] According to the loss information determined by the image latent variables corresponding to the image training samples and the image latent variables after semantic restoration, the latent variable encoder and latent variable decoder in the image generation model are trained.

[0133] In one embodiment, the image generation model also includes a visual encoder, and the training module 804 is specifically used to input the image training samples into the visual encoder in the image generation model, encode the image training samples through the visual encoder, and obtain the encoding features corresponding to the image training samples; input the encoding features into the code book to discretize the encoding features and obtain the image latent variables corresponding to the image training samples.

[0134] In one embodiment, the image generation model also includes a visual decoder, and the visual encoder and the visual decoder are pre-trained. The training module 804 is specifically used to input the image training samples into the visual encoder to be trained, and encode the image training samples through the visual encoder to be trained to obtain the encoding features corresponding to the image training samples; input the encoding features corresponding to the image training samples into the code book to discretize the encoding features corresponding to the image training samples to obtain the image latent variables corresponding to the image training samples; input the image latent variables corresponding to the image training samples into the visual decoder to be trained, and decode the image latent variables corresponding to the image training samples through the visual decoder to be trained to obtain the restored image corresponding to the image training samples; and train the visual encoder to be trained and the visual decoder to be trained according to the difference between the image training samples and the restored image.

[0135] In one embodiment, the training module 804 is specifically used to input the image latent variables and the text features corresponding to the positive guidance text into the latent variable encoder of the image generation model, and fuse the image latent variables and the text features corresponding to the positive guidance text through the cross-attention mechanism of the latent variable encoder to obtain a first fusion result; and use the first fusion result as the semantically edited image latent variables.

[0136] In one embodiment, the training module 804 is specifically used to input the semantically edited image latent variables and the text features corresponding to the reverse guided text into the latent variable decoder of the image generation model, and fuse the semantically edited image latent variables and the text features of the reverse guided text through the cross-attention mechanism in the latent variable decoder to obtain a second fusion result; and use the second fusion result as the semantically restored image latent variables.

[0137] In an exemplary embodiment, Fig. 9 As shown, an image generation device is provided, including: an information acquisition module 902, an encoding module 904, an editing module 906 and a transformation module 908, wherein:

[0138] The acquisition module 902 is used to acquire the image to be processed and acquire the positive guidance text for the image to be processed; the positive guidance text is used to guide the image to be processed to perform image transformation according to the attribute characteristics of the target image;

[0139] The encoding module 904 is used to input the image to be processed into the visual encoder in the pre-trained image generation model to obtain the image latent variable corresponding to the image to be processed, and input the forward guidance text into the language encoder in the pre-trained image generation model to obtain the text feature corresponding to the forward guidance text;

[0140] An editing module 906, used to input the image latent variables and text features into a latent variable encoder in a pre-trained image generation model to obtain semantically edited image latent variables;

[0141] The transformation module 908 is used to input the semantically edited image latent variables into the visual decoder in the pre-trained image generation model to obtain a transformed image corresponding to the image to be processed; the transformed image has the attribute characteristics of the target image.

[0142] Each module in the training device of the above-mentioned image generation model can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0143] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.10 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store training data of the image generation model. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a training method for an image generation model is implemented.

[0144] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0145] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the training method of the above-mentioned image generation model. The steps of the training method of the image generation model here may be the steps in the training method of the image generation model of each of the above-mentioned embodiments.

[0146] In one embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the processor executes the steps of the training method of the above-mentioned image generation model. The steps of the training method of the image generation model here can be the steps in the training method of the image generation model of each of the above-mentioned embodiments.

[0147] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, causes the processor to execute the steps of the training method for an image generation model described above. The steps of the training method for an image generation model here may be the steps in the training method for an image generation model described in each of the above embodiments.

[0148] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0149] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0150] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A training method for an image generation model, characterized in that: The method comprises: Acquire an image training sample set, and determine a forward guiding text and a reverse guiding text for the image training sample set; the forward guiding text is used to guide the image training sample set to perform image transformation according to target image attribute features; the reverse guiding text describes the image training sample attribute features of the image training sample set; Use any image training sample in the image training sample set to perform the following training process to train the image generation model: Inputting the forward guiding text and the reverse guiding text into the language encoder of the image generation model to obtain text features corresponding to the forward guiding text and the reverse guiding text respectively; Obtaining image latent variables corresponding to the image training samples, and inputting the image latent variables and text features corresponding to the forward-guided text into a latent variable encoder of the image generation model to obtain semantically edited image latent variables; Inputting the semantically edited image latent variables and the text features corresponding to the reverse guided text into the latent variable decoder of the image generation model to obtain the semantically restored image latent variables; The latent variable encoder and the latent variable decoder in the image generation model are trained according to the image latent variables corresponding to the image training samples and the loss information determined by the image latent variables after semantic restoration.

2. The method according to claim 1, characterized in that The image generation model further includes a visual encoder, and the step of obtaining image latent variables corresponding to the image training samples includes: Inputting the image training sample into the visual encoder in the image generation model, encoding the image training sample through the visual encoder to obtain the encoding features corresponding to the image training sample; The coding features are input into a codebook to discretize the coding features to obtain image latent variables corresponding to the image training samples.

3. The method according to claim 2, characterized in that The image generation model further includes a visual decoder, the visual encoder and the visual decoder are obtained by pre-training, and the training process of the visual encoder and the visual decoder is: Inputting the image training sample into the visual encoder to be trained, encoding the image training sample by the visual encoder to be trained, and obtaining the encoding features corresponding to the image training sample; Inputting the coding features corresponding to the image training samples into a codebook to discretize the coding features corresponding to the image training samples to obtain image latent variables corresponding to the image training samples; Inputting the image latent variables corresponding to the image training samples into the visual decoder to be trained, and decoding the image latent variables corresponding to the image training samples by the visual decoder to be trained to obtain the restored images corresponding to the image training samples; The visual encoder to be trained and the visual decoder to be trained are trained according to the difference between the image training sample and the restored image.

4. The method according to claim 1, characterized in that The step of inputting the image latent variables and the text features corresponding to the forward-guided text into the latent variable encoder of the image generation model to obtain the semantically edited image latent variables comprises: Inputting the image latent variable and the text features corresponding to the positive guidance text into the latent variable encoder of the image generation model, and fusing the image latent variable and the text features corresponding to the positive guidance text through the cross attention mechanism of the latent variable encoder to obtain a first fusion result; The first fusion result is used as the semantically edited image latent variable.

5. The method according to claim 1, characterized in that The step of inputting the semantically edited image latent variables and the text features corresponding to the reverse guided text into the latent variable decoder of the image generation model to obtain the semantically restored image latent variables includes: Inputting the semantically edited image latent variables and the text features corresponding to the reverse guided text into the latent variable decoder of the image generation model, and fusing the semantically edited image latent variables and the text features of the reverse guided text through a cross attention mechanism in the latent variable decoder to obtain a second fusion result; The second fusion result is used as the image latent variable after the semantic restoration.

6. An image generation method, characterized in that: The method comprises: Acquire an image to be processed, and acquire a positive guidance text for the image to be processed; the positive guidance text is used to guide the image to be processed to perform image transformation according to the target image attribute characteristics; Inputting the image to be processed into a visual encoder in a pre-trained image generation model to obtain image latent variables corresponding to the image to be processed, and inputting the forward-guided text into a language encoder in the pre-trained image generation model to obtain text features corresponding to the forward-guided text; Inputting the image latent variables and the text features into the latent variable encoder in the pre-trained image generation model to obtain the semantically edited image latent variables; The semantically edited image latent variables are input into the visual decoder in the pre-trained image generation model to obtain a transformed image corresponding to the image to be processed; the transformed image has the attribute characteristics of the target image.

7. A training device for an image generation model, characterized in that: The device comprises: An acquisition module is used to acquire an image training sample set and determine a forward guiding text and a reverse guiding text for the image training sample set; the forward guiding text is used to guide the image training sample set to perform image transformation according to target image attribute features; the reverse guiding text describes the image training sample attribute features of the image training sample set; The training module is used to perform the following training process using any image training sample in the image training sample set to train the image generation model: Inputting the forward guiding text and the reverse guiding text into the language encoder of the image generation model to obtain text features corresponding to the forward guiding text and the reverse guiding text respectively; Obtaining image latent variables corresponding to the image training samples, and inputting the image latent variables and text features corresponding to the forward-guided text into a latent variable encoder of the image generation model to obtain semantically edited image latent variables; Inputting the semantically edited image latent variables and the text features corresponding to the reverse guided text into the latent variable decoder of the image generation model to obtain the semantically restored image latent variables; The latent variable encoder and the latent variable decoder in the image generation model are trained according to the image latent variables corresponding to the image training samples and the loss information determined by the image latent variables after semantic restoration.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images