Text-to-image generation method, device, equipment, storage medium and product

Abstract

This application discloses a text-to-image generation method, apparatus, device, storage medium, and product, relating to the field of artificial intelligence technology. The disclosed text-to-image generation method includes: receiving input text; parsing multiple semantic entities and their spatial relationships in the input text using a large language model to obtain layout information corresponding to the input text, wherein the layout information includes the position range of each semantic entity in the image; converting the layout information into a mask set corresponding to the size of the latent space feature map of the diffusion model; inputting the mask set and the input text into the diffusion model, and performing semantic correction based on the mask set during the denoising process of the input text by the diffusion model to obtain the target image. This application can improve the matching degree between the generated image and the input text during text-to-image generation.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to methods, apparatus, devices, storage media, and products for text-to-image generation. Background Technology

[0002] Images, as an important carrier of information, can express complex concepts more intuitively and vividly than plain text, making it easier for audiences to understand and receive content. Text-to-image generation technology aims to automatically generate visually realistic images based on text descriptions. It is an important research direction at the intersection of computer vision and natural language processing, and is also regarded as one of the key technologies for realizing general artificial intelligence.

[0003] Current text-to-image generation technologies often fail to accurately resolve semantic relationships between multiple objects when dealing with complex text prompts, including quantitative relationships, attribute relationships, and spatial relationships, resulting in significant discrepancies between the generated images and the text descriptions.

[0004] In summary, improving the matching degree between the generated image and the input text during text-to-image generation has become a pressing technical problem that needs to be solved in this field. Summary of the Invention

[0005] The main objective of this application is to provide a text-to-image generation method, apparatus, device, storage medium, and product, which aims to improve the matching degree between the generated image and the input text during text-to-image generation.

[0006] To achieve the above objectives, this application proposes a text-to-image generation method, which includes:

[0007] The system receives input text, parses multiple semantic entities in the input text and the spatial relationships between the semantic entities using a large language model, and obtains layout information corresponding to the input text, wherein the layout information includes the position range of each semantic entity in the image;

[0008] The layout information is converted into a mask set corresponding to the size of the latent space feature map of the diffusion model;

[0009] The mask set and the input text are input into the diffusion model, and semantic correction is performed based on the mask set during the denoising process of the input text by the diffusion model to obtain the target image.

[0010] In one embodiment, the step of parsing multiple semantic entities and the spatial relationships between the semantic entities in the input text using a large language model to obtain the layout information corresponding to the input text includes:

[0011] The input text is reasoned through a large language model to obtain a scene relationship graph, wherein the scene relationship graph represents multiple semantic entities in the input text and the spatial relationships between the semantic entities in a structured form;

[0012] The layout information corresponding to the input text is obtained by performing layout reasoning on the input text and the scene relationship graph using the large language model.

[0013] In one embodiment, before the step of parsing multiple semantic entities and the spatial relationships between the semantic entities in the input text using a large language model to obtain the layout information corresponding to the input text, the method further includes:

[0014] Obtain a hybrid mind chain dataset, wherein the hybrid mind chain dataset includes image layout data from real-world scenarios and predicted layout data generated by an initial large language model;

[0015] The initial large language model is fine-tuned based on the hybrid thought chain dataset to obtain the large language model.

[0016] In one embodiment, the semantic correction includes iterative semantic correction and / or self-looping semantic correction;

[0017] The step of performing semantic correction based on the mask set to obtain the target image during the denoising process of the input text by the diffusion model includes:

[0018] During the denoising process of the input text by the diffusion model, the iterative semantic correction is performed based on the mask set to update the image latent variables, and / or the self-looping semantic correction is performed to update the text embedding vector;

[0019] Based on the updated image latent variables and / or the updated text embedding vector, the input text is denoised using the diffusion model to obtain the target image.

[0020] In one embodiment, the step of performing the iterative semantic correction based on the mask set to update the image latent variables includes:

[0021] In multiple time steps of the denoising process, a cross-attention loss is calculated based on the mask set and the first cross-attention map generated by the diffusion model. The cross-attention loss is used to constrain the text lexical attention of each semantic entity to focus on the corresponding mask region.

[0022] Based on the self-attention map generated by the mask set and the diffusion model, a self-attention loss is calculated, which is used to enhance the pixel correlation within the mask region corresponding to each semantic entity.

[0023] The total latent variable loss is obtained by combining the cross-attention loss with the self-attention loss.

[0024] The image latent variables are updated during the denoising process based on the total loss of the latent variables.

[0025] In one embodiment, the step of performing the self-looping semantic correction to update the text embedding vector includes:

[0026] In multiple time steps of the denoising process, the second cross-attention map is Gaussian smoothed to obtain a smoothed cross-attention map;

[0027] The text loss is calculated based on the smoothed cross-attention map, and the text loss is used to balance the attention of the diffusion model to different text words in the input text.

[0028] The text embedding vector in the denoising process is updated based on the text loss.

[0029] Furthermore, to achieve the above objectives, this application also proposes a text-to-image generation apparatus, which includes:

[0030] The layout information acquisition module is used to receive input text, parse multiple semantic entities in the input text and the spatial relationships between the semantic entities through a large language model, and obtain the layout information corresponding to the input text, wherein the layout information includes the position range of each semantic entity in the image;

[0031] The layout information conversion module is used to convert the layout information into a mask set corresponding to the size of the latent space feature map of the diffusion model;

[0032] An image generation module is used to input the mask set and the input text into the diffusion model, and to perform semantic correction based on the mask set during the denoising process of the input text by the diffusion model to obtain the target image.

[0033] In addition, to achieve the above objectives, this application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the text-to-image generation method as described above.

[0034] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the text-to-image generation method described above.

[0035] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the text-to-image generation method as described above.

[0036] This application proposes a text-to-image generation method. The method receives input text, parses multiple semantic entities and their spatial relationships using a large language model to obtain layout information corresponding to the input text. This layout information includes the positional range of each semantic entity in the image. The layout information is then converted into a mask set corresponding to the size of the latent space feature map of the diffusion model. The mask set and the input text are input into the diffusion model, and semantic correction is performed based on the mask set during the denoising process of the input text by the diffusion model to obtain the target image.

[0037] In summary, this application utilizes a large language model to parse the semantic entities and spatial relationships of the input text, obtains layout information, and converts it into a mask set adapted to the diffusion model. Then, during the denoising process of the diffusion model, semantic correction is performed based on the mask set to accurately obtain the semantic information in the input text and accurately map it onto the generated image. This allows the generated image to more accurately reflect the content described by the input text, thereby improving the matching degree between the generated image and the input text. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating an embodiment of the text-to-image generation method of this application.

[0041] Figure 2 This is a schematic diagram of the layout information generation process provided in Embodiment 2 of the text-to-image generation method of this application;

[0042] Figure 3 This is a schematic diagram of cross-attention constraints provided in Embodiment 3 of the text-to-image generation method of this application;

[0043] Figure 4 This is a schematic diagram of self-attention constraints provided in Embodiment 3 of the text-to-image generation method of this application;

[0044] Figure 5 This is a schematic diagram of the bidirectional semantic correction process provided in Embodiment 3 of the text-to-image generation method of this application;

[0045] Figure 6 This is a schematic diagram of the module structure of the text-to-image generation apparatus according to an embodiment of this application;

[0046] Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the text-to-image generation method in the embodiments of this application.

[0047] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0049] Images, as an important carrier of information, can express complex concepts more intuitively and vividly than plain text, making it easier for audiences to understand and receive content. Text-to-image generation technology aims to automatically generate visually realistic images based on text descriptions. It is an important research direction at the intersection of computer vision and natural language processing, and is also regarded as one of the key technologies for realizing general artificial intelligence.

[0050] Current text-to-image generation technologies often fail to accurately resolve semantic relationships between multiple objects when dealing with complex text prompts, including quantitative relationships, attribute relationships, and spatial relationships, resulting in significant discrepancies between the generated images and the text descriptions.

[0051] In summary, improving the matching degree between the generated image and the input text during text-to-image generation has become a pressing technical problem that needs to be solved in this field.

[0052] This application provides a solution that receives input text, parses multiple semantic entities and their spatial relationships using a large language model to obtain layout information corresponding to the input text, wherein the layout information includes the position range of each semantic entity in the image; converts the layout information into a mask set corresponding to the size of the latent space feature map of the diffusion model; inputs the mask set and the input text into the diffusion model, and performs semantic correction based on the mask set during the denoising process of the diffusion model on the input text to obtain the target image.

[0053] In summary, the embodiments of this application utilize a large language model to parse the semantic entities and spatial relationships of the input text, obtain layout information, and convert it into a mask set adapted to the diffusion model. Then, during the denoising process of the diffusion model, semantic correction is performed based on the mask set to accurately obtain the semantic information in the input text and accurately map it onto the generated image. This allows the generated image to more accurately reflect the content described by the input text, thereby improving the matching degree between the generated image and the input text.

[0054] It should be noted that the executing entity in this embodiment can be an electronic device with data processing, network communication, and program execution functions, such as a computer, host computer, controller, etc., or an electronic device capable of performing the above functions. The following uses an image generation system as an example to describe this embodiment and the subsequent embodiments.

[0055] Based on this, embodiments of this application provide a text-to-image generation method, referring to... Figure 1 , Figure 1 This is a schematic flowchart of the first embodiment of the text-to-image generation method of this application.

[0056] In this embodiment, the text-to-image generation method includes steps S10 to S30:

[0057] Step S10: Receive input text, parse multiple semantic entities in the input text and the spatial relationships between semantic entities through a large language model, and obtain the layout information corresponding to the input text. The layout information includes the position range of each semantic entity in the image.

[0058] It's important to note that the input text refers to descriptive text provided by the user, describing the content of the image to be generated. A large language model is a pre-trained language model, such as the GPT (a series of large language models), which possesses powerful natural language understanding and generation capabilities, able to parse semantic information and relationships within text. Semantic entities refer to specific objects or concepts mentioned in the text; these entities are the objects to be visualized in the image. Spatial relationships refer to the relative positional relationships between semantic entities, used to describe the layout of objects in the image. Layout information refers to data representing the positional extent of semantic entities in the image, typically represented by bounding boxes, and may include the center coordinates, width, and height of each entity.

[0059] First, the system receives a text description input from the user. Then, it uses a large language model to parse the input text. Leveraging its pre-trained knowledge and reasoning capabilities, the large language model identifies semantic entities in the text and the spatial relationships between them. The large language model outputs layout information, including the bounding box coordinates of each semantic entity in the image.

[0060] In practice, large language models can be fine-tuned to optimize layout prediction tasks, for example, by using a hybrid thought chain dataset to fine-tune the model to improve the accuracy of layout prediction.

[0061] Step S20: Convert the layout information into a mask set corresponding to the size of the latent space feature map of the diffusion model;

[0062] It's important to note that diffusion models are generative models that generate images from random noise through progressive denoising, such as the StableDiffusion model. Its core is the U-Net (convolutional neural network) structure, used to process feature representations in the latent space. The latent space feature map refers to the intermediate representation of the U-Net encoder in the diffusion model, typically with a low resolution, such as 16x16, used to efficiently process semantic information during image generation. The mask set refers to a group of binary masks, each corresponding to a semantic entity. Pixels within the mask region have a value of 1, and pixels outside the region have a value of 0, used to indicate the position of the semantic entity in the latent space.

[0063] In practical applications, the bounding boxes in the layout information are based on the original image size of 512x512. These bounding boxes need to be converted to a 16x16 size latent space feature map of the diffusion model. The conversion method is to scale the bounding boxes by dividing their coordinates and dimensions by 32 to obtain their coordinates and dimensions in the latent space. For example, for the bounding box (x, y, w, h), the converted coordinates are (x / 32, y / 32) and the dimensions are (w / 32, h / 32).

[0064] After resizing the layout information, a binary mask is generated for each semantic entity. The mask is a 16x16 matrix, matching the size of the latent space feature map of the diffusion model. Pixel values ​​inside the resized bounding box are set to 1, indicating that the region should contain the target object; pixel values ​​outside the bounding box are set to 0, representing the background. Thus, a mask set is obtained. Where k is the number of semantic entities, each It is a 16x16 binary mask. The mask set will be used for semantic correction in the subsequent diffusion model to ensure that the generated image is consistent with the layout information.

[0065] Step S30: Input the mask set and the input text into the diffusion model, and perform semantic correction based on the mask set during the denoising process of the input text in the diffusion model to obtain the target image.

[0066] It should be noted that the denoising process refers to the process by which the diffusion model uses sampling algorithms such as DDIM (Denoising Diffusion Implicit Models) to progressively remove noise from random noise to generate an image, involving iterative processing across multiple time steps. Semantic correction refers to adjusting model parameters or intermediate variables through a loss function during the denoising process to ensure that the generated image is consistent with the text description. The target image refers to the final generated image, which is semantically and visually consistent with the input text description.

[0067] The input text and mask set are fed into the diffusion model, which can use a text encoder such as CLIP (Contrastive Language-Image Pre-Training) to convert the input text into an embedding vector, which is then used as a conditional input into U-Net. During the denoising process, semantic correction is performed based on the mask set.

[0068] It's worth noting that semantic correction is typically performed early in the denoising process (e.g., the first 20 steps), with correction stopping later to maintain detail stability. Ultimately, the diffusion model outputs a target image that matches the input text description, with correct object locations and attributes.

[0069] Therefore, by using a large language model to parse the semantic entities and their spatial relationships in the input text, layout information is obtained and converted into a mask set that is compatible with the diffusion model. Then, during the denoising process of the diffusion model, semantic correction is performed based on the mask set to accurately obtain the semantic information in the input text and accurately map it onto the generated image. This makes the generated image more accurately reflect the content described by the input text, thereby improving the matching degree between the generated image and the input text.

[0070] Based on Embodiment 1 of this application, in Embodiment 2 of this application, the content that is the same as or similar to that in Embodiment 1 can be referred to the above description, and will not be repeated hereafter. Based on this, step S10 may include steps S101~S102:

[0071] Step S101: Reasoning is performed on the input text using a large language model to obtain a scene relationship graph, wherein the scene relationship graph represents multiple semantic entities in the input text and the spatial relationships between semantic entities in a structured form;

[0072] It should be noted that a scene relationship graph is a graph structure in which nodes represent semantic entities and edges represent spatial relationships between entities. For example, a triple in the form of "<object1, relation, object2>" can be used, where object1 and object2 are semantic entities, such as clock and door, and the relation is a spatial relationship, such as left and right.

[0073] Large language models such as the GPT series are used to parse the input text. Specifically, the input text is first embedded into a predefined input template, and the scene graph parser is used to complete the reasoning. The scene graph parser is the large language model used to parse the input text. The large language model generates a scene relationship graph through thought chain reasoning. The scene relationship graph can be output in the form of triples, which clarifies the spatial positional relationship between semantic entities and avoids misunderstanding the semantics or missing objects when parsing the text.

[0074] For example, if a user inputs the text "The clock is above the door, and the light switch is to the left of the door", this input text is embedded into a predefined input template. Then, the input text is subjected to LLM (Large Language Model) inference through a large language model to clarify the spatial relationships between the objects involved in each input text, resulting in a scene relationship graph. The scene relationship graph is specifically presented in the form of triples such as "<clock, above, door>, <light switch, left, door>".

[0075] Step S102: Perform layout reasoning on the input text and scene relationship graph using a large language model to obtain the layout information corresponding to the input text.

[0076] It should be noted that layout reasoning refers to the process of generating bounding box information for each semantic entity based on the input text and scene relationship graph, ensuring that spatial relationships are reasonably represented in the image.

[0077] The input text and scene relationship graph are combined and embedded into another input template, for example, "Generate layout based on the following text and scene relationship graph: text: {input text}, scene relationship graph: {scene relationship graph}". The large language model infers the location range of each semantic entity based on the text and scene relationship graph.

[0078] It is worth mentioning that the same type of large language model or different large language models can be used to generate scene relationship graphs and generate layout information. For example, GPT-3.5Turbo (a type of large language model) can be used as the scene graph parser to generate scene relationship graphs, and GPT-4o (a type of large language model) can be used as the layout predictor to generate layout information.

[0079] The scene graph parser generates a scene relationship graph as an intermediate representation based on the input text, which structures the text semantics and reduces semantic misunderstandings and object omissions. The layout inferencer generates bounding boxes of semantic entities based on the input text and the scene relationship graph, which improves the rationality and accuracy of layout prediction.

[0080] For example, such as Figure 2As shown, the user inputs the text "The clock is above the door, and the light switch is to the left of the door". This input text is parsed by the scene graph parser to obtain the scene relationship graph "<clock, above, door>, <light switch, left, door>". The input text and scene relationship graph are then input into the layout inferencer for LLM layout inference to obtain the layout information "[('clock',[220,100,80,80]).('door',[200,250,200,250]).('light switch',[120,260,40,40])]".

[0081] Furthermore, in one feasible implementation, after obtaining layout information through a large language model, the layout information undergoes post-processing optimization, including: performing non-maximum suppression on overlapping semantic entity bounding boxes to eliminate overlapping areas; cropping or repositioning bounding boxes that exceed the image boundaries to place them within the valid image area; and adjusting the bounding box size according to the semantic category of the semantic entities to conform to common object proportions. This effectively avoids problems such as overlapping objects, unreasonable positions, or disproportionate proportions in the generated image, improving the visual rationality and semantic consistency of the generated image, and further enhancing the matching degree between text and image.

[0082] Furthermore, in another feasible implementation, an importance weight can be assigned to each semantic entity in the layout information. The weight is determined based on at least one of the following: the frequency of the semantic entity's occurrence in the input text; the centrality of the semantic entity in the scene relationship graph; and the semantic category priority of the semantic entity. Then, during semantic correction, the weights of the loss function are adjusted according to the importance weights. This allows the model to focus more on important semantic entities, avoid interference from secondary objects in the generation process, improve the generation quality and positional accuracy of key objects, and thus better meet the user's visual expression needs for core content.

[0083] In one feasible embodiment, steps A10 to A20 may be included before step S10:

[0084] Step A10: Obtain the hybrid mind chain dataset, which includes image layout data from real-world scenarios and predicted layout data generated by the initial large language model.

[0085] It should be noted that the Hybrid Mind Chain Dataset is a training dataset containing both real and predicted data, used to fine-tune the large language model to improve its generalization ability and robustness in layout prediction tasks. The real-world image layout data refers to layout information extracted from real images, while the predicted layout data refers to text prompts and corresponding layout information generated by the initial large language model, obtained through model inference.

[0086] Real-world data can be directly obtained by filtering images containing 2-5 objects from the MS-COCO dataset (a large dataset for image recognition, segmentation, and description in computer vision) and acquiring their bounding box layout information. Specifically, GPT-4Vision (GPT-4withVision, a multimodal language model with visual capabilities) is used for Visual Question Answering (VQA) tasks to automatically infer the positional relationships of objects in the images and generate scene relationship information and image captions. For prediction data collection, GPT-4o is used to generate text prompts containing multiple objects and their spatial relationships. The output format is flexibly transformed through instructions to construct different types of multi-object scene relationships. Then, these text prompts are processed by a scene parser and a layout predictor to generate scene relationships and layout information. Real-world and prediction data are mixed in a 7:2 ratio and stratified according to the number of objects, with less data for more complex scenes, to ensure the model can handle layout prediction tasks of varying difficulty.

[0087] Step A20: Fine-tune the initial large language model based on the hybrid thought chain dataset to obtain the large language model.

[0088] The initial large language model refers to the large language model to be trained. In one feasible implementation, the initial large language model is fine-tuned, with the batch size set to 1, the learning rate using the default value, and a training iteration cycle of 3 epochs. The training model uses GPT-3.5Turbo and GPT-4o as the base models for fine-tuning. The training time varies depending on the model size; for example, GPT-3.5Turbo takes approximately 8 minutes, and GPT-4o takes approximately 1 hour and 25 minutes. The fine-tuned large language model exhibits better generalization ability and robustness in layout prediction tasks, and can accurately resolve semantic entities and spatial relationships in complex texts.

[0089] Therefore, in this embodiment, by fine-tuning the large language model using a hybrid thinking chain dataset, and combining real data and predicted data, the limitations of a single data source are overcome, the model's ability to recognize real-world multi-object scenes is improved, and the fine-tuned large language model can more stably and accurately infer layout information, reducing the problems of object omission and position errors in text-to-image generation.

[0090] Based on Embodiment 1 and / or Embodiment 2 of this application, in Embodiment 3 of this application, the content that is the same as or similar to Embodiment 1 and Embodiment 2 can be referred to the above description, and will not be repeated hereafter. On this basis, semantic correction includes iterative semantic correction and / or self-looping semantic correction; step S30 may include steps S301~S302:

[0091] Step S301: In the process of denoising the input text by the diffusion model, iterative semantic correction is performed based on the mask set to update the latent variables of the image, and / or self-looping semantic correction is performed to update the text embedding vector.

[0092] It should be noted that iterative semantic correction refers to updating the image latent variables through a loss function during the denoising process of the diffusion model, including cross-attention loss and self-attention loss, to constrain the attention distribution and enhance pixel correlation. Self-loop semantic correction refers to updating the text embedding vector through a loss function during the denoising process, used to balance the model's attention to different text terms. Here, the image latent variables refer to the intermediate noise representation processed by U-Net in the diffusion model, used to generate the image; the text embedding vector refers to the text representation generated by a text encoder such as CLIP, which serves as conditional input to the diffusion model.

[0093] During the denoising process of the diffusion model on the input text, iterative semantic correction is performed based on the mask set to update the image latent variables. The iterative semantic correction calculates cross-attention loss and self-attention loss at multiple time steps during denoising. Cross-attention loss ensures that the attention of text terms is focused on the corresponding mask regions, while self-attention loss strengthens the correlation between pixels within the mask regions. Simultaneously, self-loop semantic correction is performed based on the mask set to update the text embedding vector. Self-loop semantic correction calculates text loss during denoising, which is used to balance the diffusion model's attention to different text terms in the input text. Both semantic corrections can be applied simultaneously or selectively during denoising, depending on the requirements of the generation task.

[0094] Furthermore, in a feasible implementation scenario, the step of semantic correction based on the mask set further includes: dynamically selecting whether to perform semantic correction based on the noise level during the diffusion model denoising process; performing semantic correction when the noise level is higher than a set threshold, and stopping correction when it is lower than the threshold. This allows for enhanced semantic control in the early stages of image generation, ensuring the correct position and structure of objects; and reduced intervention in the later stages of image generation, preserving the naturalness of details.

[0095] In one feasible embodiment, the step S301 of "performing iterative semantic correction based on the mask set to update image latent variables" may include steps S3011 to S3014:

[0096] Step S3011: In multiple time steps of the denoising process, based on the first cross-attention map generated by the mask set and the diffusion model, calculate the cross-attention loss. The cross-attention loss is used to constrain the text lexical attention of each semantic entity to focus on the corresponding mask region.

[0097] It should be noted that the first cross-attention map represents the mean of all cross-attention layers in the attention head dimension. The cross-attention loss is a loss function used to measure the consistency between text lexical attention and the masked region, encouraging attention to be focused on the masked region.

[0098] In multiple time steps of the denoising process, the first cross-attention map is extracted from the cross-attention layer of the diffusion model. Where N is the number of lexical units in the input text, including ordinary lexical units and special lexical units such as [SoT] (Start of Text) and [EoT] (End of Text). Since [SoT] primarily focuses on global information and [EoT] primarily focuses on foreground entities, their semantic information is already covered by ordinary lexical units; therefore, the attention contributions of [SoT] and [EoT] are ignored during the calculation. The attention values ​​of the remaining lexical units are normalized using softmax (normalized exponential function) to ensure that the attention distribution of each lexical unit conforms to probability constraints. For each semantic entity's target lexical unit... Its attention map With the corresponding mask Calculate the loss.

[0099] The cross-attention loss function is:

[0100] ;

[0101] Where L is the number of pixels in the feature map, and · represents element-wise multiplication. This loss function calculates the ratio of the attention score in the mask region to the total attention score. The higher the ratio, the smaller the loss, thus constraining attention to be focused on the mask region.

[0102] Step S3012: Based on the self-attention map generated by the mask set and diffusion model, calculate the self-attention loss. The self-attention loss is used to enhance the pixel correlation within the mask region corresponding to each semantic entity.

[0103] It should be noted that the self-attention map refers to the output of the self-attention layer in the U-Net diffusion model, representing the correlation between pixels within an image region. The self-attention loss is a loss function used to enhance the attention between pixels within the masked region, maintaining the coherence of the object's internal structure.

[0104] Extracting self-attention maps from the self-attention layers of the diffusion model L is the number of pixels in the feature map, and the binary mask for each target entity. Self-attention is aggregated within the masked region to ensure that self-attention in the early diffusion stage aligns with the target region. The formula for calculating the total attention score for all image terms is as follows:

[0105] ;

[0106] in, Indicates from In the self-attention map, select the pixels located within the bounding box to obtain the self-attention map of the target region. ,in This represents the number of pixels within a bounding box. Then, the summation of all pixels within the bounding box is used to calculate the total attention score of image words within that region to all image words. .

[0107] The self-attention loss function is:

[0108] ;

[0109] This loss function encourages higher self-attention scores within the masked region, strengthens the correlation between pixels, and ensures consistency in the internal structure of the object.

[0110] Step S3013: Combine the cross-attention loss and self-attention loss to obtain the total latent variable loss;

[0111] It should be noted that the total latent variable loss refers to the total loss of iterative semantic correction, which is used to update the latent variables of the image and combines the constraints of cross attention and self attention.

[0112] Combining cross-attention loss and self-attention loss yields the total latent variable loss. , where λ is a weight hyperparameter used to balance the relative importance of the two losses, and the value of λ can be set based on the actual application scenario.

[0113] Step S3014: Update the latent variables of the image during the denoising process based on the total loss of latent variables.

[0114] In each denoising process, the total loss of the latent variables is calculated. Then, the latent variables of the image are updated using gradient descent, with the update formula being: ,in, Represents the total loss for latent variables. Regarding latent variables Find the gradient. This is the learning rate, which is adjusted proportionally to set values ​​for different time steps. During model inference, the learning rate is adjusted based on the current time step. Select a scaling factor Calculate the current learning rate ,in, It is a global scaling factor that determines the overall scale of the learning rate. For time steps The corresponding learning rate scaling factor results in a larger learning rate in early time steps and a smaller learning rate in later time steps.

[0115] The above update steps are repeated in each round of noise reduction. (A predefined maximum number of iterations) times; after reaching this number, the progress of the next time step is calculated. The next step is noise reduction.

[0116] Thus, through the specific calculation and combination of cross-attention loss and self-attention loss, cross-attention loss ensures that text lexical attention is focused on the correct region, reducing attribute leakage and object loss; self-attention loss enhances the coherence of the internal structure of objects and avoids errors in positional relationships. The latent variable update mechanism dynamically corrects the semantic direction, improving the semantic accuracy and visual quality of the generated image.

[0117] For example, in one feasible implementation scenario, Figure 3 This paper describes the application of cross-attention constraints in iterative semantic correction, used to ensure that text lexical attention is focused on the corresponding mask region during diffusion model denoising. Taking one time step in the denoising process as an example, firstly, for image features f... l The input text "A dog sits directly to the right of a cat" is first encoded to obtain a text embedding, and then converted into a key matrix K through a linear layer. Finally, the attention weight matrix (i.e., the first cross-attention map) is calculated using a softmax function. Where d represents the feature dimension, and finally, using this attention weight matrix, the semantics of "cat" and "dog" in the text are associated with the image regions respectively, generating the corresponding object attention mappings, i.e. Figure 3 Image regions corresponding to cats and dogs.

[0118] In one feasible embodiment, the step of "performing self-loop semantic correction to update the text embedding vector" in step S301 may include steps S3015 to S3017:

[0119] Step S3015: In multiple time steps of the denoising process, the second cross-attention map is Gaussian smoothed to obtain a smoothed cross-attention map.

[0120] It should be noted that the second cross-attention map represents the attention head dimension of each cross-attention layer. Gaussian smoothing refers to using a Gaussian kernel convolutional attention map to smooth the attention values, considering the influence of adjacent regions to reduce noise and highlight the overall attention distribution. The smoothed cross-attention map is the cross-attention map after Gaussian smoothing, used to calculate the text loss.

[0121] In multiple time steps of the denoising process, a second cross-attention map is extracted from the cross-attention layer of the diffusion model. ,in, 't' represents the time step, and 't' represents the layer number. Gaussian smoothing is applied to the attention map of each text word to obtain a smoothed cross-attention map. After Gaussian smoothing, the attention value of each image word becomes a linear combination of its neighboring words, thus taking into account the attention scores of local regions and making the attention distribution smoother and more continuous.

[0122] Step S3016: Calculate the text loss based on the smoothed cross-attention map. The text loss is used to balance the attention of the diffusion model to different text words in the input text.

[0123] For smoothing cross-attention maps Calculate the maximum attention value for each text word: , representing the value of each word in the region of most salient attention in the image. Then, the average of these maximum values ​​is calculated. and variance The average value represents the overall attention given to all words, while the variance represents the balance of attention.

[0124] The text loss function is:

[0125] ;

[0126] Step S3017: Update the text embedding vector in the denoising process according to the text loss.

[0127] Text embedding vectors refer to the text representations generated by the CLIP text encoder, which are conditionally input into the diffusion model and updated to more accurately align attention semantic information.

[0128] Calculate the text loss function Next, the text embedding vector of the current layer is updated. :

[0129] ;

[0130] in, Indicates the first The update of the layer is used for the first layer. The text embedding vector of the layer, where β is the learning rate. Indicates text loss Regarding text embedding vectors Calculate the gradient. The update is performed at each layer, and this update does not affect the denoising inference process of U-Net. Instead, it is performed as an independent semantic optimization module, which performs a self-loop update at each layer to achieve a more accurate alignment between text tags and attention semantic information.

[0131] Thus, by dynamically adjusting the text embedding vector, the model's attention to different text terms is balanced, preventing the model from overemphasizing certain terms while ignoring others. This ensures that all semantic entities in the generated image are correctly represented, improving the overall semantic alignment accuracy and the integrity of the generated image.

[0132] For example, in a feasible implementation scenario, such as Figure 4 As shown, Figure 4 This paper describes the application of self-attention constraints in iterative semantic correction, used to enhance the pixel correlation within the mask regions corresponding to each semantic entity during the diffusion model denoising process. Taking one time step in the denoising process as an example, firstly, the image features f... l The process is divided into two paths: one path generates the query matrix Q through a linear layer, and the other path first copies f. l The same feature matrix is ​​obtained, and then a key matrix K is generated through a linear layer. Finally, the attention weight matrix (i.e., the second cross-attention map) is calculated using the softmax function. Where d represents the feature dimension, the features corresponding to Q and K are then split according to the number of attention heads, and feature fusion is performed by element-wise addition (⊕) to obtain intermediate features of multiple attention heads. Then, a binary mask is applied to each target entity. Self-attention is aggregated within the masked region to ensure that self-attention in the early diffusion stage aligns with the target region, and the total attention score for all image units is calculated. Finally, the features are weighted using the total attention score to obtain the final single-channel semantic feature matrix, which is the process from multiple stacked matrices to a single square matrix shown in the figure.

[0133] Step S302 uses a diffusion model to denoise the input text based on the updated image latent variables and / or the updated text embedding vector to obtain the target image.

[0134] Following semantic correction, the denoising process continues using updated image latent variables and / or text embedding vectors. In one feasible implementation, the diffusion model uses the DDIM sampling algorithm, with a total of 50 inference steps. Semantic correction is applied to the first 30 steps, with t decreasing from 50 to 20, after which semantic correction is stopped to maintain detail stability. The denoising process progressively reduces noise, ultimately generating the target image.

[0135] Therefore, by using iterative semantic correction and self-looping semantic correction, the image latent variables and text embedding vectors are dynamically adjusted during the diffusion denoising process, effectively correcting the semantic offset problem. Specifically, iterative semantic correction ensures that attention is focused on the correct regions, while self-looping semantic correction balances the attention given to text terms, thereby improving the semantic accuracy and visual quality of the generated image and making the generated image more consistent with the text description.

[0136] For example, in a feasible implementation scenario, such as Figure 5 As shown, Figure 5 This paper demonstrates the overall process of a bidirectional semantic correction algorithm. This algorithm combines Iterative Semantic Correction (ISC) and Self-Recursive Semantic Correction (SRSC) during the diffusion model denoising process to dynamically correct the semantic direction. In the diffusion model denoising step, for the image latent variable z at time step t > 30... t After passing through the network layers, iterative semantic correction is first performed, incorporating self-attention (SelfAttn) and cross-attention (CrossAttn) mechanisms, with the latent variable z... t z is obtained after iterative semantic correction. t *, then, z t *As z t Perform self-loop semantic correction to complete this iteration, and continue to calculate the image latent variables for the next time step. Semantic correction is used to gradually generate content that is closer to the real image.

[0137] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the text-to-image generation method of this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0138] This application also provides a text-to-image generation apparatus; please refer to... Figure 6 The text-to-image generation apparatus includes:

[0139] The layout information acquisition module 10 is used to receive input text, parse multiple semantic entities in the input text and the spatial relationships between semantic entities through a large language model, and obtain the layout information corresponding to the input text. The layout information includes the position range of each semantic entity in the image.

[0140] The layout information conversion module 20 is used to convert the layout information into a mask set corresponding to the size of the latent space feature map of the diffusion model.

[0141] The image generation module 30 is used to input the mask set and the input text into the diffusion model, and to perform semantic correction based on the mask set during the denoising process of the input text in the diffusion model to obtain the target image.

[0142] Optionally, the layout information acquisition module 10 is also used for:

[0143] By reasoning about the input text using a large language model, a scene relationship graph is obtained. The scene relationship graph represents multiple semantic entities in the input text and the spatial relationships between semantic entities in a structured form.

[0144] By using a large language model to perform layout reasoning on the input text and scene relationship graph, the layout information corresponding to the input text is obtained.

[0145] Optionally, the text-to-image generation apparatus also includes a model fine-tuning module (not shown), which is used for:

[0146] Obtain the hybrid mind chain dataset, which includes image layout data from real-world scenarios and predicted layout data generated by the initial large language model;

[0147] The initial large language model was fine-tuned based on the hybrid thought chain dataset to obtain the large language model.

[0148] Optionally, semantic correction includes iterative semantic correction and / or self-looping semantic correction; the image generation module 30 is also used for:

[0149] During the denoising process of the input text by the diffusion model, iterative semantic correction is performed based on the mask set to update the image latent variables, and / or self-looping semantic correction is performed to update the text embedding vector;

[0150] Based on the updated image latent variables and / or the updated text embedding vector, the input text is denoised using a diffusion model to obtain the target image.

[0151] Optionally, the image generation module 30 is also used for:

[0152] In multiple time steps of the denoising process, cross-attention loss is calculated based on the cross-attention map generated by the mask set and the diffusion model. The cross-attention loss is used to constrain the text lexical attention of each semantic entity to focus on the corresponding mask region.

[0153] Based on the self-attention map generated by the mask set and diffusion model, the self-attention loss is calculated. The self-attention loss is used to enhance the pixel correlation within the mask region corresponding to each semantic entity.

[0154] The total latent variable loss is obtained by combining cross-attention loss and self-attention loss.

[0155] The latent variables of the image are updated during the denoising process based on the total loss of the latent variables.

[0156] Optionally, the image generation module 30 is also used for:

[0157] In multiple time steps of the denoising process, the cross-attention map is Gaussian smoothed to obtain a smoothed cross-attention map.

[0158] The text loss is calculated based on a smoothed cross-attention map. The text loss is used to balance the attention of the diffusion model to different text terms in the input text.

[0159] The text embedding vectors in the denoising process are updated based on the text loss.

[0160] The text-to-image generation apparatus provided in this application, employing the text-to-image generation method described in the above embodiments, can improve the matching degree between the generated image and the input text during text-to-image generation. Compared with the prior art, the beneficial effects of the text-to-image generation apparatus provided in this application are the same as those of the text-to-image generation method described in the above embodiments, and other technical features in the text-to-image generation apparatus are the same as those disclosed in the text-to-image generation method described in the above embodiments, and will not be repeated here.

[0161] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the text-to-image generation method in the first embodiment described above.

[0162] The following is for reference. Figure 7 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of this application. The electronic devices in these embodiments may include, but are not limited to, mobile terminals such as laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0163] like Figure 7As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the electronic device to exchange data with other devices wirelessly or via wired communication. Although the diagrams show electronic devices with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented alternatively.

[0164] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0165] The electronic device provided in this application, employing the text-to-image generation method described in the above embodiments, can improve the matching degree between the generated image and the input text during text-to-image generation. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the text-to-image generation method provided in the above embodiments, and other technical features of this electronic device are the same as those disclosed in the text-to-image generation method of the previous embodiment, and will not be repeated here.

[0166] It should be understood that the various parts disclosed in the embodiments of this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0167] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0168] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the text-to-image generation method in the above embodiments.

[0169] The computer-readable storage medium provided in this application embodiment may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0170] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0171] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by an electronic device, the electronic device: receives input text, parses multiple semantic entities in the input text and the spatial relationships between semantic entities through a large language model, and obtains layout information corresponding to the input text, wherein the layout information includes the position range of each semantic entity in the image; converts the layout information into a mask set corresponding to the size of the latent space feature map of the diffusion model; inputs the mask set and the input text into the diffusion model, and performs semantic correction based on the mask set during the denoising process of the diffusion model on the input text, thereby obtaining the target image.

[0172] Computer program code for performing the operations of the embodiments of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0173] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0174] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0175] The readable storage medium provided in this application embodiment is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described text-to-image generation method, which can improve the matching degree between the generated image and the input text during text-to-image generation. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application embodiment are the same as the beneficial effects of the text-to-image generation method provided in the above embodiments, and will not be repeated here.

[0176] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the text-to-image generation method described above.

[0177] The computer program product provided in this application can solve the technical problem of text-to-image generation. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the text-to-image generation method provided in the above embodiments, and will not be repeated here.

[0178] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A text-to-image generation method, characterized in that, The text-to-image generation method includes: The system receives input text, parses multiple semantic entities in the input text and the spatial relationships between the semantic entities using a large language model, and obtains layout information corresponding to the input text, wherein the layout information includes the position range of each semantic entity in the image; The layout information is converted into a mask set corresponding to the size of the latent space feature map of the diffusion model; The mask set and the input text are input into the diffusion model, and semantic correction is performed based on the mask set during the denoising process of the input text by the diffusion model to obtain the target image; The semantic correction includes iterative semantic correction and / or self-looping semantic correction; The step of performing semantic correction based on the mask set to obtain the target image during the denoising process of the input text by the diffusion model includes: During the denoising process of the input text by the diffusion model, the iterative semantic correction is performed based on the mask set to update the image latent variables, and / or the self-looping semantic correction is performed to update the text embedding vector; Based on the updated image latent variables and / or the updated text embedding vector, the input text is denoised using the diffusion model to obtain the target image; The step of performing the iterative semantic correction based on the mask set to update the image latent variables includes: In multiple time steps of the denoising process, a cross-attention loss is calculated based on the mask set and the first cross-attention map generated by the diffusion model. The cross-attention loss is used to constrain the text lexical attention of each semantic entity to focus on the corresponding mask region. Based on the self-attention map generated by the mask set and the diffusion model, a self-attention loss is calculated, which is used to enhance the pixel correlation within the mask region corresponding to each semantic entity. The total latent variable loss is obtained by combining the cross-attention loss with the self-attention loss. The image latent variables are updated during the denoising process based on the total loss of the latent variables.

2. The text-to-image generation method as described in claim 1, characterized in that, The step of parsing multiple semantic entities in the input text and the spatial relationships between the semantic entities using a large language model to obtain the layout information corresponding to the input text includes: The input text is reasoned through a large language model to obtain a scene relationship graph, wherein the scene relationship graph represents multiple semantic entities in the input text and the spatial relationships between the semantic entities in a structured form; The layout information corresponding to the input text is obtained by performing layout reasoning on the input text and the scene relationship graph using the large language model.

3. The text-to-image generation method as described in claim 1, characterized in that, Before the step of parsing multiple semantic entities in the input text and the spatial relationships between the semantic entities using a large language model to obtain the layout information corresponding to the input text, the method further includes: Obtain a hybrid mind chain dataset, wherein the hybrid mind chain dataset includes image layout data from real-world scenarios and predicted layout data generated by an initial large language model; The initial large language model is fine-tuned based on the hybrid thought chain dataset to obtain the large language model.

4. The text-to-image generation method as described in claim 1, characterized in that, The step of performing the self-loop semantic correction to update the text embedding vector includes: In multiple time steps of the denoising process, the first cross-attention map is Gaussian smoothed to obtain a smoothed cross-attention map; The text loss is calculated based on the smoothed cross-attention map, and the text loss is used to balance the attention of the diffusion model to different text words in the input text. The text embedding vector in the denoising process is updated based on the text loss.

5. A text-to-image generation apparatus, characterized in that, The device includes: The layout information acquisition module is used to receive input text, parse multiple semantic entities in the input text and the spatial relationships between the semantic entities through a large language model, and obtain the layout information corresponding to the input text, wherein the layout information includes the position range of each semantic entity in the image; The layout information conversion module is used to convert the layout information into a mask set corresponding to the size of the latent space feature map of the diffusion model; An image generation module is used to input the mask set and the input text into the diffusion model, and to perform semantic correction based on the mask set during the denoising process of the input text by the diffusion model to obtain the target image; The semantic correction includes iterative semantic correction and / or self-looping semantic correction; The image generation module is also used for: During the denoising process of the input text by the diffusion model, the iterative semantic correction is performed based on the mask set to update the image latent variables, and / or the self-looping semantic correction is performed to update the text embedding vector; Based on the updated image latent variables and / or the updated text embedding vector, the input text is denoised using the diffusion model to obtain the target image; The image generation module is also used for: In multiple time steps of the denoising process, a cross-attention loss is calculated based on the mask set and the first cross-attention map generated by the diffusion model. The cross-attention loss is used to constrain the text lexical attention of each semantic entity to focus on the corresponding mask region. Based on the self-attention map generated by the mask set and the diffusion model, a self-attention loss is calculated, which is used to enhance the pixel correlation within the mask region corresponding to each semantic entity. The total latent variable loss is obtained by combining the cross-attention loss with the self-attention loss. The image latent variables are updated during the denoising process based on the total loss of the latent variables.

6. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the text-to-image generation method as claimed in any one of claims 1 to 4.

7. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the text-to-image generation method as described in any one of claims 1 to 4.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the text-to-image generation method as described in any one of claims 1 to 4.

Images