A method and system for generating images from text based on local detail editing
By dividing text into independent object attribute descriptions and using generative adversarial networks and feature localization modules to generate feature maps and attention maps, the problem of difficulty in handling complex text descriptions in existing methods is solved, and the controllability and diversity of image generation details are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PEKING UNIV
- Filing Date
- 2022-12-23
- Publication Date
- 2026-05-12
AI Technical Summary
Existing methods struggle to handle highly complex and flexible natural language descriptions, making it difficult for models to understand and encode multiple object attributes in text descriptions as features, and failing to achieve consistency between fine-grained text descriptions and semantics of local image regions.
The input text is divided into several independent object attribute descriptions. An initial image is generated by a generative adversarial network. Feature maps and attention maps are generated using a feature localization module and an attention mechanism. Image details are controlled by shape loss, attention loss and discriminator loss, enabling controllable editing of local details.
It enables the modeling of the attributes of generated objects based on text descriptions, which can generate higher quality and more diverse images, improving the controllability and diversity of image generation details.
Smart Images
Figure CN116245967B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image generation, and more specifically to a method and system for generating images from text based on local detail editing. Background Technology
[0002] Text-to-image generation aims to generate semantically consistent and content-realistic images based on a given natural language description. In recent years, with the rapid development of generative adversarial networks (GANs), existing methods have made significant progress in synthesizing content-realistic images. Furthermore, due to the broad application prospects of text-to-image generation, such as visual reading, graphic design, and criminal investigation, this field has become one of the most active research areas in recent years.
[0003] Text-to-image generation technology faces two main research challenges: ensuring semantic consistency between images and text, and generating high-resolution, realistic images. To ensure semantic consistency, existing methods typically employ text encoders and image encoders to learn cross-modal representations of text and images. The main idea is to simultaneously train a fixed-length text encoder and an image encoder using text-image pairs, thereby uncovering the semantic relationships within these pairs. In recent years, a series of models have adopted the StackGAN architecture (Zhang Han, et al. Stackgan: Text to photo-realistic imagesynthesis with stacked generative adversarial networks. Proceedings of the IEEE international conference on computer vision. 2017) to generate images. These models learn cross-modal representations of text and images through pre-trained LSTM (Long Short Term Memory) text encoders and CNN (Convolutional Neural Network) image encoders to address the semantic consistency problem. AttnGAN uses the DAMSM (Deep Attentional Multimodal Similarity Model) model (Xu Tao, et al. AttnGAN: Fine-grained text to image generation with attentional generative adversarial networks. Proceedings of the IEEE conference on computer vision and pattern recognition. 2018). By introducing an attention mechanism, it searches for the correlation between local regions of an image and words in the text in a semi-supervised manner, so that the details of the image are consistent with the text description.
[0004] However, the fixed-length encoding used in existing methods struggles to handle highly complex and flexible natural language descriptions. This results in models that typically perform well with single object attributes but struggle to understand and encode multiple object attributes within a textual description as features. Consequently, existing methods fail to meet users' requirements for generated objects in terms of detail, namely, maintaining consistency between fine-grained textual descriptions and the semantics of local image regions. Summary of the Invention
[0005] This invention proposes a text-to-image generation method based on local detail editing, which can edit local details of an image according to the input fine-grained text description, thereby achieving automatic generation of images with controllable details.
[0006] The principle of the text-to-image generation method based on local detail editing proposed in this invention is as follows: first, the text is divided into several independent object attribute descriptions and an initial image is generated; then, the local details of the initial image are modified according to these descriptions, thereby achieving automatic generation of images with controllable details.
[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0008] A method for generating images from text based on local detail editing includes the following steps:
[0009] (1) The input text is divided into several independent object attribute descriptions by the parser;
[0010] (2) Generate an initial image through a generative adversarial network and map the initial image into the latent space of the generative adversarial network;
[0011] (3) The feature localization module finds the corresponding region of the object attribute description in step (1) in the initial image in step (2), and generates the corresponding attention map and feature map based on the attention mechanism;
[0012] (4) Modify the latent space vector from step (2) based on the feature map and attention map, and re-feed it into the generative adversarial network to obtain an image that conforms to the fine-grained description of the text.
[0013] Furthermore, in the above method, the parser in step (1) consists of a text chunking model and a syntax tree parser. Since noun phrases in the text are often the core part of the object attribute description, the parser first uses the text chunking model to divide the text into non-overlapping phrases, thereby obtaining the noun phrases in the text description. After obtaining the noun phrases, the syntax tree parser is used to obtain the grammatical structure of the input text, merging the core noun phrases in the sentence with its neighboring verb phrases, prepositional phrases, noun clauses, etc., to form independent object attribute descriptions.
[0014] Furthermore, in the above method, the generative adversarial network in step (2) is a stacked generative adversarial network. First, the input object attributes are encoded as features, and then an initial image is generated through a two-stage generative network. The method for mapping the initial image x to the latent space of the generative adversarial network is as follows: first, a vector t is randomly sampled in the latent space of the generative adversarial network, and then the following mapping loss function L is used. proj Modify the vector:
[0015] L proj=||F(x)-F(G(t))||2
[0016] Where G(.) represents the Generative Adversarial Network (GAN), and F(.) represents the feature extraction model. This loss function aims to make the image generated by the GAN based on t as close as possible to the original image x, thereby mapping x to the latent space of the GAN.
[0017] Furthermore, in the above method, the feature localization method in step (3) first calculates the attention map based on the similarity matrix of local image features and text description. To eliminate the influence of overlapping regions in the attention map, it is assumed that each independent object attribute should focus on different regions of the object. Therefore, the feature localization module designs a deduplication algorithm to eliminate overlapping regions in the attention map. This algorithm first filters out attention maps with fewer overlapping regions with other attention maps, and then merges the remaining attention maps by merging the maximum values. Obtain the global attention map Finally, the overlapping areas are removed using the following formula:
[0018]
[0019] in Attention map Median less than The region is represented by ∩, which indicates the overlapping area of two attention maps.
[0020] Furthermore, in the above method, step (4) designs shape loss, attention loss, and discriminator loss to control the shape, local features, and texture details of the object, thereby achieving controllable editing of image details.
[0021] Furthermore, in the above method, the shape loss L in step (4) s Attention deficit L a Discriminator loss L d The definition is as follows:
[0022] L s =||F(x)-F(t)||2
[0023] L a =Σ||F(x·mask) i )-F(t i ·mask i )||2
[0024] L d =softplus(-D(x))
[0025] Where F(.) represents the feature extraction model, x represents the image being edited, and t represents the target image for detail editing. i and maski Let represent the feature map and attention map, respectively. D is the discriminator of the generative adversarial network. The Softplus function can be expressed as:
[0026] softplus(x) = log(1 + e) x )
[0027] Furthermore, in the above method, in step (4) during the first stage, the target image t is set as a reference image stitched together from the feature map and the attention map:
[0028]
[0029] In this stage, the loss function only uses shape loss to correct the shape of the object.
[0030] Furthermore, in the above method, in step (4) during the second stage, the target image t remains unchanged, and the loss function incorporates the attention loss L. a To correct local image features, the overall pose of the generated object remains unchanged during the process of correcting local image features. This stage uses both shape loss and attention loss in the loss function.
[0031] Furthermore, in the above method, in step (4) of the third stage, the target image t is updated to the image modified in the last round of the previous stage. The goal of this stage is to add image detail and texture to make the image more realistic, while the image content does not change too much. Therefore, the attention loss is removed from the loss function, and the discriminator loss is added.
[0032] Corresponding to the above method, the present invention also provides a text-to-image generation system based on local detail editing, which includes:
[0033] The syntax analysis module is responsible for dividing the input text into independent object attribute descriptions;
[0034] The image generation module is responsible for feeding the object attribute description into the text encoder and the generative adversarial network, generating an initial image, and mapping the initial image into the network's latent space.
[0035] The feature localization module is responsible for finding the corresponding regions in the initial image where the object's attribute descriptions are located, and generating feature maps and attention maps based on the attention mechanism.
[0036] The local detail editing module is responsible for correcting local areas of the initial image based on feature maps and attention maps to make them conform to the fine-grained description of the text.
[0037] The advantages of this invention are as follows: Compared with existing methods, this method can model the attributes of the generated object based on the input text description, and edit local regions of the generated image based on these attributes, thereby achieving automatic image generation with controllable details. This method can generate object details better, thus producing higher quality and more diverse images.
[0038] The reason this method achieves the aforementioned effects is that it utilizes attribute modeling to extract fine-grained information from the input text. These attribute descriptions are then fed separately into the text encoder, avoiding the loss of fine-grained information that would result from directly encoding highly complex and flexible natural language text. Secondly, by generating attention maps and feature maps of object attributes through feature localization, this method enables controllable editing of image details, resulting in images with richer and more diverse details. Attached Figure Description
[0039] Figure 1 This is a flowchart of the text-to-image generation method based on local detail editing of the present invention.
[0040] Figure 2 This is the network structure diagram of the present invention. Detailed Implementation
[0041] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0042] The text-to-image generation method based on local detail editing of the present invention has the following process: Figure 1 As shown, it includes the following steps:
[0043] (1) The input text is divided into several independent object attribute descriptions by the parser.
[0044] The parser comprises a text chunking model and a syntax tree parser. Observing that noun phrases in text are often the core of object attribute descriptions, the parser first uses the text chunking model to divide the text into non-overlapping phrases, thus obtaining the noun phrases in the text description. After obtaining the noun phrases, the syntax tree parser uses the syntax tree parser to obtain the grammatical structure of the input text, merging the core noun phrases in the sentence with their neighboring verb phrases, prepositional phrases, noun clauses, etc., to form independent object attribute descriptions. The merging algorithm specifically works as follows: a sentence component is defined as a criterion for independent attribute descriptions (noun phrases containing adjectives). The syntax tree is traversed from bottom to top. When a node is reached, if it meets the criterion, it is added to the partitioning result; otherwise, the node is merged with its sibling nodes until the criterion is met.
[0045] (2) Generate an initial image through a generative adversarial network and map the initial image into the latent space of the generative adversarial network.
[0046] The specific steps for generating the initial image are as follows: First, an LSTM text encoder is used to encode the input object attributes into features. Then, a stacked generative adversarial network (GAN) is used to generate the initial image in two stages. The method for mapping image x to the latent space of the GAN is as follows: First, a vector t is randomly sampled from the StyleGAN latent space. Then, the loss function L is minimized using gradient descent. proj :
[0047] L proj =||F(x)-F(G(t))||2
[0048] Where G(.) represents the Generative Adversarial Network (GAN), and F(.) represents the feature extraction model. This loss function aims to make the image generated by the GAN based on t as close as possible to the original image x, thereby mapping x to the latent space of the GAN.
[0049] (3) The corresponding region of the object attribute description in the image is found through the feature localization module, and the corresponding attention map and feature map are generated based on the attention mechanism.
[0050] The method for generating attention maps is as follows: First, the attention map is calculated based on the similarity matrix of local image features and text descriptions. To eliminate the influence of overlapping regions in the attention maps, it is assumed that each independent object attribute should focus on different regions of the object. Therefore, the feature localization module designs a deduplication algorithm to eliminate overlapping regions in the attention maps. This algorithm first selects attention maps with fewer overlapping regions with other attention maps, and then merges the remaining attention maps by merging the maximum values. Obtain the global attention map Finally, the overlapping areas are removed using the following formula:
[0051]
[0052] in Attention map Median less than The region is represented by ∩, which indicates the overlapping area of two attention maps.
[0053] In one embodiment of the present invention, the deduplication algorithm specifically includes:
[0054] For the i-th attention map, its saliency region is represented as a. i The deduplication algorithm first calculates the global attention region A in the first i-1 images. i-1 :
[0055] A i-1 =a1∪a2∪…∪a i-1
[0056] Here, ∪ represents taking the larger value at each point in the two attention maps.
[0057] Then, attention maps with minimal overlap with other attention maps are selected. Specifically, attention map i is ignored if it meets the following conditions:
[0058]
[0059] Where S(.) refers to the area of the salient region in the attention map. i >A i-1 In the attention graph, a i Median greater than A i-1 The region is defined by ∩, which represents the overlapping area of two attention maps. The first formula requires that the attention region should have distinctive local features, where α is the discrimination threshold. The second formula requires that the attention region should not overlap too much with other regions, where β is the overlap ratio. For the remaining attention maps... and the corrected global attention region Finally, the overlapping areas are removed using the following formula:
[0060]
[0061] in, Attention map Median less than The area.
[0062] (4) Modify the latent space vectors based on the feature map and attention map to make the modified image conform to the fine-grained description of the text. Design shape loss, attention loss and discriminator loss in three stages to control the shape, local features and texture details of the object, thereby realizing the editing of local details of the image.
[0063] The editing process for local details is as follows: Figure 2 As shown. Shape loss L s Attention deficit L a Discriminator loss L d The definition is as follows:
[0064] L s =||F(x)-F(t)||2
[0065] L a =Σ||F(x·mask) i )-F(t i ·mask i )||2
[0066] L d =softplus(-D(x))
[0067] Where F(.) represents the feature extraction model, x represents the image being edited, and t represents the target image for detail editing. i and mask i Let represent the feature map and attention map, respectively. D is the discriminator of the generative adversarial network. The Softplus function can be expressed as:
[0068] softplus(x) = log(1 + e) x )
[0069] In the first stage, the target image t is set as a "reference image" stitched together from feature maps and attention maps. It contains all the features of the text description, but they are inconsistent with each other. Considering that this stage only modifies the shape of the generated object, this method proposes to address this issue in later stages by correcting local image features. The loss function L in this stage can be expressed as L = L s .
[0070] In the second stage, the target image t remains unchanged, and the loss function incorporates an attention loss L. a This stage corrects local image features. To ensure the overall pose of the object remains unchanged during the correction process, the loss function in this stage uses both shape loss and attention loss. The loss function L in this stage can be expressed as L = L s +λ a L a , where λ a It is a hyperparameter used to balance shape loss and attention loss.
[0071] In the third stage, the target image t is updated to the image modified in the previous stage. This design is because the goal of this stage is to modify the image's detail texture to make it more realistic, while keeping the image content largely unchanged. Therefore, the attention loss is removed from the loss function, and a discriminator loss is added. The loss function L in this stage can be expressed as L = L s +λ d L d , where λ d It is a hyperparameter used to balance shape loss and attention loss.
[0072] The experimental results below demonstrate that, compared with existing methods, the text-to-image generation method based on local detail editing of this invention can generate higher quality and more diverse images.
[0073] This embodiment uses the Caltech-UCSD Birds-200-2011 (CUB) and Oxford-Flower-102 (Oxford102) datasets. The CUB dataset, proposed in the paper "The caltech-ucsd birds-200-2011dataset" (authors Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie), is a widely used bird image dataset for evaluating text-to-image generation tasks. The dataset contains a total of 11,788 images of 200 bird species, with 8,855 images from 150 categories used for training and 2,933 images from 50 categories used for testing. The partitioning of the training and testing sets and the text used in this dataset are based on StackGAN (Zhang, Han, et al. "StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks." Proceedings of the IEEE international conference on computer vision. 2017.). Each image in the dataset has ten text descriptions.
[0074] The Oxford102 dataset, proposed in the paper "Automated flower classification over a large number of classes" (by Maria-Elena Nilsback and Andrew Zisserman), consists of flower images. The dataset contains a total of 8189 images across 102 categories, with 6149 images from 82 categories used for training and 2040 images from 20 categories used for testing. The splitting of the training and testing sets, as well as the accompanying text, were also proposed using StackGAN. Each image in the dataset has ten text descriptions.
[0075] Existing Method 1: The StackGAN-v2 method in the paper "Stackgan++: Realistic image synthesis with stacked generative adversarial networks" (authors Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris N. Metaxas).
[0076] Existing Method 2: The DM-GAN method in the paper "Dm-gan: Dynamic memory generative adversarial networks for text-to-image synthesis" (authors Minfeng Zhu, Pingbo Pan, Wei Chen, and Yi Yang).
[0077] Existing Method 3: The LAFITE method in the paper "LAFITE: Towards Language-Free Training for Text-to-Image Generation" (authors Yufan Zhou, Ruiyi Zhang, Changyou Chen, Chunyuan Li, ChrisTensmeyer, Tong Yu, Jiuxiang Gu, Jinhui Xu and Tong Sun).
[0078] This invention: The method of this embodiment.
[0079] The experiments used the Inception Score (IS) and Frechet Inception Distance (FID) metrics to evaluate the quality of the generated images. These two metrics measure the diversity and quality of the generated images, respectively. The IS metric was proposed in the paper "Improved techniques for training gans" (authors Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, and Xi Chen). This metric first fine-tunes the Inception-v3 classification network trained on ImageNet on the test set of the dataset, and then uses the Inception-v3 classification network to predict class labels for both real and synthetic images. The KL divergence between the conditional class distribution of the real labels and the marginal class distribution of the predicted labels is calculated. A high IS value indicates that the generated images have high diversity, meaning that the number of images in each class is similar, and each image clearly belongs to one class.
[0080] The FID metric was proposed in the paper "Gans trained by a two time-scale update rule converge to a local nash equilibrium" (Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter). This metric uses an Inception-v3 classification network trained on ImageNet to extract features from both real and synthetic images, then calculates the average distance between them. A lower FID indicates higher quality generated images, meaning the generated images are closer to the real images. Experiments evaluated the IS and FID results of the above method on the CUB and Oxford102 datasets, as shown in Table 1.
[0081] Table 1. Comparative experimental results of the present invention
[0082]
[0083] As shown in Table 1, this invention significantly improves the quality and diversity of generated images. None of the three existing methods model the object attributes of the input text, nor can they achieve controllable image detail editing. This invention mines fine-grained information from the text through attribute modeling and edits object attributes using feature localization and local feature modification modules, enriching the details of the generated images and thus producing higher quality and more diverse images.
[0084] In the above embodiments, the method for extracting text features in step (2) of the present invention uses an LSTM (Long-Short-Term Memory) neural network. In other embodiments, a CLIP pre-trained model and a recurrent neural network (RNN) can also be used as alternatives.
[0085] Another embodiment of the present invention provides a text-to-image generation system based on local detail editing, comprising:
[0086] The syntax analysis module is responsible for dividing the input text into independent object attribute descriptions;
[0087] The image generation module is responsible for feeding the object attribute description into the text encoder and the generative adversarial network, generating an initial image, and mapping the initial image into the network's latent space.
[0088] The feature localization module is responsible for finding the corresponding regions in the initial image where the object's attribute descriptions are located, and generating feature maps and attention maps based on the attention mechanism.
[0089] The local detail editing module is responsible for correcting local areas of the initial image based on feature maps and attention maps to make them conform to the fine-grained description of the text.
[0090] For the specific implementation process of each module, please refer to the description of the method of the present invention above.
[0091] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps of the method of the present invention.
[0092] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) storing a computer program that, when executed by a computer, implements the various steps of the method of the present invention.
[0093] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. A method for generating images from text based on local detail editing, comprising the following steps: The parser divides the input text into several independent object attribute descriptions; An initial image is generated by a generative adversarial network (GAN), and the initial image is mapped into the latent space of the GAN. The feature localization module finds the corresponding region in the initial image that describes the object's attributes, and generates the corresponding attention map and feature map based on the attention mechanism. The latent space vector is modified based on the feature map and attention map, and then fed back into the generative adversarial network to obtain an image that conforms to the fine-grained description of the text. The feature localization module employs a deduplication algorithm to eliminate overlapping regions in the attention maps. This algorithm first selects attention maps with minimal overlap with other attention maps, and then merges the remaining attention maps using a maximum value merging method. Obtain the global attention map Finally, the overlapping areas are removed using the following formula: in Attention map Median less than The region is represented by ∩, which indicates the overlapping area of two attention maps.
2. The method as described in claim 1, characterized in that, The parser consists of a text chunking model and a syntax tree parser. The parser first uses the text chunking model to divide the text into non-overlapping phrases, thus obtaining noun phrases in the text description. After obtaining the noun phrases, the syntax tree parser obtains the grammatical structure of the input text, merging the core noun phrases in the sentence with their neighboring verb phrases, prepositional phrases, and noun clauses to form independent object attribute descriptions. The merging algorithm is as follows: First, a criterion is defined for a sentence component to be an independent attribute description. Then, the syntax tree is traversed from bottom to top. When a node is reached, if it meets the criterion, it is added to the partitioning result; otherwise, the node is merged with its sibling nodes until the criterion is met.
3. The method as described in claim 1, characterized in that, The process of mapping the initial image to the latent space of the generative adversarial network includes: firstly, randomly sampling a vector t in the latent space of the generative adversarial network, and then using the following mapping loss function L. proj Modify the vector: L proj =||F(x)-F(G(t))||2 Where x represents the initial image, G(.) represents the generative adversarial network, and F(.) represents the feature extraction model; the loss function makes the image generated by the generative adversarial network based on t as close as possible to the initial image x, thereby mapping x to its latent space.
4. The method as described in claim 1, characterized in that, The process of selecting attention maps with minimal overlap with other attention maps includes: For the i-th attention map, its saliency region is represented as a. i The deduplication algorithm first calculates the global attention region A in the first i-1 images. i-1 : A i-1 =a1∪a2∪...∪a i-1 Where ∪ represents taking the larger value at each point in the two attention maps; Then, attention maps with less overlap with other attention maps are selected. Attention map i is ignored when it meets the following conditions: Where S(.) refers to the area of the salient region in the attention map, a i >A i-1 In the attention graph, a i Median greater than A i-1 The region is defined by ∩, which represents the overlapping area of two attention maps. The first formula requires that the attention region should have distinctive local features, where α is the discrimination threshold. The second formula requires that the attention region should not overlap with other regions too much, where β is the overlap ratio.
5. The method as described in claim 1, characterized in that, The process of modifying the latent space vector based on the feature map and attention map, and then re-feeding it into the generative adversarial network to obtain an image that conforms to the fine-grained description of the text, includes: designing shape loss, attention loss, and discriminator loss to control the shape, local features, and texture details of the object, thereby achieving controllable editing of image details; the shape loss function L... s Attention loss function L a Discriminator loss function L d The definition is as follows: L s =||F(x)-F(t)||2 L a =∑||F(x·mask i )-F(t i ·mask i )||2 L d =softplus(-D(x)) Where F(.) represents the feature extraction model, x represents the image being edited, and t represents the target image for detail editing. i and mask i Let represent the feature map and attention map, respectively. D is the discriminator of the generative adversarial network. The Softplus function is defined as follows: softplus(x)=log(1+e x )。 6. The method as described in claim 5, characterized in that, The controllable editing of the image details is divided into three stages: In the first stage, the target image t is set as a reference image stitched together from the attention module feature map and the attention map, which contains features of all attributes in the text description; The loss function uses shape loss L s The shape of the object being corrected is expressed by the loss function L as L = L s ; In the second stage, the target image t remains unchanged, and the loss function incorporates an attention loss L. a To correct local image features, the loss function L is expressed as L = L s +λ a L a , where λ a It is a hyperparameter used to balance shape loss and attention loss; In the third stage, the target image t is updated to the image modified in the last round of the second stage. By introducing a discriminator loss, the image details and textures are modified to make the image more realistic. The loss function L is expressed as L = L s +λ d L d , where λ d It is a hyperparameter used to balance shape loss and attention loss.
7. A text-to-image generation system based on local detail editing, employing the method described in any one of claims 1 to 6, characterized in that, include: The syntax analysis module is responsible for dividing the input text into several independent object attribute descriptions; The image generation module is responsible for feeding the object attribute description into the text encoder and the generative adversarial network, generating an initial image, and mapping the initial image into the network's latent space. The feature localization module is responsible for finding the corresponding regions in the initial image where the object's attribute descriptions are located, and generating feature maps and attention maps based on the attention mechanism. The local detail editing module is responsible for correcting local areas of the initial image based on feature maps and attention maps to make them conform to the fine-grained description of the text.
8. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 6.