Text image generation method based on semantic alignment and text fusion

Abstract

The invention discloses a text image generation method based on semantic alignment and text fusion, which comprises the following steps of: respectively extracting semantic features of a real image and a generated image, comparing and adjusting high-level semantics of the real image and the generated image through a semantic alignment module in a discriminator, and updating parameters of a generator according to an output result of the discriminator. It is ensured that the generated image is consistent with a real image in the semantic level and detail processing, the defect of text description is overcome, and therefore the performance of the whole generation model is improved; meanwhile, subtle differences implied in text description can be more comprehensively captured through a star module convolutional layer, so that the detail expressive force and the generation quality of the image are remarkably enhanced; besides, deformable convolution is added to the generator, complex detail information is effectively reserved, the generated image is more real visually, and adaptability and consistency of the model in different situations and styles are enhanced.

Description

Technical Field

[0001] The invention belongs to the technical field of image generation, and in particular relates to a text-to-image generation method based on semantic alignment and text fusion. Background Art

[0002] Compared with language descriptions, images often contain more and more complex information, but images can convey information more intuitively. In addition, because humans are more exposed to image content, image information is often easier to remember than text information. Images are a good medium for processing information, and the difficulty of processing image content is much greater than that of simple text information. Therefore, if computers can understand the complex relationship between text and images well, they can take into account the advantages of both images and text when processing information, and can easily obtain a large number of high-resolution images that conform to the semantics of the text.

[0003] Generating images from text is a technology that generates images that fit the semantics of the text by inputting text descriptions and noise information. It can also generate a series of images that are semantically consistent but not exactly the same by adjusting the input noise information. There are two main difficulties in generating images from text: understanding the semantics of the text and ensuring the consistency of the generated image with the text, and generating high-quality images with details under fixed semantic content conditions. Existing methods based on generative adversarial networks (GANs) still have problems such as the lack of realism and details in the generated images, and the inability to ensure the consistency of the semantics of the given text and the generated image.

[0004] In DFGAN (Deep Fusion Generative Adversarial Network), sentence information is effectively integrated into visual features by connecting multiple deep text-image fusion blocks in series. This is a great improvement in improving text semantic consistency. In addition, the rapid development of multi-modal large models has significantly improved the capabilities of artificial intelligence systems in vision and text understanding. In particular, models such as CLIP (Contrastive Language-Image Pre-training) have significantly improved multimodal alignment capabilities with their large-scale data training and contrastive learning framework. CLIP enables the model to build high-quality semantic mappings between images and texts by performing contrastive learning on large-scale image-text pairs, providing a strong foundation for downstream tasks, especially in visual and language understanding tasks. This is very helpful in improving text semantic consistency. In the latest research, GALIP (Generative Adversarial Network CLIPs Model) uses the pre-trained text encoder and image encoder of CLIP, and uses CLIP's ability to understand images and align text semantics to help generate semantically aligned images, and generates more realistic images through CLIP's ViT.

[0005] In the semantic-aware discriminator (SeD) for generating super-resolution images, semantics are extracted from real images during training to assist the training of the generated image process, making the generated image more realistic and with more detailed expressions. Traditional text-generated image models usually rely solely on the discriminator to determine whether the generated image conforms to the semantics of the text description. However, due to the low information density of the text description, many image details are difficult to be fully covered by the text description, resulting in missing or messy details in the generated image. When humans draw images, they often refer to the appearance features of similar objects to enrich the details. Therefore, introducing real images as a basis for judgment in the design of the discriminator can provide a more comprehensive reference framework. However, achieving semantic alignment between generated images and real images is a core technical challenge. The most direct method is to judge the similarity between the generated image and the real image by pixel-level comparison. However, this method has significant limitations: the generated model may overfit the detailed features of the real image, thereby generating results that are highly similar to the real image, which is easily regarded as plagiarism in practical applications. Summary of the invention

[0006] The purpose of the present invention is to propose a text-to-image method based on semantic alignment and text fusion to generate more realistic and detailed images.

[0007] The present invention proposes a text-to-image method based on semantic alignment and text fusion, which comprises the following steps: Step 1: Obtain text and corresponding images to construct a data set; use the image corresponding to the text as the real image of the text; Step 2: construct a text-generated image model; the text-generated image includes a generator and a discriminator; the generator is used to generate an image that fits the semantics of the text according to the text; the discriminator includes a second visual encoder, a feature extraction module, a semantic alignment module and an image evaluation module; the second visual encoder is used to extract the tested image generated by the generator With the real image X real Visual features of the image being tested and real image features ; The feature extraction module extracts and enhances the visual features obtained by the second visual encoder layer by layer to obtain the real image X real The corresponding visual summary feature f r-end and the image under test Corresponding visual summary features ; The semantic alignment module is used to align the features of the image under test and real image features Compare and correct the high-level semantic features of the se ; The image evaluation module semantically aligns the features fse , text features T and visual summary features f r-end and The concatenated features are used as input to obtain identification results; The semantic alignment module comprises a plurality of semantic alignment blocks connected in sequence; the plurality of semantic alignment blocks have the same structure, and all comprise a first mapping convolution layer, a spatial conversion layer, and a second convolution layer connected in sequence; each semantic alignment block sequentially maps features through the first mapping convolution layer and the spatial conversion layer and Features Fusion, obtain fusion alignment semantics se j ; The second convolutional layer in the first semantic alignment block receives the output of the corresponding spatial conversion layer as input; the second convolutional layers in the remaining semantic alignment blocks all receive the concatenated features of the output of the corresponding spatial conversion layer and the previous semantic alignment block as input; Step 3: Input the dataset into the text-to-image generation model for training, and update the generator according to the output of the discriminator; use the trained text-to-image generation model to generate an image that fits the semantics of the text based on the unlabeled text.

[0008] Preferably, the number of semantic alignment blocks in the semantic alignment module is three.

[0009] Preferably, the generator includes a text encoder, a Reshape layer, a text feature fusion module, a star module convolution layer, a first visual encoder and a picture production block; the text encoder is used to obtain a text embedding vector T according to the input text; the Reshape layer is used to obtain the initial features according to the random noise vector Z; the text feature fusion module is used to perform affine transformation and nonlinear activation processing on the initial features according to the text embedding vector T to generate an intermediate feature f; the star module convolution layer is used to process the output of the text feature fusion module to obtain the intermediate feature ; The first visual encoder is used to extract intermediate features Global and local information of ; The image generation block is used to generate the intermediate feature f * and the encoding feature f v The spliced ​​fused feature map is upsampled to obtain the generated image.

[0010] Preferably, the text feature fusion module comprises six text feature fusion layers connected in sequence; the six text feature fusion layers have the same structure, and all comprise four affine blocks and convolution layers connected in sequence.

[0011] Preferably, the picture generation block includes six layers of upsampling generation blocks connected in sequence; the first three layers of upsampling generation blocks have the same structure, both including a series-connected upsampling layer, a first convolutional layer, a first affine block, a second convolutional layer, and a second affine block; the last three layers of upsampling generation blocks have the same structure, both including a series-connected upsampling layer, a convolutional layer, a first affine block, a deformable convolutional layer, and a second affine block.

[0012] Preferably, the text encoder, the first visual encoder and the second visual encoder each include twelve layers of Transformer modules connected in sequence.

[0013] Preferably, the first visual encoder and the second visual encoder have the same structure and both adopt the VisionTransformer architecture.

[0014] Preferably, the feature extraction module comprises three feature extraction blocks connected in sequence; the input of the first feature extraction block is the output of the second layer Transformer module in the second visual encoder; the input of the second and third feature extraction blocks are the features obtained by combining the output of the previous feature extraction block with the output of the fifth and ninth layer Transformer modules in the second visual encoder, respectively.

[0015] Preferably, the feature extraction blocks have the same structure, and both include a first convolutional layer, a first ReLU function, a second convolutional layer, and a second RuLU function connected in sequence.

[0016] Preferably, the image evaluation module comprises two image evaluation layers connected in sequence; the two image evaluation layers have the same structure, both comprising two convolutional layers connected in sequence and a ReLU activation function.

[0017] The present invention has the following beneficial effects: 1. The present invention extracts semantic features of real images and generated images respectively, compares and adjusts the high-level semantics of the two through a semantic alignment module, and effectively improves the robustness of the discriminator, enabling it to more sensitively capture the difference between the generated image and the real semantic image, thereby more accurately identifying the authenticity and quality of the generated results, ensuring that the generated image is consistent with the real image in terms of semantic level and detail processing, making up for the shortcomings of text description, and thus improving the performance of the entire generation model.

[0018] 2. The present invention can more comprehensively capture the subtle differences implicit in text descriptions through the star module convolution layer, and map these details to the generated image, thereby improving the limitation of traditional convolution that cannot efficiently capture complex nonlinear relationships, and enhancing the ability of fine detail processing and complex spatial reasoning, so that the generator can better express the rich texture and shape changes in the image, and the generated image achieves a balance between global consistency and local details, thereby significantly enhancing the image's detail expression and generation quality.

[0019] 3. The present invention uses deformable convolution in the generator. Each convolution kernel can adaptively adjust the sampling position on the feature map, accurately capture and reconstruct the subtle structure and spatial information described by the input text, so as to generate a more detailed image, effectively retain the complex detail information, and make the generated image more visually realistic. At the same time, deformable convolution provides the generator with extensive support for diverse features, which can not only better match the semantic requirements of the input text, but also enhance the adaptability and consistency of the model in different situations and styles. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a structural architecture diagram of the text-to-image model in the present invention.

[0021] Figure 2 This is a comparison chart of the effects of the present invention and the GALIP model on the CUB dataset.

[0022] Figure 3 This is a comparison chart of the effects of the present invention and the GALIP model on the COCO dataset. DETAILED DESCRIPTION

[0023] The present invention will be further described below in conjunction with the accompanying drawings.

[0024] like Figure 1 As shown, a text generation image method based on semantic alignment and text fusion includes the following steps: Step 1: Get the dataset The acquired datasets include the Caltech-UCSD Birds 200-2011 (CUB) dataset, the COCO dataset, and English sentences corresponding to the image semantics in the CUB dataset and the COCO dataset; CUB is a dataset with only bird pictures, with a total of 200 bird images, of which the training set contains 8,855 images of 150 species and the test set contains 2,933 images of 50 species. Each image corresponds to 10 English sentences with the same semantics but different expressions; the COCO dataset has richer image types and more complex scenes, with a total of 171 image categories, of which the training set contains 80k images and the test set contains 40k images. Each image corresponds to 5 English sentences with the same semantics but different expressions.

[0025] Step 2: Build a text-to-image model Text-generated images include a generator and a discriminator; the generator is used to generate a generated image corresponding to the input text. The image generated by the generator module can more accurately process the spatial details and local structures in the generated image, while further improving the semantic consistency between the image and the text. The discriminator is used to distinguish the generated image from the real image.

[0026] 2-1. The generator includes a text encoder, a Reshape layer, a text feature fusion module, a star module convolution layer (starconv), a first visual encoder, and an image generation block. The text encoder includes twelve layers of Transformer modules connected in sequence, which obtain the text embedding vector T according to the input word-segmented text sequence; the Transformer module encodes the relationship between different words through the self-attention mechanism. All words in the text sequence are encoded as , the encoding of the text sequence is ; Where L represents the number of words in the sentence; D represents the dimension; the output of the text encoder is the EOS tag vector of the last layer of the Transformer module, which represents the embedding of the entire text sequence. The Reshape layer is used to reshape the noise vector Z into a data format with a specified image size to obtain the initial features, laying the foundation for the subsequent text semantic fusion and image generation process.

[0027] The text feature fusion module is used to perform affine transformation and nonlinear activation processing on the initial features output by the Reshape layer according to the text embedding vector T, and generate intermediate features f that are highly related to the text semantics; the text feature fusion module includes six text feature fusion layers connected in sequence; the six text feature fusion layers have the same structure, which includes four affine blocks and convolution layers connected in sequence; in each affine block, the text vector T is used as a conditional input, and the scale parameters are generated through the fully connected network. and bias parameters These parameters are dynamically adjusted according to the text semantics, reflecting the conditional guidance of image features. Input of the affine block Adjust according to the affine formula:

[0028] Formula (1) Among them, X * is the output of the affine block.

[0029] The image features are adjusted pixel by pixel through the text feature fusion module. Each text feature fusion layer gradually refines the features through conditional affine transformation and nonlinear activation, which can effectively capture the complexity of text semantics and integrate it into the image features to make it more consistent with the text semantics.

[0030] The design principle of the star module convolution layer is to deeply analyze the multi-level semantic features described by the input text through specific convolution operations, so that the model can capture detailed visual information at different scales. The star module convolution layer uses two deep convolutions to process the features output by the text feature fusion module, which can efficiently capture local patterns in the features and retain spatial structural information while reducing the number of parameters and computational costs. The multi-layer nonlinear transformation and channel interaction mechanism of the star module convolution layer can more efficiently capture the deep relationship between multimodal features, improve the expressiveness of features, and thus generate intermediate features with richer semantic expressions and stronger nonlinear features. .

[0031] The first visual encoder uses a Vision Transformer (ViT)-based architecture, which consists of twelve layers of Transformer modules connected in sequence; The image is divided into multiple fixed-size patches and then input into the first visual encoder after linear projection transformation; the first visual encoder can effectively capture the global and local information of the image and generate coding features rich in visual information .

[0032] The image generation block is used to generate the intermediate features f * and the encoding feature f vThe spliced ​​fused feature map is upsampled to obtain the generated image, and the spliced ​​fused feature map contains both semantic information and visual information; the image generation block includes six layers of upsampling generation blocks connected in sequence; the structures of the first three layers of upsampling generation blocks are the same, all of which include a series of upsampling layers, a first convolutional layer, a first affine block, a second convolutional layer, and a second affine block; the upsampling layer is used to obtain a feature map with a higher resolution; the structures of the last three layers of upsampling generation blocks are the same, all of which include a series of upsampling layers, convolutional layers, a first affine block, a deformable convolutional layer, and a second affine block; the convolutional layer is used to reconstruct the spatial structure and visual details of the feature map; the affine block is used to inject text semantics to achieve a close fusion of semantics and image details; the deformable convolutional layer forms a new convolution kernel structure by adding extended convolution points in different directions, generally including a center point and several extended points around the center, and each extended point corresponds to a different direction; for the position of each convolution kernel in the deformable convolutional layer , define multiple directions , each direction corresponds to an adjustable displacement , forming a set of direction points ;in, ; k is the size of the convolution kernel; the convolution operation of the feature map X input to the deformable convolution layer is expressed as: Formula (2) in, Output features for the deformable convolutional layer at position The value of is the convolution weight corresponding to each direction; By displacement In the feature map The pixel value sampled in .

[0033] 2-2. The discriminator includes a second visual encoder, a feature extraction module, a semantic alignment module and an image evaluation module; the second visual encoder has the same structure as the first visual encoder and is used to extract the real image X layer by layer. real The image under test output by the generator module Visual features of different scales and semantic information to obtain real image features and the image features under test ; Real image X real With the image under test The text conditions are the same.

[0034] The feature extraction module extracts and enhances the visual features obtained by the second visual encoder layer by layer, thereby improving the expressiveness of the features; the feature extraction module includes three feature extraction blocks connected in sequence; the input of the first feature extraction block is the output of the second layer Transformer module in the second visual encoder; the input of the second and third feature extraction blocks are the output of the previous feature extraction block combined with the output of the fifth and ninth layers Transformer modules in the second visual encoder. The three feature extraction blocks have the same structure, including the first convolution layer, the first ReLU function, the second convolution layer, and the second RuLU function connected in sequence; the feature extraction module obtains the real image X by fusing and deeply analyzing the multi-level features extracted by the second visual encoder. real The corresponding visual summary feature f r-end and the image under test Corresponding visual summary features ; Through the feature extraction block, higher-level visual semantic information can be gradually obtained, providing richer and hierarchical visual guidance for the identification process.

[0035] The semantic alignment module is used to fuse text semantics and image semantics; the semantic alignment block includes three semantic alignment blocks connected in sequence; the three semantic alignment blocks have the same structure, including the first mapping convolution layer, the spatial transformation layer (MST) and the second convolution layer connected in sequence; the first mapping convolution layer is used to adjust the number of channels to twice the original input; the spatial transformation layer can realize the fusion of different modal semantics, and the fused alignment semantic se output by the spatial transformation layer in each semantic alignment block j for: Formula (3) Where j is the sequence of semantically aligned blocks.

[0036] Feature map output by the first semantic alignment block ; Feature maps output by the second and third semantic alignment blocks for: Formula (4) The feature map output by the last semantic alignment block The semantic alignment feature f is the output of the semantic alignment module. se The semantic alignment operation is mainly to enhance the fine-grained discrimination ability of the discriminative network and to achieve local consistency across modalities based on multimodal feature alignment.

[0037] The image evaluation module semantically aligns the features f se , text features T and visual summary features f r-end and The concatenated features are used as input to obtain the final loss value. The image evaluation module includes two image evaluation layers connected in sequence; the two image evaluation layers have the same structure, both of which include two convolutional layers connected in sequence and a ReLU activation function. The image evaluation module ensures that the features of the visual and textual modalities are fully integrated during the evaluation process, and improves the model's ability to evaluate the semantic and visual consistency of the generated images.

[0038] Step 3: Input the training set into the text-generated image model for training, and update the parameters in the generator through the output of the discriminator; use the test set to test the trained model of the present invention and the existing model, and use the FID and ClipScore scores to evaluate the images generated by the model; when testing on the CUB dataset, each semantic text in the test set generates 10 corresponding images, and a total of 29,330 images are generated to calculate the FID score. When testing on the COCO dataset, each semantic text generates an image, and a total of 40k images are generated to calculate the FID score. The results of comparison with currently popular models such as DFGAN and GALIP are shown in Table 1.

[0039] Table 1 Comparison of FID and ClipScore scores of various models on CUB and COCO datasets Table 1 lists the FID scores of several popular models and the proposed model on the CUB and COCO datasets. On the CUB dataset, compared with GALIP, the proposed model reduces the FID score from 10.07 to 9.56. On the COCO dataset, the FID score is reduced from 5.85 to 5.62. From the perspective of quantitative analysis, the images generated by the proposed model are closer to the real images than those generated by other methods, the latent semantic consistency between the text and the generated image is better, and the image quality is greatly improved.

[0040] The real image, the image generated by the model of the present invention and the image generated by the GALIP model are visually compared. The comparison results are as follows: Figure 2 , Figure 3 shown.

[0041] Figure 2The comparison example of the image generation effect of the model of the present invention and the GALIP model on the CUB dataset is shown. In the first image, the image generated by the present invention better reflects the orange bird body, black wings and a small amount of white details described in the text, and the generation effect is excellent in terms of semantic consistency and detail restoration. In contrast, although the image generated by the GALIP model can reflect some text semantics, it fails to accurately reproduce the orange body and the white features of black feathers, and its similarity to the real image is not as good as that of the model of the present invention. The sixth image shows the text description of the generated black bird. The image generated by the model of the present invention is not only highly consistent with the text description at the semantic level, but also particularly meticulous in detail processing. The gloss and texture of the feathers are natural, which enhances the realism of the image. In contrast, although the bird image generated by the GALIP model is roughly consistent with the text in terms of semantics, the detail processing is relatively rough, especially in terms of feather texture and morphology. Therefore, the model of the present invention is not only superior to the GALIP model in semantic consistency, but also richer in detail expression, making the generated image more realistic. In other image generation examples, although both models can reflect the basic semantics of the text, the model of the present invention has a more accurate understanding of the text semantics, its semantic consistency is significantly higher than that of the GALIP model, and it shows a higher level in detail restoration.

[0042] Figure 3 The figure shows a comparison of the generation effects of the two models on the COCO dataset. Since the images in the COCO dataset are diverse and the scenes are complex, the challenge of the generation model is how to ensure the realism of the generated images as much as possible. In the first image, the giraffe generated by the model of the present invention is not only morphologically close to the posture of a real giraffe, but also carefully processes the patterns and textures to avoid large-scale distortion of the image; while the giraffe image generated by the GALIP model in this scene has certain distortion. In the sixth image, the model of the present invention successfully depicts the dynamic posture of the baseball throwing player, and the generated character shape is natural, and there is no problem of character distortion like the one generated by the GALIP model. In addition, the model of the present invention also realistically depicts other characters and objects in the scene, showing a strong ability to process details.

[0043] In general, compared with the images generated by the GALIP model, the model of the present invention not only performs better in semantic fit, but also has significant advantages in semantic consistency and detail capture. The model of the present invention can more accurately present the detailed features of the text description, and further enhance the detail effect during the generation process, making the generated image closer to the real image. Through comparative analysis, it can be seen that the improvements in semantic understanding and detail processing of the model of the present invention make it perform better than existing models in the task of generating images.

Claims

1. A text-to-image method based on semantic alignment and text fusion, characterized by: The following steps are involved: Step 1: Obtain text and corresponding images to construct a data set; use the image corresponding to the text as the real image of the text; Step 2: Build a text-to-image model; the text-to-image model includes a generator and a discriminator; the generator is used to generate an image that fits the semantics of the text based on the text; The discriminator comprises a second visual encoder, a feature extraction module, a semantic alignment module and an image evaluation module; the second visual encoder is used to extract the tested image generated by the generator. With the real image X real Visual features of the image being tested and real image features ; The feature extraction module extracts and enhances the visual features obtained by the second visual encoder layer by layer to obtain the real image X real The corresponding visual summary feature f r-end and the image under test Corresponding visual summary features ; The semantic alignment module is used to align the features of the image under test and real image features Compare and correct the high-level semantic features of the se ; The image evaluation module semantically aligns the features f se , text features T and visual summary features f r-end and The concatenated features are used as input to obtain identification results; The semantic alignment module comprises a plurality of semantic alignment blocks connected in sequence; the plurality of semantic alignment blocks have the same structure, and all comprise a first mapping convolution layer, a spatial conversion layer, and a second convolution layer connected in sequence; each semantic alignment block sequentially maps features through the first mapping convolution layer and the spatial conversion layer and Features Fusion, obtain fusion alignment semantics se j ; The second convolutional layer in the first semantic alignment block receives the output of the corresponding spatial transformation layer as input; The second convolutional layers in the remaining semantic alignment blocks all take the concatenated features of the corresponding spatial transformation layer and the output of the previous semantic alignment block as input; Step 3: Input the data set into the text-to-image model for training; use the trained text-to-image model to generate an image that fits the semantics of the text based on the unlabeled text.

2. The method for generating images from text based on semantic alignment and text fusion according to claim 1, characterized in that: The generator includes a text encoder, a Reshape layer, a text feature fusion module, a star module convolution layer, a first visual encoder and a picture production block; the text encoder is used to obtain a text embedding vector T according to the input text; the Reshape layer is used to obtain the initial feature according to the random noise vector Z; the text feature fusion module is used to perform affine transformation and nonlinear activation processing on the initial feature according to the text embedding vector T to generate an intermediate feature f; the star module convolution layer is used to process the output of the text feature fusion module to obtain the intermediate feature ; The first visual encoder is used to extract intermediate features Global and local information of ; The image generation block is used to generate the intermediate feature f * and the encoding feature f v The spliced ​​fused feature map is upsampled to obtain the generated image.

3. The text-to-image method based on semantic alignment and text fusion according to claim 2, characterized in that: The text feature fusion module comprises six text feature fusion layers connected in sequence; the six text feature fusion layers have the same structure, and all comprise four affine blocks and convolution layers connected in sequence.

4. The method for generating images from text based on semantic alignment and text fusion according to claim 2, characterized in that: The picture generation block includes six layers of upsampling generation blocks connected in sequence; the first three layers of upsampling generation blocks have the same structure, including a series of upsampling layers, a first convolutional layer, a first affine block, a second convolutional layer, and a second affine block; the last three layers of upsampling generation blocks have the same structure, including a series of upsampling layers, convolutional layers, a first affine block, a deformable convolutional layer, and a second affine block.

5. The method for generating images from text based on semantic alignment and text fusion according to claim 2, characterized in that: The text encoder, the first visual encoder and the second visual encoder each include twelve layers of Transformer modules connected in sequence.

6. The method for generating images from text based on semantic alignment and text fusion according to claim 5, characterized in that: The first visual encoder and the second visual encoder have the same structure and both adopt the Vision Transformer architecture.

7. The method for generating images from text based on semantic alignment and text fusion according to claim 5, characterized in that: The feature extraction module includes three feature extraction blocks connected in sequence; The input of the first feature extraction block is the output of the second-layer Transformer module in the second visual encoder; the input of the second and third feature extraction blocks are the features of the output of the previous feature extraction block combined with the output of the fifth and ninth-layer Transformer modules in the second visual encoder, respectively.

8. The method for generating images from text based on semantic alignment and text fusion according to claim 7, characterized in that: The feature extraction blocks have the same structure, and both include a first convolutional layer, a first ReLU function, a second convolutional layer, and a second RuLU function connected in sequence.

9. The method for generating images from text based on semantic alignment and text fusion according to claim 1, characterized in that: The number of semantic alignment blocks in the semantic alignment module is three.

10. The method for generating images from text based on semantic alignment and text fusion according to claim 1, characterized in that: The image evaluation module comprises two image evaluation layers connected in sequence; the two image evaluation layers have the same structure, both comprising two convolutional layers connected in sequence and a ReLU activation function.

Images