A Text-to-Image Generation Method Based on Thought Chain and Visual Prior Guidance

Abstract

This invention relates to the fields of artificial intelligence and computer vision technology, and discloses a text-to-image generation method based on thought chain and visual prior guidance, implemented using a diffusion model. The method includes the following steps: obtaining original text prompts and constructing noise latent variables using Gaussian random noise; logically deconstructing the original text prompts using a large language model to generate a multi-dimensional semantic description set S; initializing the current denoising time step t and performing denoising processing, which updates the noise latent variables; decrementing the current denoising time step t by a step size of 1; determining whether the current denoising time step t is greater than 0, and if so, performing denoising processing again; if t=0, mapping the updated noise latent variables back to the pixel space through an image decoder to generate the final high-quality target image. This invention effectively solves the problems of semantic ambiguity, chaotic spatial layout, and visual fragmentation inherent in existing technologies in complex environments.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer vision technology, specifically to a method for generating images from text based on thought chains and visual prior guidance. Background Technology

[0002] With the rapid development of AI-generated content technology, text-to-image generation has become a research hotspot in the field of computer vision. This technology aims to understand user-input natural language descriptions through deep learning models and transform them into high-quality visual images with corresponding semantics. It is widely used in digital art creation, advertising creative design, game scene development, metaverse content construction, and industrial design assistance, significantly improving the efficiency and richness of content production.

[0003] Currently, the mainstream technical solutions in this field are mainly based on diffusion models. Among them, the most representative technical solutions related to this invention include: image generation methods based on latent diffusion models (LDM), image generation methods based on grounding and layout guidance, and large language models and image generation technologies.

[0004] In recent years, large language models (LLMs) based on the Transformer architecture have made groundbreaking progress. Through pre-training on massive amounts of text data, LLMs have acquired rich linguistic knowledge and general common sense. With the continuous expansion of parameter scale (such as GPT-4, LLaMA, etc.), these models have demonstrated powerful contextual understanding, complex logical reasoning, code generation, and intent recognition capabilities. In particular, the introduction of instruction fine-tuning and human feedback reinforcement learning techniques has enabled LLMs to accurately follow human instructions, providing a foundation for their participation as a "cognitive center" in multimodal tasks. Meanwhile, text-to-image (T2I) technology has also developed rapidly, with latent diffusion models becoming the mainstream approach. This technology compresses images into a latent space using an autoencoder and utilizes a U-Net network combined with text guidance for denoising and reconstruction, achieving high-fidelity image generation. However, basic diffusion models often lack the ability to logically plan for complex scenes and automatically optimize for aesthetic details; therefore, combining the reasoning capabilities of LLMs to enhance the controllability and quality of image generation has become a current research hotspot, such as:

[0005] (1) A layout generation method based on LLM for visual planning. In order to use LLM to solve the problem of spatial layout chaos in image generation, Feng et al. proposed the LayoutGPT model (see foreign literature: Feng, W., et al. "LayoutGPT: Compositional Visual Planning and Generation with Large Language Models." NeurIPS. 2023). The function of this technical solution is to use a large language model as a "visual planner" to transform abstract text descriptions into specific scene layouts (such as object bounding boxes), thereby guiding image generation;

[0006] (2) A prompt word optimization method based on visual feedback reinforcement learning: In response to the problem that the initial prompt words input by users are often too simple and difficult to trigger the model to generate high-aesthetic quality images, Cao et al. proposed the BeautifulPrompt model. The function of this technical solution is to automatically rewrite and expand the low-quality user input into high-quality prompt words rich in details, so as to significantly improve the visual effect of the generated image.

[0007] (3) For example, the patent document with publication number CN114022582B specifically discloses a text-to-image method. Based on the Transformer module and the AttnGAN network, the text is encoded by the text encoder to obtain sentence features and word features. The sentence features are obtained by the conditional enhancement module to obtain feature vectors. These feature vectors are fused with random noise vectors and input into the Transformer module for learning. The improved feature vectors are output and input into the generator to generate a rough 64×64 pixel initial image. The initial synthesized image and the improved feature vectors are input into the discriminator for discrimination and the generator is trained according to the loss function. The improved feature vectors from the previous step are input into the neural network together with the word features for upsampling to obtain a fused vector. This fused vector is then input into the generator to obtain a 128×128 pixel image and a 256×256 pixel image.

[0008] While the aforementioned existing technologies have promoted the development of text-to-image generation technology to some extent, they still have the following technical shortcomings in practical applications:

[0009] 1) The Latent Diffusion Model (LDM) suffers from deficiencies in complex logic and spatial reasoning. Due to the lack of explicit logical reasoning modules, it struggles to understand complex spatial relationships (such as "left" and "containment") and quantitative concepts, easily leading to "attribute leakage" (e.g., incorrectly rendering the red color from "red hat" onto clothing). Furthermore, the generation process is based on denoising-driven random sampling, lacking a unified plan from global to local perspectives, resulting in generated scenes that are often loosely structured, with objects lacking inherent perspective and logical connections.

[0010] 2) The LLM-based layout generation method (LayoutGPT) has shortcomings in visual fusion and detail coherence. It suffers from a serious "planning-generation" disconnect and an over-reliance on retrieval examples, which can easily create illusions and output incorrect layouts that overlap, go out of bounds, or are out of proportion. Moreover, such errors will be directly passed to the downstream generation stage and cannot be corrected by itself.

[0011] 3) Prompt-based engineering optimization methods (such as BeautifulPrompt) have defects in semantic fidelity and structural control. They sacrifice semantic drift for aesthetic quality and cannot reach the underlying structure of the generative model. When faced with scenarios that require precise geometric control or complex multi-object interaction, simply relying on text modification still cannot guide the model to generate a spatial structure that conforms to strict logic.

[0012] Therefore, a technical solution is needed to address the issues of missing global logical planning, chaotic spatial layout, and visual disconnect between local details and the overall environment in text-to-image generation. Summary of the Invention

[0013] To address the technical problems of lack of global logical planning, chaotic spatial layout, and visual disconnect between local details and the overall environment in existing technologies, this invention provides a text-to-image generation method based on thought chain and visual prior guidance, implemented using a diffusion model, comprising the following steps:

[0014] Get the original text prompt words Noise latent variables are constructed using Gaussian random noise. ;

[0015] Based on large language model Preset thought chain instruction protocol For the original text prompt words Logical deconstruction is performed to generate a multidimensional semantic description set S; the multidimensional semantic description set S includes three-dimensional structured sub-descriptions. The three dimensions include layout dimension, attribute dimension, and detail dimension;

[0016] Initialize the denoising process, including: setting the maximum value T of the denoising time step, defining the current denoising time step t, and 0≤t≤T; initializing the current denoising time step t to T;

[0017] At the current denoising time step t, based on the noise latent variable A denoising process is performed on the multidimensional semantic description set S, which is used to update the noise latent variables. ;

[0018] Decrease the current denoising time step t by a step size of 1;

[0019] Determine if the current denoising time step t is greater than 0. If it is, perform the denoising process again; if t=0, update the noise latent variable. The final high-quality target image is generated by mapping back to pixel space through an image decoder.

[0020] Furthermore, the noise reduction process includes the following:

[0021] The noise latent variable The main generator network is used to generate attention features for the main text branch. ;

[0022] Load a visual attention feature generation model, which is used to generate visual attention features from the noise latent variables. Obtaining visually guided feature vectors Output visual attention features ;

[0023] The main text branch attention feature Visual attention characteristics By superimposing the features, a fused feature is obtained. ;

[0024] The fusion features With noise latent variables The noise residual prediction value is obtained by comparison; the noise latent variable is then used to... Denoising is performed by subtracting the predicted noise residual, generating a new noise latent variable, and then updating the original noise latent variable with the new noise latent variable. .

[0025] Furthermore, the main text branch attention feature The generation process includes the following:

[0026] At the current denoising time step t, the noise latent variables are extracted by the master generator network. Original text features;

[0027] Map the original text features to text-specific keys. Sum It interacts with the query vector Q to generate the main text branch attention features. The main text branch attention features The expression is:

[0028] .

[0029] Furthermore, the visual attention features The output process includes the following:

[0030] An auxiliary feature extraction network is configured, the input data of which is consistent with the input data of the main generator network. The auxiliary feature extraction network extracts two-dimensional visual feature tensors that are spatially aligned to each dimension of the multidimensional semantic description set S. The input data includes the current denoising time step t and the noise latent variable. and a multidimensional semantic description set S;

[0031] A temporal dynamic weighted fusion mechanism is set up to calculate the two-dimensional visual feature tensor of each dimension. dynamic weight vector The final visual guidance feature vector is obtained by performing a weighted summation. ;

[0032] An independent, parallel visual enhancement channel is set up as a bypass of the main generator network to guide the visual feature vector. Mapped to vision-specific keys Sum It interacts with the query vector Q to output the visual attention features. The visual attention features The expression is:

[0033] .

[0034] Furthermore, the expression for the multidimensional semantic description set S is:

[0035] ;

[0036] Where k represents the dimension, with values ​​of 1, 2, and 3 corresponding to the layout dimension, attribute dimension, and detail dimension, respectively.

[0037] Furthermore, the spatially aligned two-dimensional visual feature tensor in the multidimensional semantic description set S The extraction process includes the following:

[0038] The structured sub-description of the three dimensions Input the auxiliary feature extraction network, and extract the three-dimensional two-dimensional visual feature tensor from a specific downsampling layer of the text encoder of the auxiliary feature extraction network. Three spatially aligned two-dimensional visual feature tensors were obtained. The expression is:

[0039] ;

[0040] in, This refers to the auxiliary feature extraction network; Indicates a text encoder. Indicates layout; Indicates an attribute; Indicates details.

[0041] Furthermore, the time-series dynamic weighted fusion mechanism includes the following:

[0042] Convolutional layers are used to process the three-dimensional visual feature tensors. Project onto a low-dimensional bottleneck space;

[0043] Constructing a time-aware gating subnet includes: taking the sinusoidal position code of the current denoising time step t as input and outputting a normalized dynamic weight vector. The dynamic weight vector The expression is:

[0044] ;

[0045] in, This represents the sinusoidal position code of the current denoising time step t; Represents a time-aware gating subnet;

[0046] Based on the dynamic weight vector The two-dimensional visual feature tensor for each dimension of the low-dimensional bottleneck space The weighted sum is then applied and the result is restored to the target feature dimension through a mapping layer to obtain the visual guidance feature vector. The visual guidance feature vector The expression is:

[0047] ;

[0048] in, Indicates the convolution operation; Represents the mapping layer. This represents the weight values ​​for different dimensions.

[0049] Furthermore, in the initial stage of denoising, the dynamic weight vector of the time-series dynamic weighting fusion mechanism... The layout dimension is tilted to establish the image structure; in the later stage of noise reduction, the dynamic weight vector... Towards the details Tilt to optimize texture for precise guidance throughout the process.

[0050] Furthermore, the fusion feature The acquisition process includes the following:

[0051] Configure a learnable, zero-initialization gating mechanism to access the visual attention features. The attention features of the main text branch are superimposed in the form of residuals. The above is used to obtain the fusion features. The fusion feature The expression is:

[0052] ;

[0053] Where α is the visual guidance feature vector The weight.

[0054] Furthermore, the learnable zero-initialization gating mechanism is used to ensure that the visually guided feature vector The weight α is zero in the initial training phase and gradually increases during the learning and training process.

[0055] The beneficial effects of this invention are:

[0056] (1) This invention introduces a large language model for structured semantic reasoning and combines visual feature mapping of isomorphic networks with temporal dynamic attention mechanism to effectively solve the problems of semantic ambiguity, spatial layout disorder and visual fragmentation in the generation of complex scenes in existing technologies.

[0057] (2) This invention utilizes a structured semantic decomposition module to transform unstructured text into an explicit “layout-attribute-detail” hierarchical description, reducing the model’s erroneous inference of implicit semantics (such as attribute mismatch), making the generated content respond more accurately to text instructions, thereby improving the CLIP-T (text-image consistency) score.

[0058] (3) Adopting a time-series dynamic weighted fusion mechanism, following the generation rule of "contour first, details later" of the diffusion model, the feature weights are adaptively adjusted at different stages (emphasizing structure in the early stage and texture in the later stage), thereby optimizing the overall texture and detail of the image and reducing the FID (image realism) value.

[0059] (4) By using isomorphic visual feature mapping and independent channel injection strategy, the visual prior features and the generation network are aligned in the feature space, which effectively reduces the rejection phenomenon during feature fusion, so that the generated image maintains the preset spatial structure while the visual transition is more natural, thereby improving the CLIP-I score. Attached Figure Description

[0060] Figure 1 This is a flowchart of the text-to-image generation method based on thought chain and visual prior guidance provided by the present invention.

[0061] Figure 2 This is a flowchart of the noise reduction process provided by the present invention;

[0062] Figure 3 The visual attention features provided by this invention The output process flowchart. Detailed Implementation

[0063] The technical solution of the present invention is further described below, but the scope of protection is not limited to what is described.

[0064] This invention provides a text-to-image generation method based on thought chain and visual prior guidance, implemented using a diffusion model, such as... Figure 1 As shown, it includes the following steps:

[0065] Step S100: Obtain the original text prompt words. Noise latent variables are constructed using Gaussian random noise. ;

[0066] Step S200, based on the large language model Preset thought chain instruction protocol For the original text prompt words Logical deconstruction is performed to generate a multidimensional semantic description set S; the multidimensional semantic description set S includes three-dimensional structured sub-descriptions. The three dimensions include layout dimension, attribute dimension, and detail dimension;

[0067] The MindChain Instruction Protocol Used to guide the model in hierarchical reasoning and self-correction; the multidimensional semantic description set S is an explicit multidimensional semantic description set, the large language model As the core of semantic reasoning, the problem of attribute entanglement under complex prompts eliminates ambiguity from the semantic source and ensures the logical correctness of the generated content.

[0068] The expression for the multidimensional semantic description set S is:

[0069] (1)

[0070] Where k represents the dimension, with values ​​of 1, 2, and 3 corresponding to the layout dimension, attribute dimension, and detail dimension, respectively.

[0071] Step S300, initialize the denoising process, including: setting the maximum value T of the denoising time step, defining the current denoising time step t, and 0≤t≤T; initializing the current denoising time step t to T;

[0072] Step S400, at the current denoising time step t, based on the noise latent variable A denoising process is performed on the multidimensional semantic description set S, which is used to update the noise latent variables. ;

[0073] The noise reduction process, such as Figure 2 As shown, it includes the following:

[0074] Step S410, the noise latent variable The main generator network is used to generate attention features for the main text branch. ;

[0075] The main text branch attention feature The generation process includes the following:

[0076] Step S411: At the current denoising time step t, extract the noise latent variables through the master generator network. Original text features;

[0077] Step S412: Map the original text features to text-specific keys. Sum It interacts with the query vector Q to generate the main text branch attention features. The main text branch attention features The expression is:

[0078] (2)

[0079] Step S420: Load the visual attention feature generation model, which is used to generate visual attention features from the noise latent variables. Obtaining visually guided feature vectors Output visual attention features ;

[0080] The visual attention features The output process, such as Figure 3 As shown, it includes the following:

[0081] Step S421: Set up an auxiliary feature extraction network. The input data of the auxiliary feature extraction network is consistent with the input data of the main generator network. The auxiliary feature extraction network extracts two-dimensional visual feature tensors that are spatially aligned in each dimension of the multidimensional semantic description set S. The input data includes the current denoising time step t and the noise latent variable. The network provides a multidimensional semantic description set S. The auxiliary feature extraction network employs a state-sharing strategy to extract spatially aligned visual features, addressing the dimensionality mismatch between textual and visual features. The network receives input identical to that of the main generator network to ensure that the input distribution of the auxiliary branch (visual branch) is consistent with that of the main branch. The output includes noise latent variables. and the current denoising time step The auxiliary feature extraction network eliminates the domain gaps during cross-modal feature injection, ensuring that the extracted visual features are strictly aligned with the generation process in terms of mathematical distribution, thus achieving a lossless mapping from the text semantic space to the visual feature space.

[0082] The spatially aligned two-dimensional visual feature tensor in the multidimensional semantic description set S The extraction process includes the following:

[0083] The structured sub-description of the three dimensions Input the auxiliary feature extraction network to extract the three-dimensional two-dimensional visual feature tensor from a specific downsampling layer (such as the second downsampling block) of the text encoder of the auxiliary feature extraction network. Three spatially aligned two-dimensional visual feature tensors were obtained. The expression is:

[0084] (3)

[0085] in, This refers to the auxiliary feature extraction network; This represents a text encoder used to translate structured sub-descriptions. Convert to an embedded vector; Indicates layout; Indicates an attribute; Indicates details.

[0086] Step S422: Set up a temporal dynamic weighted fusion mechanism to calculate the two-dimensional visual feature tensors of each dimension. dynamic weight vector The final visual guidance feature vector is obtained by performing a weighted summation. Achieve adaptive fusion and precise guidance of features from different semantic dimensions;

[0087] Combining the physical generation principle of the diffusion model of "first constructing the global outline and then filling the local texture", the guidance strategy can be adaptively adjusted according to the current denoising time step: in the early stage of generation, the weight of the "layout dimension" feature is automatically increased to establish the skeleton of the image, and in the later stage of generation, the weight of the "detail dimension" feature is automatically increased to optimize the texture, thus avoiding the structural rigidity or loss of details caused by static feature injection.

[0088] The time-series dynamic weighted fusion mechanism includes the following:

[0089] Step S421-1: Use a convolutional layer to process the three-dimensional visual feature tensors. Projected to a low-dimensional bottleneck space; used to reduce computational redundancy and align feature distributions.

[0090] Step S421-2, constructing a time-aware gating subnet, including taking the sinusoidal position code of the current denoising time step t as input and outputting a normalized dynamic weight vector. ;

[0091] After the sinusoidal position code is input at the current denoising time step t, the generation process is learned through a multilayer perceptron (MLP) to meet the stage requirements, and a normalized dynamic weight vector is output via the Softmax function. The dynamic weight vector... It accurately reflects the degree to which the model depends on the three dimensions of "layout", "attributes" or "details" at the current denoising time step t.

[0092] The dynamic weight vector The expression is:

[0093] (4)

[0094] in, The sinusoidal position code represents the current denoising time step t, used to introduce timing signals; This represents a time-aware gating subnet.

[0095] Step S422-3, based on the dynamic weight vector The two-dimensional visual feature tensor for each dimension of the low-dimensional bottleneck space The weighted sum is then applied and the result is restored to the target feature dimension through a mapping layer to obtain the visual guidance feature vector. The dynamic weight vector Weight values ​​including three dimensions ;

[0096] The visual guidance feature vector The expression is:

[0097] (5)

[0098] in, Indicates the convolution operation; Represents the mapping layer. This represents the weight values ​​for different dimensions.

[0099] In the initial stage of denoising, the time-series dynamic weighted fusion mechanism uses the dynamic weight vector... The layout dimension is tilted to establish the image structure; in the later stage of noise reduction, the dynamic weight vector... Towards the details Tilt to optimize texture for precise guidance throughout the process.

[0100] Step S423: Set up an independent and parallel visual enhancement channel in the bypass of the main generator network, and transmit the visual guidance feature vector. Mapped to vision-specific keys Sum It interacts with the query vector Q to output the visual attention features. The independent and parallel visual enhancement channels are set up to prevent high-intensity visual prior features from interfering with the original text semantic understanding ability of the main generator network; the bypass of the main generator network is in the cross-attention layer of the main generator network (U-Net).

[0101] The visual attention features The expression is:

[0102] (6)

[0103] Step S430, the main text branch attention feature Visual attention characteristics By superimposing the features, a fused feature is obtained. ;

[0104] The fusion feature The acquisition process includes the following:

[0105] Configure a learnable, zero-initialization gating mechanism to access the visual attention features. The attention features of the main text branch are superimposed in the form of residuals. The above is used to obtain the fusion features. The fusion feature The expression is:

[0106] (7)

[0107] Where α is the visual guidance feature vector The weights are used to control the intervention intensity of visual branches.

[0108] The learnable zero-initialization gating mechanism is used to ensure that the visually guided feature vector The weight α is zero in the initial training phase and gradually increases during the learning and training process.

[0109] The independent and parallel visual enhancement channels, combined with a zero-initialization gating mechanism, protect the original model's text understanding capabilities while introducing visual enhancements. To prevent high-intensity visual features from disrupting the pre-trained model's original language cognition, the visual enhancement channels employ a parallel branch design instead of directly modifying the original parameters. A learnable zero-initialization adjustment factor is introduced, ensuring that the visual branch has zero weights in the initial training phase and smoothly intervenes as training progresses. This guarantees that the model seamlessly absorbs explicit visual priors without experiencing abrupt changes in feature distribution or "catastrophic forgetting," significantly improving the model's robustness.

[0110] Step S440, the fusion feature With noise latent variables The noise residual prediction value is obtained by comparison; the noise latent variable is then used to... Denoising is performed by subtracting the predicted noise residual, generating a new noise latent variable, and then updating the original noise latent variable with the new noise latent variable. .

[0111] Step S500: Decrease the current denoising time step t by a step size of 1.

[0112] Step S600: Determine whether the current denoising time step t is greater than 0. If so, perform the denoising process again; if t=0, update the acoustic latent variable. The final high-quality target image is generated by mapping back to pixel space through an image decoder.

[0113] This invention, based on a fundamental diffusion model, utilizes a large language model to achieve explicit structured semantic deconstruction, resolving the attribute entanglement problem under complex prompts. Existing technologies typically feed text encoding directly into the generative network, leading to difficulties in distinguishing attribute attribution in multi-object scenes (e.g., confusing "red car" and "blue clothes"). This invention, through a structured semantic deconstruction module, leverages the reasoning capabilities of a large language model to forcibly transform the implicit text input by the user into an explicit structured description across three dimensions: layout, attributes, and details. This "reasoning first, generation later" mechanism eliminates ambiguity at the semantic source, ensuring the logical correctness of the generated content.

[0114] This invention employs a "homogeneous network" (i.e., an auxiliary feature extraction network) and a "state sharing" strategy to extract visual priors, eliminating domain gaps during cross-modal feature injection. Unlike existing technologies that use external image encoders for feature extraction, leading to mismatches with the generator network's feature space, this invention's isomorphic visual feature mapping module innovatively reuses an auxiliary feature extraction network with a structure completely identical to the main generator network as the feature extractor. More importantly, the auxiliary feature extraction network shares the exact same noise latent variables and time steps as the main network. This design ensures that the extracted visual features are mathematically aligned strictly with the generation process, achieving a lossless mapping from the text semantic space to the visual feature space.

[0115] The design incorporates a "temporally dynamic weighting mechanism" that conforms to the diffusion generation principle, achieving adaptive and precise guidance throughout the entire process. Combining the physical generation principle of the diffusion model—"first constructing the global outline, then filling in local textures"—this invention introduces a time-aware gating unit into the temporally dynamic weighted fusion module. The model can adaptively adjust the guidance strategy based on the current denoising time step: automatically increasing the weight of "layout dimension" features in the early stages of generation to establish the image skeleton, and automatically increasing the weight of "detail dimension" features in the later stages of generation to optimize textures. This dynamic adjustment avoids the structural rigidity or detail loss problems caused by static feature injection.

[0116] By constructing an "independent and parallel visual enhancement channel" in conjunction with a "zero initialization" strategy, the original model's text understanding ability is preserved while visual enhancement is introduced. To prevent high-intensity visual features from damaging the pre-trained model's original language cognition, the independent and parallel visual enhancement channel of this invention employs a parallel branch design instead of directly modifying the original parameters. In particular, a learnable zero-initialization adjustment factor is introduced, ensuring that the weight of the visual branch is zero in the initial training phase, and that it smoothly intervenes as training progresses. This design guarantees that the model seamlessly absorbs explicit visual priors without experiencing abrupt changes in feature distribution or "catastrophic forgetting," significantly improving the system's robustness.

[0117] Furthermore, experimental results show that the present invention achieves significantly better performance than existing mainstream models (such as Stable Diffusion and GLIGEN) in CLIP-T metric (measuring text-image consistency), FID metric (measuring image generation quality and realism), and CLIP-I metric (measuring visual structural consistency), demonstrating the superiority of this method in generating high-fidelity, strongly logically related images.

[0118] The above-disclosed embodiments are merely specific examples of the present invention. However, the present invention is not limited thereto, and any variations that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A text-to-image generation method based on thought chain and visual prior guidance, characterized in that, Based on the diffusion model, the implementation includes the following steps: Get the original text prompt words Noise latent variables are constructed using Gaussian random noise. ; Based on large language model Preset thought chain instruction protocol For the original text prompt words Logical deconstruction is performed to generate a multidimensional semantic description set S; the multidimensional semantic description set S includes three-dimensional structured sub-descriptions. The three dimensions include layout dimension, attribute dimension, and detail dimension; Initialize the denoising process, including: setting the maximum value T of the denoising time step, defining the current denoising time step t, and 0≤t≤T; initializing the current denoising time step t to T; At the current denoising time step t, based on the noise latent variable A denoising process is performed on the multidimensional semantic description set S, which is used to update the noise latent variables. ; Decrease the current denoising time step t by a step size of 1; Determine if the current denoising time step t is greater than 0. If it is, perform the denoising process again; if t=0, update the noise latent variable. The final high-quality target image is generated by mapping back to pixel space through an image decoder. The noise reduction process includes the following: The noise latent variable The main generator network is used to generate attention features for the main text branch. ; Load a visual attention feature generation model, which is used to generate visual attention features from the noise latent variables. Obtaining visually guided feature vectors Output visual attention features ; The main text branch attention feature Visual attention characteristics By superimposing the features, a fused feature is obtained. ; The fusion features With noise latent variables The noise residual prediction value is obtained by comparison; the noise latent variable is then used to... Denoising is performed by subtracting the predicted noise residual, generating a new noise latent variable, and then updating the original noise latent variable with the new noise latent variable. ; The visual attention features The output process includes the following: An auxiliary feature extraction network is configured, the input data of which is consistent with the input data of the main generator network. The auxiliary feature extraction network extracts two-dimensional visual feature tensors that are spatially aligned to each dimension of the multidimensional semantic description set S. The input data includes the current denoising time step t and the noise latent variable. and a multidimensional semantic description set S; A temporal dynamic weighted fusion mechanism is set up to calculate the two-dimensional visual feature tensor of each dimension. dynamic weight vector The final visual guidance feature vector is obtained by performing a weighted summation. ; An independent, parallel visual enhancement channel is set up as a bypass of the main generator network to guide the visual feature vector. Mapped to vision-specific keys Sum It interacts with the query vector Q to output the visual attention features. The visual attention features The expression is: ； The time-series dynamic weighted fusion mechanism includes the following: Convolutional layers are used to process the three-dimensional visual feature tensors. Project onto a low-dimensional bottleneck space; Constructing a time-aware gating subnet includes: taking the sinusoidal position code of the current denoising time step t as input and outputting a normalized dynamic weight vector. ; Based on the dynamic weight vector The two-dimensional visual feature tensor for each dimension of the low-dimensional bottleneck space The weighted sum is then applied and the result is restored to the target feature dimension through a mapping layer to obtain the visual guidance feature vector. .

2. The text-to-image generation method based on thought chain and visual prior guidance as described in claim 1, characterized in that, The main text branch attention feature The generation process includes the following: At the current denoising time step t, the noise latent variables are extracted by the master generator network. Original text features; Map the original text features to text-specific keys. Sum It interacts with the query vector Q to generate the main text branch attention features. The main text branch attention features The expression is: 。 3. The text-to-image generation method based on thought chain and visual prior guidance as described in claim 1, characterized in that, The expression for the multidimensional semantic description set S is: ； Where k represents the dimension, with values ​​of 1, 2, and 3 corresponding to the layout dimension, attribute dimension, and detail dimension, respectively.

4. The text-to-image generation method based on thought chain and visual prior guidance as described in claim 3, characterized in that, The spatially aligned two-dimensional visual feature tensor in the multidimensional semantic description set S The extraction process includes the following: The structured sub-description of the three dimensions Input the auxiliary feature extraction network, and extract the three-dimensional two-dimensional visual feature tensor from the second downsampling block of the text encoder of the auxiliary feature extraction network. Three spatially aligned two-dimensional visual feature tensors were obtained. The expression is: ； in, This refers to the auxiliary feature extraction network; Indicates a text encoder. Indicates layout; Indicates an attribute; Indicates details.

5. The text-to-image generation method based on thought chain and visual prior guidance as described in claim 4, characterized in that, The dynamic weight vector The expression is: ； in, This represents the sinusoidal position code of the current denoising time step t; Represents a time-aware gating subnet; The visual guidance feature vector The expression is: ； in, Indicates the convolution operation; Represents the mapping layer. This represents the weight values ​​for different dimensions.

6. The text-to-image generation method based on thought chain and visual prior guidance as described in claim 5, characterized in that, In the initial stage of denoising, the time-series dynamic weighted fusion mechanism uses the dynamic weight vector... The layout dimension is tilted to establish the image structure; in the later stage of noise reduction, the dynamic weight vector... Towards the details Tilt to optimize texture for precise guidance throughout the process.

7. The text-to-image generation method based on thought chain and visual prior guidance as described in claim 6, characterized in that, The fusion feature The acquisition process includes the following: Configure a learnable, zero-initialization gating mechanism to access the visual attention features. The attention features of the main text branch are superimposed in the form of residuals. The above is used to obtain the fusion features. The fusion feature The expression is: ； Where α is the visual guidance feature vector The weight.

8. The text-to-image generation method based on thought chain and visual prior guidance as described in claim 7, characterized in that, The learnable zero-initialization gating mechanism is used to ensure that the visually guided feature vector The weight α is zero in the initial training phase and gradually increases during the learning and training process.

Images