A text-to-image generation method based on cognitive heuristic diffusion

Abstract

The application discloses a text-to-image generation method based on cognitive heuristic diffusion and relates to the technical field of computer vision. In view of the 'double visual conflict' problem of global semantic deviation and local detail ambiguity existing in the existing single-stage generation paradigm, the technical scheme of the application has a two-stage cognitive heuristic diffusion generation framework, and the process comprises: the first stage is a reward feedback guided semantic alignment stage, a pseudo caption is used as an intermediary to construct semantic consistency loss, and iteration optimization is carried out in the latent space to ensure the rationality of the global semantic structure; the second stage is an attention entropy guided detail optimization stage, the attention entropy is used to locate the significant area with high uncertainty, and the entropy weighted optimization loss is used to enhance the details of the key area. In the application, by decoupling the semantic construction and detail refining process, the cooperative promotion of semantic consistency and local detail fidelity is effectively realized.

Description

Technical Field

[0001] This invention relates to a text-to-image generation method based on cognitive heuristic diffusion, belonging to the field of computer vision. Background Technology

[0002] Text-to-image generation (TGE) aims to generate high-quality images with semantic consistency and rich detail based on natural language prompts, and is an important research direction at the intersection of computer vision and natural language processing. With the emergence of large-scale text-image pairing datasets and the development of multimodal pre-trained models, TGE technology has shown broad application prospects in creative design, content creation, and virtual reality. Diffusion models, with their stable training process, excellent generation quality, and good controllability, have rapidly become the mainstream framework for TGE in recent years. Achieving high-quality image synthesis through iterative denoising significantly improves the visual fidelity and semantic coherence of the generated results.

[0003] However, most existing methods adopt an end-to-end single-stage generation paradigm, that is, simultaneously completing the construction of high-level semantic structures and the restoration of low-level details in a single diffusion process. This highly coupled generation mechanism makes it difficult to effectively coordinate global semantic control and local detail generation, leading to the following dual visual conflicts. On the one hand, the model relies too much on semantic guidance, which, while ensuring the rationality of the overall structure, often lacks detailed expressiveness. The generated image may have blurred key areas or texture distortion, resulting in global semantic deviation. For example, when generating "an old man sitting in a park reading a newspaper," although the scene layout can be correct, facial features and newspaper text are often difficult to identify. On the other hand, if local detail optimization is overemphasized, it is easy to destroy the overall semantic consistency of the image. When dealing with multiple objects and complex relationship cues, problems such as missing objects and misplaced attributes may occur, resulting in blurred local details. For example, when generating "two children playing a red soccer ball on a playground," the number of people may be incorrect or the color may be misjudged.

[0004] The dual-process theory in cognitive psychology reveals that human thinking and decision-making processes are dominated by two independent yet collaborative systems. System 1, characterized by rapid, intuitive, and automatic processing, excels at holistic judgment and initial perception; while System 2, characterized by slow, deliberate, and logical reasoning, is responsible for in-depth analysis and correction of the results of System 1. Current single-stage generative models exhibit a significant visual conflict between semantic consistency and detail fidelity, which we summarize as a "dual visual conflict." Its essence lies in the model's difficulty in simultaneously coordinating semantic constraints and detail generation within a unified optimization objective. The dual-process theory in cognitive psychology addresses this dual visual conflict through a divide-and-conquer approach, resolving each system separately to coordinate their respective strengths.

[0005] Previous studies have attempted to improve generation quality through multi-stage approaches. However, most have focused on improving resolution through size-related conditional coding and cascaded structures, or increasing image size through stage-by-stage generation. These methods have not demonstrated a decoupling of semantics and detail generation processes. Even with the introduction of staged strategies, the fundamental conflict between semantics and detail remains unresolved. A two-stage process is more conducive to mitigating dual visual conflicts. Therefore, we seek to explore a more advanced theory to guide the model in understanding the differences between semantics and detail in text-generated images from a cognitive reasoning perspective, in accordance with human thought. Summary of the Invention

[0006] To address the problems existing in the prior art, this application proposes a text-to-image generation method based on cognitive heuristic diffusion. It has a two-stage cognitive heuristic diffusion generation framework, which improves the dual visual conflict through the two-stage process, enabling the model to fully learn more comprehensive text semantics. It can further improve the accuracy of generated details during the collaborative optimization process, and guide the model to understand the differences in semantics and details in the text-generated image from the perspective of cognitive reasoning and human thinking.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is a text-to-image generation method based on cognitive heuristic diffusion, comprising the following steps: 1) Obtain the target text prompt and input the text prompt into the text encoder to obtain the text embedding representation, while initializing the latent variables of the diffusion model; The diffusion model is reversed for denoising; in the reverse denoising process of the diffusion model, the latent variables are conditionally constrained based on the text semantic embedding, intermediate latent representations are generated step by step, and the semantically aligned intermediate image is obtained through the decoder. 2) Use an image captioning generation model to perform reverse reasoning on the intermediate image to generate pseudo captions; Calculate the semantic consistency reward between the pseudo-subtitle and the target text prompt; Based on the semantic consistency reward, a semantic consistency loss function is constructed, and the gradient of the loss function with respect to the current latent variable is calculated. The latent variable is then updated using the gradient to obtain a semantically aligned latent representation. 3) Using semantically aligned latent representations as input, cross-attention maps are extracted from the network layers of the diffusion model during the diffusion denoising process; The attention entropy of spatial location is calculated based on the cross-attention map to quantify the attention uncertainty of the model in different regions; Attention entropy is used to identify salient regions with high uncertainty, and an entropy-weighted optimization loss function is constructed. 4) By minimizing the entropy-weighted optimization loss function, the salient regions are enhanced with details to generate the final image.

[0008] In the optimized text-to-image generation method based on cognitive heuristic diffusion described above, in step 1), the text conditional encoding and initial diffusion generation stage employ a pre-trained cross-modal text encoder to perform semantic embedding modeling on the input text prompt; the semantic representation output by the text encoder is injected as a conditional vector into each inverse denoising time step of the diffusion model, constraining the global semantic direction of the latent space generation. In step 2), when using the image captioning generation model to perform reverse reasoning to generate pseudo captions for the intermediate image, a frozen image captioning generation model is introduced to perform reverse semantic decoding on the intermediate generated image to generate corresponding pseudo caption descriptions, and semantic embedding encoding is performed on the pseudo captions and the original text prompts respectively. In step 2), a semantic consistency loss function is constructed by calculating the similarity between the pseudo-subtitle semantic embedding and the original text semantic embedding, and a semantic reward signal is constructed. In each time step of diffusion reverse denoising, guided by the semantic reward signal, the gradient of the semantic consistency loss on the latent variables is fed back to the latent space to iteratively correct the latent representation. The generation process is always constrained by the semantic reward to suppress semantic drift. In step 2), the semantic consistency loss only participates in backpropagation within a preset mid-to-late time step interval, in order to avoid imposing too strong semantic constraints on the potential space in the early diffusion stage, which would lead to a decrease in generative diversity.

[0009] In the optimized text-to-image generation method based on cognitive heuristic diffusion, step 3) uses the latent representation output from the semantic alignment stage as the initial input, and enters the second stage reverse denoising process while keeping the original text conditions unchanged. In the cross-attention layer of the diffusion model, the attention distribution map between image spatial location and text tag is extracted, and the attention entropy value corresponding to each spatial location is calculated based on the information entropy theory to quantify the semantic uncertainty of the model in different regions. By normalizing the attention entropy and setting a threshold, high-entropy regions are identified as potential areas of missing details or regions of uncertainty in generation. In step 3), when entering the attention entropy-guided detail optimization stage, the latent representations obtained in steps 1) and 2) are kept as the initial state, and only the optimization objective of the reverse denoising process is adjusted without changing the original text conditional input. In the optimized text-to-image generation method based on cognitive heuristic diffusion described above, in step 4), a position-weighted detail optimization loss function is constructed in the latent space for the high-entropy region. By applying weighted constraints to the difference between the current predicted latent representation and the reference latent representation output during the semantic alignment stage, the model is guided to enhance the local structure, texture, and boundary details in high-uncertainty regions while maintaining the overall semantic structure. After inverse denoising is completed, the decoder outputs the final generated image.

[0010] In the optimized text-to-image generation method based on cognitive heuristic diffusion, step 2) involves inputting the pseudo-caption description and the original text prompt into the same semantic embedding space, calculating their cosine similarity, and constructing a semantic reward function to measure the semantic consistency between the current generated result and the target text. The specific process of generating pseudo-captions and calculating the semantic consistency reward includes... Let the target text prompt be denoted as The target text embedding is obtained by encoding it using the CLIP text encoder. ; Use the current latent variables The input is processed by a denoising network, and a coarse intermediate image is obtained after decoding. ; Image caption generation model using frozen parameters Describe and generate corresponding pseudo-subtitles. ; The pseudo-subtitles were processed using the CLIP text encoder. Encode the pseudo-subtitles to obtain the embedded subtitles. ; Compute target text embedding With pseudo-subtitle embedding The cosine similarity between them is used as a semantic reward signal. ,

[0011] Semantic consistency loss function Defined as ; The gradient feedback of this loss on the latent variables is superimposed onto the original denoising objective function of the diffusion model, thereby achieving explicit constraints on semantic consistency without compromising the stability of the diffusion process.

[0012] In the optimized text-to-image generation method based on cognitive heuristic diffusion, in step 1), the diffusion model introduces a frozen image caption generation model during the reverse denoising process. The intermediate image generated at each preset time step or key time step is subjected to reverse semantic decoding to obtain the corresponding pseudo caption description, which characterizes the linguistic semantic expression of the currently generated image. In step 2), the diffusion model introduces a frozen image captioning generation model during the reverse denoising process. It performs reverse semantic decoding on the intermediate images generated at each preset or key time step to obtain corresponding pseudo-caption descriptions, which are used to characterize the linguistic semantic expression of the currently generated image. Specifically, this includes calculating the semantic consistency loss function when updating latent variables using gradients. Relative to the current latent variable gradient ,according to Update the latent variables in the latent space; The update step is performed at each denoising time step generated by diffusion or at a specified time step interval to guide the generated image structure to maintain global consistency with the target text cue through a semantic feedback loop.

[0013] In the optimized text-to-image generation method based on cognitive heuristic diffusion, in step 1), the image caption generation model uses a frozen BLIP-2 model to describe the generated image and obtain its corresponding pseudo-captions. ; The text encoder uses a CLIP text encoder that introduces freezing, which processes the original text prompts separately. and pseudo-subtitles Encode; The diffusion model uses the Stable Diffusion model as its basic architecture, and during fine-tuning, it only uses the title text corresponding to the image as a conditional input.

[0014] In the optimized text-to-image generation method based on cognitive heuristic diffusion described above, in step 3), when calculating the attention entropy, during the denoising time step... From the backbone network (U-Net) of the diffusion model, the first Layer-based extraction of cross-attention maps ; Cross-attention map ,in, and For the spatial dimensions of the feature map, The number of etymological terms in the text prompt; For each spatial location of the image feature map Calculate the entropy of its attention distribution over all text sources. , ; in, Indicates at time step At that time, the first The spatial location is the first Attention weights for each text word origin.

[0015] The attention weight distribution between image spatial location and text tag is extracted from the multi-layer cross-attention module of the diffusion model, and the corresponding attention entropy value is calculated based on the attention weight to characterize the semantic uncertainty of the model in different spatial regions.

[0016] In the optimized text-to-image generation method based on cognitive heuristic diffusion described above, in step 3), when identifying salient regions and constructing the entropy-weighted optimized loss function, the calculated attention entropy is... Normalization is performed to obtain the normalized entropy value. ; Set entropy threshold and entropy threshold The value range is set to 0.55 to 0.95; Normalized entropy value Greater than the threshold The region was identified as a salient region with high uncertainty. , This will be the focus of subsequent detailed optimization stages to achieve adaptive control of local generation quality.

[0017] Calculate position weights based on normalized entropy values. ; Constructing an entropy-weighted optimization loss function To maintain consistency with the diffusion process, ; in, For the latent variables predicted by the current model, As a potential variable for reference; The reference latent variables are derived from the latent representation after the first-stage semantic alignment, processed by a single... The result after noise reduction.

[0018] The optimized text-to-image generation method described above, based on cognitive heuristic diffusion, improves the acquisition of reference latent variables. At that time, the latent variables obtained after the first stage of semantic alignment As input, a denoising network based on a diffusion model is used for single-step denoising prediction, and the predicted denoised latent variable is used as the reference latent variable. .

[0019] The optimized cognitive heuristic diffusion text-to-image generation method described above, in step S4), constructs a spatially weighted loss function in the latent space for the high-uncertainty region. This allows the high-entropy region to obtain a larger gradient update magnitude during backpropagation, thereby prioritizing the improvement of structural and texture details in that region. The weighted loss constrains the difference between the latent representation predicted at the current time step and the reference latent representation output in the semantic alignment stage of step S1), thus providing stable guidance for the detail optimization process and preventing local detail enhancement from damaging the overall semantic structure. After completing all the backpropagation denoising steps, the decoder restores the final latent representation to the image space output, thereby obtaining a final generated image that achieves a balance between global semantic consistency and local detail quality.

[0020] The beneficial effects of this application are as follows: In this application's technical solution, guided by the dual-process theory in cognitive psychology, the model decouples and makes decisions about the generation task from a more fundamental perspective, mimicking how humans perceive the world. It integrates rapid, intuitive semantic construction information with deeply processed detail optimization information to form a two-stage collaborative deep knowledge representation algorithm. This algorithm identifies and balances the relationship between semantic consistency and detail fidelity, attempting to resolve the "double visual conflict" dilemma of global semantic bias and local detail ambiguity in existing single-stage generation paradigms. The model explicitly models the generation task process as an architecture of "intuitive construction first, then deep refinement," more clearly demonstrating the logical reasoning process of text-to-image generation. Through coarse-grained semantic reward feedback and fine-grained, focused entropy-aware reasoning, a novel modeling approach for diffusion generation is proposed to alleviate the difficulty of simultaneously achieving generation quality and semantic reliability under complex text conditions.

[0021] The technical solution of this application proposes a reward feedback-guided semantic alignment stage (System 1) to improve the ability to extract the global semantic structure from text prompts and the ability to initially integrate visual-linguistic semantic consistency, thereby enhancing the grasp of the macroscopic semantics of the generated image and ensuring the rationality of the global structure. To further optimize the visual quality of local regions, an attention entropy-guided detail optimization stage (System 2) is designed to deeply segment salient regions with high uncertainty in the image, ensuring that the model can accurately capture and repair key regions lacking detail without affecting the global semantics through an entropy weighting mechanism, achieving in-depth improvement of fine-grained semantics. Attached Figure Description

[0022] Figure 1 This is a schematic diagram illustrating the principle of the cognitive heuristic two-stage diffusion generation framework proposed in this application. Figure 2 Hyperparameters in the entropy-guided refinement module of this application The results of the ablation experiment; Figure 3-1 Color illustrations comparing qualitative results of different methods in text-to-image generation tasks in this application; Figure 3-2 Black and white illustrations comparing qualitative results of different methods in text-to-image generation tasks in this application; Figure 4-1 Color illustrations showing the effect comparison diagram under the two-stage generation framework in this application; Figure 4-2 A black-and-white illustration illustrating the effect comparison within the two-stage generation framework of this application; Figure 5 This is a comparison chart showing the impact of this application on the attention distribution under entropy-weighted loss. Detailed Implementation

[0023] The technical features of the present invention will be further illustrated below with reference to specific embodiments.

[0024] One of the technical solutions of the text-to-image generation method based on cognitive heuristic diffusion in this application mainly includes the following process: The input text prompts are fed into a pre-trained diffusion model for initial denoising in the latent space. A semantic alignment module guided by reward feedback constructs a reverse inference loop from vision to language, forming semantic consistency cues based on semantic comparison between pseudo-captions and the original prompts. These semantic consistency cues are then combined with a gradient backpropagation strategy to iteratively correct latent variables, constructing an "image-text-semantics" feedback loop in the feature space to ensure the rationality of the global semantic structure. Subsequently, an attention entropy-guided detail optimization module uses the semantically aligned latent representation as a foundation, combining a cross-attention mechanism to extract the uncertainty distribution in high-dimensional features. The discreteness of the attention map is then re-integrated as an entropy feature, spatially distinguishing high-uncertainty, high-entropy regions. Finally, the identified high-entropy regions are specifically enhanced in the detail optimization module using a position-weighted loss function, forming novel detail texture cues to assist the model in self-correction of local fine-grained generation. The model then performs deep association and clarification processing on blurred regions.

[0025] like Figure 1 As shown, this application provides a text-to-image generation method based on cognitive heuristic diffusion. In one embodiment of this application, the implementation of the technical solution specifically includes the following steps: 1) Text conditional encoding and initial diffusion generation stage (semantic alignment stage guided by reward feedback): The input text prompt is fed into the text encoder to obtain a text semantic embedding representation; In the reverse denoising process of the diffusion model, the latent variables are conditionally constrained based on the text semantic embedding, intermediate latent representations are generated step by step, and the semantically aligned intermediate generated image is obtained through the decoder. A frozen image captioning generation model is introduced to perform reverse semantic decoding on the intermediate generated image to generate corresponding pseudo caption descriptions, and semantic embedding encoding is performed on the pseudo captions and the original text prompts respectively; By calculating the similarity between the pseudo-subtitle semantic embedding and the original text semantic embedding, a semantic reward signal is constructed. Guided by this semantic reward signal, the potential variables at the current time step in the diffusion process are updated by gradient feedback to suppress semantic drift and enhance the global semantic consistency of the generated results.

[0026] 2) Semantic consistency loss construction and latent space optimization: Based on the semantic similarity between the pseudo-subtitles and the original text prompts, a semantic consistency loss function is constructed. In each time step of the diffusion reverse denoising, the gradient of the semantic consistency loss with respect to the latent variables is fed back to the latent space to iteratively correct the latent representation, so that the generation process is always constrained by the semantic reward. Thus, while ensuring the diversity of generation, intermediate representations that are highly consistent with the semantics of the text are generated first.

[0027] 3) Attention entropy-guided detail optimization stage: Using the latent representation output from the semantic alignment stage as the initial input, and keeping the original text conditions unchanged, the second stage reverse denoising process is initiated. In the cross-attention layer of the diffusion model, the attention distribution map between the image spatial location and the text tag is extracted, and the attention entropy value corresponding to each spatial location is calculated based on the information entropy theory to quantify the semantic uncertainty of the model in different regions. By normalizing the attention entropy and setting a threshold, high-entropy regions are identified as potential missing details or uncertain regions.

[0028] 4) Weighted detail optimization based on attention entropy: For the high-entropy region, a position-weighted detail optimization loss function is constructed in the latent space to give the high-entropy region a higher optimization weight during backpropagation. By weighting the difference between the current predicted latent representation and the reference latent representation output in the semantic alignment stage, the model is guided to enhance the local structure, texture and boundary details of the high-uncertainty region while keeping the overall semantic structure unchanged. After completing the inverse denoising, the decoder outputs the final generated image.

[0029] In step 3), when entering the attention entropy-guided detail optimization stage, the latent representations obtained in steps 1) and 2) are kept as the initial state. Only the optimization objective of the reverse denoising process is adjusted, without changing the original text conditional input. The attention weight distribution between image spatial location and text marker is extracted from the multi-layer cross-attention module of the diffusion model, and the corresponding attention entropy value is calculated based on the attention weight to characterize the semantic uncertainty of the model in different spatial regions.

[0030] Specifically, in one embodiment of this application, firstly, by using Stable Diffusion as the basic generation architecture, combined with CLIP text encoder to extract the semantic embedding of input prompts, and initialize the noise distribution of the latent space.

[0031] The semantic alignment module, guided by reward feedback, captures semantic drift during the generation process, aggregates visual-linguistic back-reasoning information, and achieves coarse-grained filtering and consistent alignment of the global semantics of the generated image.

[0032] The attention entropy-guided optimization of details (AEGR) module further models the spatial uncertainty of latent features, and achieves detail differentiation and optimization of local fuzzy regions through entropy mapping and weighted learning.

[0033] Finally, semantic rewards and entropy weight constraints are integrated to achieve synergistic optimization of semantic consistency and detail fidelity.

[0034] Specifically, the steps of the text-to-image generation method based on cognitive heuristic diffusion are as follows: 1) Stable Diffusion v1.4 is used as the base generative model to extract initial information. U-Net serves as its core denoising network, generating latent feature representations for subsequent modules. This is applied to the input text prompts. The CLIP text encoder predicts the corresponding high-dimensional semantic embedding. Initial Gaussian noise Input diffusion model, in Guided by the process, noise is gradually reduced, and an intermediate coarse image is generated via an image decoder. .

[0035] 2) Integrating language understanding and semantic feedback into the task reasoning process to simulate a cognitive system. The visual semantic features of the generated intermediate images can intuitively reflect the model's execution of text instructions. Complex text prompts often lead to semantic drift, which can be corrected through reverse verification via language mediation. Based on this, the reward feedback-guided semantic alignment module converts visual information into linguistic descriptions for comparison, thereby rationally planning global semantics and enhancing the model's understanding of the provided diagrams.

[0036] Specifically, the frozen BLIP-2 model is used as an image caption generator to process intermediate images. Reverse description to generate pseudo-subtitles Subsequently, the frozen CLIP text encoder was used to process the original text prompts. and pseudo-subtitles Encode the two to obtain their representations in the high-level semantic embedding space. and Here, semantic differences are quantified as cosine similarity between vectors, serving as a reward signal. Semantic reward signal Calculate as follows: in, and These represent the semantic vectors of the original prompt and the pseudo-title, respectively. This score reflects whether the model-generated image successfully conveys the core semantic information contained in the prompt text.

[0037] Semantic information is further quantified and utilized here, and the semantic consistency features obtained through intuition are processed into a novel semantic consistency loss. : in, The optimization objective is obtained through intuitive perception of visual-linguistic consistency. The gradient feedback of this loss on the latent variables is superimposed onto the original denoising objective function of the diffusion model, thereby achieving explicit constraints on semantic consistency without compromising the stability of the diffusion process. To guide generation using this signal, the loss on the current latent variables is calculated. gradient And update the variables in the latent space according to the following rules: This step is performed at each denoising time step. Through the gradient backpropagation mechanism, it forces the latent variables to evolve in a direction that is more semantically compatible, thereby effectively alleviating the problems of semantic shift and ambiguity.

[0038] 4) To address the issue of blurred local details, the attention entropy-guided detail optimization module uses the semantically aligned latent variable $z_t$ as input to simulate cognitive system II, employing a cross-attention mechanism to identify uncertain regions. The discreteness of the attention distribution directly reflects the model's degree of confirmation regarding the semantic correspondence of a specific region.

[0039] For U-Net Cross-attention graph of layers Calculate each spatial location attention entropy : in, Indicates at time step At that time, the first The attention weights of each image location to the $j$-th text word source are then calculated. Subsequently, the attention entropy is normalized using Min-Max to obtain... And set a threshold. (In this embodiment, it is preferred) To identify salient regions using binarization: This region, as a high-entropy region, represents a potentially less refined area.

[0040] 5) Finally, to enhance the generation of details in high-entropy regions, the model constructs a location-weighted loss function, incorporating entropy values ​​as weights into the optimization objective. The training objective then enhances the detailed texture of key regions by minimizing the distance between the predicted latent variable and the reference latent variable. in, The weights are determined by the normalized entropy value. For the clean latent variables predicted by the current model, These serve as reference latent variables from the first stage. This weighting mechanism allows high-entropy regions to obtain higher optimization gradients during training, guiding the model to generate more precise details in blurred regions, thus achieving progressive optimization from macroscopic semantics to microscopic details.

[0041] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should be protected by the present invention.

Claims

1. A text-to-image generation method based on cognitive heuristic diffusion, characterized in that: Includes the following steps: 1) Obtain the target text prompt and input the text prompt into the text encoder to obtain the text embedding representation, while initializing the latent variables of the diffusion model; The diffusion model is reversed for denoising; in the reverse denoising process of the diffusion model, the latent variables are conditionally constrained based on the text semantic embedding, intermediate latent representations are generated step by step, and the semantically aligned intermediate image is obtained through the decoder. 2) Use an image captioning generation model to perform reverse reasoning on the intermediate image to generate pseudo captions; Calculate the semantic consistency reward between the pseudo-subtitle and the target text prompt; Based on the semantic consistency reward, a semantic consistency loss function is constructed, and the gradient of the loss function with respect to the current latent variable is calculated. The latent variable is then updated using the gradient to obtain a semantically aligned latent representation. 3) Using semantically aligned latent representations as input, cross-attention maps are extracted from the network layers of the diffusion model during the diffusion denoising process; The attention entropy of spatial location is calculated based on the cross-attention map to quantify the attention uncertainty of the model in different regions; Attention entropy is used to identify salient regions with high uncertainty, and an entropy-weighted optimization loss function is constructed. 4) By minimizing the entropy-weighted optimization loss function, the salient regions are enhanced with details to generate the final image.

2. The text-to-image generation method based on cognitive heuristic diffusion according to claim 1, characterized in that: In step 1), the text conditional encoding and initial diffusion generation stage uses a pre-trained cross-modal text encoder to perform semantic embedding modeling on the input text prompt; the semantic representation output by the text encoder is injected as a conditional vector into each inverse denoising time step of the diffusion model, which is a global semantic constraint on the generation direction of the latent space. In step 2), when using the image captioning generation model to perform reverse reasoning to generate pseudo captions for the intermediate image, a frozen image captioning generation model is introduced to perform reverse semantic decoding on the intermediate generated image to generate corresponding pseudo caption descriptions, and semantic embedding encoding is performed on the pseudo captions and the original text prompts respectively. In step 2), a semantic consistency loss function is constructed by calculating the similarity between the pseudo-subtitle semantic embedding and the original text semantic embedding, and a semantic reward signal is constructed. In each time step of diffusion reverse denoising, guided by the semantic reward signal, the gradient of the semantic consistency loss on the latent variables is fed back to the latent space to iteratively correct the latent representation. The generation process is always constrained by the semantic reward to suppress semantic drift. In step 2), the semantic consistency loss only participates in backpropagation within the preset mid-to-late time step interval.

3. The text-to-image generation method based on cognitive heuristic diffusion according to claim 1, characterized in that: In step 3), the latent representation output from the semantic alignment stage is used as the initial input, and the second stage reverse denoising process is entered while keeping the original text conditions unchanged. In the cross-attention layer of the diffusion model, the attention distribution map between image spatial location and text tag is extracted, and the attention entropy value corresponding to each spatial location is calculated based on the information entropy theory to quantify the semantic uncertainty of the model in different regions. By normalizing the attention entropy and setting a threshold, high-entropy regions are identified as potential areas of missing details or regions of uncertainty in generation. In step 3), when entering the attention entropy-guided detail optimization stage, the latent representations obtained in steps 1) and 2) are kept as the initial state, and only the optimization objective of the reverse denoising process is adjusted without changing the original text conditional input.

4. The text-to-image generation method based on cognitive heuristic diffusion according to claim 1, characterized in that: In step 4), for the high-entropy region, a position-weighted detailed optimization loss function is constructed in the potential space; By applying weighted constraints to the difference between the current predicted latent representation and the reference latent representation output during the semantic alignment stage, the model is guided to enhance the local structure, texture, and boundary details in high-uncertainty regions while maintaining the overall semantic structure. After inverse denoising is completed, the decoder outputs the final generated image.

5. The text-to-image generation method based on cognitive heuristic diffusion according to claim 1, characterized in that: In step 2), the specific process of generating pseudo-subtitles and calculating semantic consistency rewards includes, Let the target text prompt be denoted as The target text embedding is obtained by encoding it using the CLIP text encoder. Use the current latent variables The input is processed by a denoising network, and a coarse intermediate image is obtained after decoding. ; Image caption generation model using frozen parameters Describe and generate corresponding pseudo-subtitles. ; The pseudo-subtitles were processed using the CLIP text encoder. Encode the pseudo-subtitles to obtain the embedded subtitles. ; Compute target text embedding With pseudo-subtitle embedding The cosine similarity between them is used as a semantic reward signal. , Semantic consistency loss function Defined as .

6. The text-to-image generation method based on cognitive heuristic diffusion according to claim 1, characterized in that: In step 1), the diffusion model introduces a frozen image caption generation model during the reverse denoising process. It performs reverse semantic decoding on the intermediate images generated at each preset time step or key time step to obtain the corresponding pseudo caption description and characterize the linguistic semantic expression of the currently generated image. In step 2), when updating the latent variables using gradients, the semantic consistency loss function is calculated. Relative to the current latent variable gradient ,according to Update the latent variables in the latent space; The update step is performed at each denoising time step generated by diffusion or at a specified time step interval to guide the generated image structure to maintain global consistency with the target text cue through a semantic feedback loop.

7. The text-to-image generation method based on cognitive heuristic diffusion according to claim 5, characterized in that: In step 1), the image caption generation model uses a frozen BLIP-2 model to describe the generated image and obtain its corresponding pseudo captions. ; The text encoder uses a CLIP text encoder that introduces freezing, which processes the original text prompts separately. and pseudo-subtitles Encode; The diffusion model uses the Stable Diffusion model as its basic architecture, and during fine-tuning, it only uses the title text corresponding to the image as a conditional input.

8. The text-to-image generation method based on cognitive heuristic diffusion according to claim 1, characterized in that: In step 3), when calculating the attention entropy, during the denoising time step... From the backbone network (U-Net) of the diffusion model, the first Layer-based extraction of cross-attention maps ; Cross-attention map ,in, and For the spatial dimensions of the feature map, The number of etymological terms in the text prompt; For each spatial location of the image feature map Calculate the entropy of its attention distribution over all text sources. , ; in, Indicates at time step At that time, the first The spatial location is the first Attention weights for each text word origin.

9. The text-to-image generation method based on cognitive heuristic diffusion according to claim 1, characterized in that: In step 3), when identifying salient regions and constructing the entropy-weighted optimization loss function, the calculated attention entropy is... Normalization is performed to obtain the normalized entropy value. ; Set entropy threshold and entropy threshold The value range is set to 0.55 to 0.95; Normalized entropy value Greater than the threshold The region was identified as a salient region with high uncertainty. , ; Calculate position weights based on normalized entropy values. ; Constructing an entropy-weighted optimization loss function To maintain consistency with the diffusion process, ; in, For the latent variables predicted by the current model, As a potential variable for reference; The reference latent variables are derived from the latent representation after the first-stage semantic alignment, processed by a single... The result after noise reduction.

10. The text-to-image generation method based on cognitive heuristic diffusion according to claim 9, characterized in that: In obtaining reference latent variables At that time, the latent variables obtained after the first stage of semantic alignment As input, a denoising network based on a diffusion model is used for single-step denoising prediction, and the predicted denoised latent variable is used as the reference latent variable. .

Images