Feature optimization method in text-to-image generation model

Abstract

The invention belongs to the field of text-to-image generation, and discloses a feature optimization method for a text-to-image generation model, and the method comprises the specific steps: S1, inputting an input text prompt into a pre-trained text encoder for processing, and obtaining an initial text embedded feature representation; s2, designing an embedding distinguishing mechanism based on a geometrical relationship between target object noun embedding features in the initial text embedding features, and optimizing the representation of the embedding features prompted by the original text; s3, inputting the optimized text embedding features as conditional signals into a denoising network, and introducing an attention separation mechanism in a diffusion reverse denoising process to guide image generation; the invention provides a feature optimization method for a text-to-image generation model, and the method comprises an embedded distinguishing mechanism and an attention separation mechanism, so as to better improve the semantic consistency and the overall image quality of the generated image in a multi-object scene.

Description

Technical Field

[0001] This invention belongs to the field of text-to-image generation technology, specifically relating to a feature optimization method in a text-to-image generation model. Background Technology

[0002] Text-to-image synthesis is a crucial research task in the field of multimodal artificial intelligence. Its core significance lies in enhancing the consistency between natural language understanding and visual content generation, providing key technical support for cross-modal semantic alignment and generative modeling. This task not only theoretically promotes the unified modeling and expression of heterogeneous language and visual modalities, but also significantly reduces the cost of visual content creation in practice, serving application scenarios such as intelligent design, digital content production, and art-assisted creation.

[0003] Currently, research on text-to-image generation has mainly evolved from methods based on generative adversarial networks (GANs) to methods based on diffusion models. Early studies mostly employed GANs, using adversarial training between the generator and discriminator to map text semantics to image space, and leveraging staged generation and cross-modal attention mechanisms to improve generation resolution and detail quality. However, these methods have inherent limitations in training stability, sample diversity, and text-image semantic alignment, restricting further improvements in generation performance. With the rise of diffusion models, research focus has gradually shifted to generation paradigms based on progressive denoising. These methods significantly improve generation quality and semantic consistency through stable training mechanisms, and combine strong text encoders and cross-modal attention to achieve more precise text conditional guidance. Building on this, to improve the controllability of generation results, related research has introduced structured prior conditions such as layout, segmentation, and edges to enhance spatial and structural constraints, and achieved personalized and style-consistent generation through model adaptation and efficient parameter fine-tuning. However, these methods typically rely on additional training processes, facing problems of high computational cost and limited generalization ability. Therefore, in recent years, diffusion process control methods that do not require training have gradually emerged. By directly adjusting the distribution of attention features or text embedding feature representations during the inference stage, effective control over semantic consistency and generation behavior of multiple objects can be achieved without updating model parameters, providing a new research direction for efficient and flexible text-to-image generation. Summary of the Invention

[0004] The purpose of this invention is to provide a feature optimization method in a text-to-image generation model, which solves the semantic confusion problem in the geometric relationship of embedded features in existing generation methods.

[0005] This application provides a technical solution: a feature optimization method in a text-to-image generation model, comprising the following steps:

[0006] S1: The input text prompt is fed into a pre-trained text encoder for processing to obtain the initial text embedding feature representation;

[0007] S2: Based on the geometric relationship between the target object noun embeddings in the initial text embedding features, an embedding discrimination mechanism is designed to optimize the embedding representation of the original text prompt;

[0008] S2 includes the following steps:

[0009] S21: Use natural language processing tools to parse the original text prompts, extract sub-prompts containing the target object, and input the sub-prompts into a text encoder to obtain the corresponding sub-prompt embedding representation;

[0010] S22: Based on the original cue embedding and its sub-cue embeddings of the target object, calculate the geometric relationship representation variable between the embedding optimization direction variable and the target object embedding;

[0011] S23: Based on the geometric relationship between the target object embeddings, the original text embedding features are adjusted in the embedding optimization direction to obtain the optimized text embedding feature representation;

[0012] S3: The optimized text embedding features are used as conditional signals to input the denoising network, and an attention separation mechanism is introduced in the diffusion reverse denoising process to guide image generation.

[0013] S3 includes the following steps:

[0014] S31: Using the optimized text embedding features as conditional input, the image generation is guided by a cross-attention mechanism during the denoising process;

[0015] S32: Through the attention separation mechanism, a specific loss function is used to apply attention features to guide the first half of the denoising process, thereby further optimizing image generation;

[0016] The calculation process of embedding the optimization direction variable in S22 specifically includes:

[0017] First, the input prompt is subjected to Stanza segmentation and dependency parsing to identify the target object and its related semantic relationships. Then, clauses containing the target object are constructed based on the syntactic structure to maintain local context consistency. Subsequently, the original prompt and corresponding clauses are input into the CLIP text encoder to obtain the embedded representation of the target object in both global and local contexts. The formula is as follows:

[0018] ;

[0019] in, Represents the target object Text embedding features corresponding to the original prompt context Representing the same target object Embedded within the context of its corresponding clause.

[0020] The calculation process for the geometric relationship representation variables between target object embeddings specifically includes:

[0021] First, CLIP text embedding features of the target object are extracted from the text prompts, and the unnormalized Euclidean distance and normalized cosine similarity between object embeddings are calculated, where the Euclidean distance is linearly normalized. Then, lexical embeddings are constructed from the AOP noun set, and low-dimensional mapping is performed using t-SNE and UMAP respectively. Rank similarity (RS) and shared nearest neighbor overlap rate (SNN-O) are calculated in the low-dimensional space as local similarity scores. Finally, the normalized Euclidean distance, cosine similarity, and the two types of local similarity indicators are linearly fused according to preset weights. The formula is as follows:

[0022] ;

[0023] in, Indicates two target objects. , These are the normalized Euclidean distance and cosine similarity, respectively. This represents the local semantic similarity score calculated based on low-dimensional neighborhood relationships, while These are the weighting parameters set according to the experiment.

[0024] Preferably, the calculation process of the optimized text embedding feature representation in S23 specifically includes:

[0025] The geometric relationship representation variable between the target object embeddings is used as a moderating factor to adaptively update the text embedding features of the target object. Specifically, for the target object in the original prompt... Text embedding features In its corresponding distinguishing vector Applying an offset in the direction yields the optimized text embedding feature representation:

[0026] ;

[0027] in, The target object in the original prompt Text embedding features; This represents the corresponding discriminant vector. Control the offset magnitude; Embedded to the enhanced target object.

[0028] Preferably, the specific loss function of the attention separation mechanism in S32 includes:

[0029] In the reverse denoising stage, the attention feature distribution of the Cross-Attention layer in the diffusion model is first modulated, modeling the attention feature update as a differentiable reconstruction process. By minimizing the reconstruction constraints between the original attention feature map and the modulated attention feature map, and simultaneously evaluating the quality of the attention features of the target object, the Cross-Attention weights are dynamically optimized. The formula is as follows:

[0030] ;

[0031] in, and These represent the original and regulated attention feature maps, respectively. The target token set. The second term serves as a reconstruction constraint, used to limit the adjustment range, while the error term... Used to evaluate the quality of the attentional characteristics of the target object.

[0032] Building upon this, an attention separation mechanism is introduced. For each pair of target objects, a normalized attention feature distribution is extracted from its corresponding Cross-Attention layer, and a weighted discriminative constraint is calculated based on the confusion scores between objects. This constraint promotes the spatial separation of attention feature distributions for different target objects during the attention feature optimization process, and the discriminative strength adaptively adjusts according to the degree of semantic confusion between objects. The formula is as follows:

[0033] ;

[0034] in, Represents the target object Normalized attention feature distribution in the Cross-Attention layer To confuse fractions, This is an adjustment coefficient. This term adaptively adjusts the attention discrimination strength based on the semantic similarity between objects: the more similar the semantics, the stronger the discrimination constraint is applied; conversely, it gradually weakens, thereby avoiding excessive intervention.

[0035] Finally, the attention reconstruction loss and the discriminative regularization term for confusion perception are jointly optimized, and the Cross-Attention weights are updated simultaneously during the reverse denoising process, thereby achieving attention feature regulation and separation in multi-object scenarios. The formula is as follows:

[0036] . Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the feature optimization method in the text-to-image generation model of this application.

[0038] Figure 2 A schematic diagram of the flowchart representing the geometric relationship variables between the embedded objects of this application;

[0039] Figure 3 This is a qualitative comparative analysis diagram of the feature optimization methods in the text-to-image generation model of this application;

[0040] Figure 4 This is a visualization of the results obtained from the feature optimization method in the text-to-image generation model of this application. Detailed Implementation

[0041] The technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0042] like Figure 1 As shown, a feature optimization method in a text-to-image generation model includes the following steps:

[0043] S1: The input text prompt is fed into a pre-trained text encoder for processing to obtain the initial text embedding feature representation;

[0044] S2: Based on the geometric relationship between the target object noun embeddings in the initial text embedding features, an embedding discrimination mechanism is designed to optimize the embedding representation of the original text prompt;

[0045] Step S2 includes:

[0046] S21: Use natural language processing tools to parse the original text prompts, extract sub-prompts containing the target object, and input the sub-prompts into a text encoder to obtain the corresponding sub-prompt embedding representation;

[0047] S22: Based on the original cue embedding and its sub-cue embeddings of the target object, calculate the geometric relationship representation variable between the embedding optimization direction variable and the target object embedding;

[0048] S23: Based on the geometric relationship between the target object embeddings, the original text embedding features are adjusted in the embedding optimization direction to obtain the optimized text embedding feature representation;

[0049] S3: The optimized text embedding features are used as conditional signals to input the denoising network, and an attention separation mechanism is introduced in the diffusion reverse denoising process to guide image generation.

[0050] Step S3 includes:

[0051] S31: Using the optimized text embedding features as conditional input, the image generation is guided by a cross-attention mechanism during the denoising process;

[0052] S32: Based on the attention separation mechanism, a specific loss function is used to apply attention guidance in the first half of the denoising process to further optimize image generation;

[0053] Example

[0054] The feature optimization method in the text-to-image generation model of this application specifically includes the following steps:

[0055] Step 1: Use user input as a description for the image to be generated.

[0056] Step 2: Select CLIP as the image encoder and U-net as the denoising network.

[0057] Step 3: Construct a generative framework based on feature optimization methods in the text-to-image generation model, including a text encoder, embedding discrimination mechanism, attention separation mechanism, denoising network, and VAE decoder. The corresponding processing steps are as follows:

[0058] Based on the user's input, the text encoder obtains the original text embedding features, and an embedding discrimination mechanism is implemented on the original text embedding features.

[0059] like Figure 2 As shown, this application starts from the geometric relationship between the text embedding features of target objects and introduces a geometric relationship representation variable between the embeddings of target objects. This variable is used to explicitly represent the confusion relationship between target objects. First, the text embedding features of each target object are extracted from the CLIP text encoder, and the unnormalized Euclidean distance and normalized cosine similarity between object pairs are calculated in the global semantic space. After linear normalization of the Euclidean distance, it is fused with the cosine similarity according to weights to form a preliminary confusion measure based on global geometric differences. To further characterize the local structure in the high-dimensional semantic space, lexical embeddings are constructed from the created AOP noun set, and low-dimensional representations are constructed using t-SNE (n_components=2, perplexity∈{30,50}, repeated 5 times) and Umap (n_neighbors=30, min_dist=0.1, metric=cosine), respectively. Rank similarity (RS) and shared nearest neighbor overlap rate (SNN-O) are calculated in the low-dimensional space to estimate the semantic proximity of objects in their local neighborhoods. To reduce the impact of randomness, the median of each local similarity index is taken as the final result after multiple runs. The calculation formula is as follows:

[0060] ;

[0061] in, Indicates two target objects. , These are the normalized Euclidean distance and cosine similarity, respectively. This represents the local semantic similarity score calculated based on low-dimensional neighborhood relationships, while These are the weighting parameters set according to the experiment.

[0062] Secondly, this application introduces an embedding optimization direction variable, used to optimize the original embeddings in a directional manner based on the strength of the variable representing the geometric relationship between the target object embeddings. First, the original prompt is segmented and subjected to dependency parsing to identify the target object and construct a clause containing that object. Next, the original prompt and the corresponding clause are input into the CLIP text encoder to obtain the embedding representations of the target object in the global and local contexts. Finally, the difference between the two embeddings is taken as the discriminant vector, used to characterize the offset direction of the target object in the semantic space and enhance the semantic discriminability between objects. The calculation formula is as follows:

[0063] ;

[0064] in, Represents the target object Text embedding features corresponding to the original prompt context Representing the same target object Embedded within the context of its corresponding clause.

[0065] Finally, the original embeddings are optimized using the geometric relationship variables between the target object embeddings and the embedding optimization direction variables.

[0066] ;

[0067] in, The target object in the original prompt Text embedding features; This represents the corresponding discriminant vector. Control the offset magnitude; Embedded to the enhanced target object.

[0068] The optimized text embedding features are used as conditional signals input into the denoising network.

[0069] The entire denoising process consists of fifty time steps. In the first twenty-five time steps, an attention separation mechanism is applied to the Cross-Attention layer of the U-Net structure, which is the last downsampling layer and the first upsampling layer. First, the attention feature modulation in the diffusion model is modeled as an attention feature reconstruction process. Differentiable constraints are applied to the attention feature distribution of the Cross-Attention layer to obtain the modulated attention feature map. The magnitude of attention feature updates is limited by the reconstruction loss, while the quality of the attention features embedded in the target object is evaluated. This process guides the model to strengthen the target semantic response during the generation process by dynamically optimizing the Cross-Attention weights. The formula is as follows:

[0070] ;

[0071] in, and These represent the original and regulated attention feature maps, respectively. The target token set. The second term serves as a reconstruction constraint, used to limit the adjustment range, while the error term... Used to evaluate the quality of the attentional characteristics of the target object.

[0072] Building upon this, an attention separation mechanism is introduced. Specifically, for each pair of target objects, the corresponding normalized attention feature distribution is extracted from the Cross-Attention layer, and a weighted discriminative constraint term is constructed based on the confusion scores between the objects. This allows semantically similar objects to achieve stronger separation in the attention feature space, while weakening the constraint on objects with significant semantic differences. The formula is as follows:

[0073] ;

[0074] in, Represents the target object Normalized attention feature distribution in the Cross-Attention layer To confuse fractions, This is the adjustment coefficient. This term adaptively adjusts the discrimination strength of the attention feature based on the semantic similarity between objects: the more similar the semantics, the stronger the discrimination constraint is applied; conversely, it is gradually weakened, thereby avoiding over-intervention.

[0075] Finally, the attention feature reconstruction loss and the discriminative constraint of confusion perception are jointly optimized, and the Cross-Attention weights are updated simultaneously during the reverse denoising process to achieve attention feature regulation and separation in multi-object scenarios. The formula is as follows:

[0076] ;

[0077] To more intuitively illustrate the effectiveness of the feature optimization method in the text-to-image generation model provided in this application, this application uses three metrics to evaluate the model performance: FID (Fréchet Inception Distance), CLIP score, and target object generation success rate. FID measures the difference between the generated image and the real image distribution to evaluate the overall generation quality; CLIP score is used to evaluate the semantic consistency between the generated image and the input text prompt; and ODS.Rate (Object Detection Success Rate) measures the object rendering integrity in multi-object generation.

[0078] Table 1 below shows a comparison of the FID performance of the feature optimization method in the text-to-image generation model of this application with other retrieval methods on the MS-COCO dataset. The red text indicates the method proposed in this application.

[0079] Table 1. Comparison of FID between the feature optimization method in this application's text-to-image generation model and other methods on the MS-COCO dataset.

[0080]

[0081] The experiments followed a unified procedure: 10,000 unique texts were randomly selected from the MS-COCO dataset as prompts, and each method generated corresponding images and calculated the corresponding FID scores. On three model versions of Stable Diffusion (1.4, 1.5, and 2.1), this application demonstrated stable and competitive FID scores, indicating that it improved semantic controllability without sacrificing overall generation quality, exhibiting good generalization and stability.

[0082] As shown in Table 2 below, the feature optimization method in the text-to-image generation model of this application is compared with other methods on the CLIP scores of the ABC-6K and CC-500 datasets. The red indicates the method proposed in this application.

[0083] Table 2. Comparison of CLIP scores of the feature optimization method in the text-to-image generation model of this application with other methods on the ABC-6K and CC-500 datasets.

[0084]

[0085] Experimental results show that this application achieves high semantic consistency across different CLIP model configurations on both the ABC-6K and CC-500 datasets. Compared to comparative methods, this application achieves superior overall scores across most model versions, including Stable Diffusion V1.4, V1.5, and V2.1, demonstrating stable cross-model consistency. Particularly in the V2.1 model, this application outperforms existing methods in CLIP scores on both the ABC-6K and CC-500 datasets, validating its effectiveness and stability across different datasets and model settings.

[0086] As shown in Table 3 below, the feature optimization method in the text-to-image generation model of this application is compared with other methods in terms of the success rate of target object generation on the CC-500 dataset. The red line indicates the method proposed in this application.

[0087] Table 3 compares the success rate of target object generation on the CC-500 dataset between the feature optimization method in this application's text-to-image generation model and other methods.

[0088]

[0089] This application builds its evaluation based on the CC-500 dataset, primarily because each text prompt in this dataset contains two target categories. Two object detectors (OWL-V2 and GroundingDINO) are used for evaluation on each image. If the detection result can simultaneously identify both target categories in the prompt text within the image, it is considered a successful generation; otherwise, it is considered a failure. The detection confidence threshold is set to two configurations: 0.3 and 0.5. Experimental results show that this application achieves a high overall detection success rate under different confidence levels and detector settings. Under OWL-V2 detection, this application achieves success rates of 81.40%, 78.57%, and 82.89% and 33.06%, 34.88%, and 33.88% respectively across the three model versions, placing it among the best performing methods of its kind. Under GroundingDINO detection, this application consistently demonstrates superior performance, achieving detection success rates of 82.55%, 82.55%, and 85.88% and 56.48%, 59.80%, and 63.95% on the three model versions, respectively. This demonstrates the effectiveness of this application in multi-object generation tasks.

[0090] To further verify the superiority of the feature optimization method in the text-to-image generation model of this application, the following were performed: Figure 3Qualitative comparative analysis was conducted. This application demonstrates significant advantages in scenarios such as multi-object generation, attribute binding, and semantic differentiation. For multi-object prompts, this application can stably generate target objects with the correct number, complete structure, and clear category distinctions, effectively avoiding object missing and feature mixing phenomena. In cross-category combination scenarios, the generated results are semantically clear, and the objects have high recognizability. For example, in the "a photo of a bird and a plane" example, the benchmark model and some comparative methods all showed object missing phenomena in the qualitative results. Magnet generated multiple objects of the same category in this example, while Syntax-Guided Generation generated additional content with weak semantic relevance to the prompt, reflecting that there are still certain challenges in cross-category multi-object prompts. In the attribute binding task, this application can accurately assign attributes such as color to corresponding objects, maintaining good attribute consistency and overall structural clarity. Even in prompts with similar or ambiguous semantics, the generated results can still correctly express the target semantics, avoiding misinterpretation or semantic deviation. For example, in the "a green apple and a blue backpack" example, which emphasizes attribute binding, the baseline model failed to generate complete and clear target objects. While some comparative methods generated corresponding categories, issues such as color attribute mismatch, excessive object count, or low overall recognizability were still observed in this example. In contrast, the apple and backpack generated by this application in this example showed a better match with their corresponding color attributes and maintained a relatively clear structure. Overall, this application demonstrates more stable generation quality and higher semantic consistency in complex multi-object and multi-attribute scenarios, indicating its significant advantages in mitigating object missingness, semantic confusion, and attribute binding errors.

[0091] To demonstrate the performance improvement of this application compared to Stable Diffusion, this application... Figure 4 The visualization results shown were analyzed. The results indicate that after incorporating this application, the attention feature responses corresponding to the target object are more concentrated, and the discriminative power of the relevant semantic embeddings is significantly enhanced, thereby further improving the semantic consistency between the generated results and the text prompts.

[0092] Experimental results on multiple benchmark datasets show that the feature optimization method in the text-to-image generation model of this application outperforms other generation methods in multiple metrics, verifying its effectiveness and reliability in text-to-image tasks.

[0093] Although the content of this application has been described in detail through the above preferred embodiments, it should be understood that the above description should not be considered as a limitation of this application. Various modifications and substitutions to this application will be apparent to those skilled in the art after reading the above content. Therefore, the scope of protection of this application should be defined by the appended claims.

Claims

1. A feature optimization method in a text-to-image generation model, characterized in that, The method includes: S1: The input text prompt is fed into a pre-trained text encoder for processing to obtain the initial text embedding feature representation; S2: Based on the geometric relationship between the target object noun embeddings in the initial text embedding features, an embedding discrimination mechanism is designed to optimize the embedding feature representation of the original text prompt; Step S2 includes: S21: Use natural language processing tools to parse the original text prompts, extract sub-prompts containing the target object, and input the sub-prompts into the text encoder to obtain the corresponding sub-prompt embedding feature representations; S22: Based on the original cue embedding features and sub-cue embedding features of the target object, calculate the optimization direction variable of the embedding features and the geometric relationship representation variable between the embedding features of the target object; S23: Based on the geometric relationship between the embedded features of the target object, the original text embedding features are adjusted in the direction of the embedding feature optimization to obtain the optimized text embedding feature representation; S3: The optimized text embedding features are used as conditional signals to input the denoising network, and an attention separation mechanism is introduced in the diffusion reverse denoising process to guide image generation. Step S3 includes: S31: Using the optimized text embedding features as conditional input, the image generation is guided by a cross-attention mechanism during the denoising process; S32: Through the attention separation mechanism, a specific loss function is used to apply attention guidance in the first half of the denoising process, further optimizing image generation.

2. The feature optimization method in a text-to-image generation model according to claim 1, characterized in that, The pre-trained text encoder is CLIP: In Stable Diffusion, enter text prompts. via CLIP text encoder Mapped to text embedding feature representation: ； This embedded feature, acting as a conditional signal, is injected into the U-Net denoising network through a cross-attention mechanism to guide image generation during the diffusion process.

3. The feature optimization method in a text-to-image generation model according to claim 1, characterized in that, Step S21 includes: S211. Use the natural language processing tool Stanza to parse the original prompt text and obtain sub-prompt text containing a single object noun while preserving the original context; S212. The subtext prompt is fed into the text encoder to obtain the text embedding feature representation of the subtext prompt.

4. The feature optimization method in a text-to-image generation model according to claim 1, characterized in that, Step S22 includes: S221. Calculate the optimization direction variable of the embedding features using the original text embedding feature representation and the text embedding feature representation of the sub-prompt. The formula is as follows: ； in, Represents the target object Text embedding features corresponding to the original prompt context Representing the same target object Embedding features in the context of its corresponding clauses; S222. Calculate the Euclidean distance, cosine similarity, Umap and t-SNE scores between the two object noun embedding features in the original text embedding features to obtain the geometric relationship representation variable CS between the target object embedding features, as shown in the following formula: ;in, Indicates two target objects. , These are the normalized Euclidean distance and cosine similarity, respectively. This represents the local semantic similarity score calculated based on low-dimensional neighborhood relationships, while These are the weighting parameters set according to the experiment.

5. The feature optimization method in a text-to-image generation model according to claim 1, characterized in that, In step S23, the optimized text embedding feature representation is adjusted by adjusting the variables of the original text embedding features in the embedding optimization direction based on the geometric relationship between the target object embedding features, as shown in the following formula: ； in, The target object in the original prompt Text embedding features; The variable represents the geometric relationship between the embedded features of the target object. Control the offset magnitude; Embedded to the enhanced target object.

6. The feature optimization method in a text-to-image generation model according to claim 1, characterized in that, The denoising network is based on Stable Diffusion. The core idea of ​​this model is to map a high-dimensional pixel space to a low-dimensional latent space, and learn the diffusion process in the latent space to significantly reduce computational costs and improve generation efficiency, given an image. It is encoded into a latent representation by a variational autoencoder (VAE). In the potential space, the forward diffusion process gradually moves towards Adding Gaussian noise yields a series of latent variables. The reverse process is handled by a parameterized denoising network. Learning, among which, Conditional text embedding features, typically obtained by a frozen CLIP text encoder, are denoted as follows: Let the vector representation of the conditional cue word be... Denoising network The training objective is to minimize the following loss function: ； in, To add Gaussian noise, and The noise predicted by the denoising network aims to minimize the mean square error between the predicted noise and the actual noise. During the sampling phase, the model extracts Gaussian noise... Initially, a denoising network is used to gradually remove noise, ultimately yielding latent variables that approximate the original data distribution. Then through the VAE decoder Reconstructed into an image.

7. The feature optimization method in a text-to-image generation model according to claim 1, characterized in that, In step S31, the cross-attention mechanism includes: injecting textual conditional information into the feature space of the diffusion model through this mechanism, assuming the text embedding feature matrix output by the CLIP text encoder is... ,in The length of the text sequence. For the text feature dimension, the text embedding feature is projected into a key and a value through a linear transformation, respectively. The diffusion model is in the 1st... The intermediate feature representations generated by U-Net at each time step are denoted as follows: ,in Let be the length of the flattened spatial feature map. This feature is used as a query through linear projection. The calculation process for cross-attention is defined as follows: ； in, Projection matrix For learnable parameters, The feature dimension of the attention head is represented, and the resulting attention matrix is ​​obtained from it. It characterizes the semantic correspondence strength between each location in the image space and the text token.

8. The feature optimization method in a text-to-image generation model according to claim 1, characterized in that, In step S32, the specific loss function is an attention feature regularization term that combines the geometric relationships between objects to represent variables. ； ； in, and These represent the original and regulated attention feature maps, respectively. The target object set is defined by the second term, which serves as a reconstruction constraint to limit the control range, while the error term... Used to evaluate the quality of the attentional characteristics of the target object. Represents the target object Normalized attention feature distribution in the Cross-Attention layer To confuse fractions, As an adjustment coefficient, this term adaptively adjusts the attention discrimination strength based on the semantic similarity between objects: the more semantically similar the objects, the stronger the discrimination constraint is applied; conversely, it gradually weakens, thus avoiding over-intervention. The joint objective loss function for the overall optimization objective is: 。

Images