Text-to-image multi-target generation method and system based on evidence diffusion model

Abstract

The invention provides a text-to-image multi-target generation method and system based on an evidence diffusion model, and relates to the technical field of text graphics, and the method comprises the steps: obtaining a text prompt; inputting the preprocessed text cue into an evidence diffusion model, firstly extracting a noun cross-attention graph, inputting the noun cross-attention graph into a multi-layer perceptron evidence network, mapping the noun cross-attention graph to an evidence space through nonlinear transformation, outputting an evidence value of each pixel about each semantic category, constructing Dirichlet distribution, and obtaining an evidence diffusion model; the Dirichlet distribution is optimized by introducing pixel evidence loss; and converting all noun cross attention maps into a basic belief distribution function by using a D-S evidence theory, calculating a conflict coefficient of an overlapping region of the noun cross attention maps based on the basic belief distribution function, constructing Token conflict loss, and carrying out joint optimization calculation on pixel evidence loss, Token conflict loss and model self loss until an image is generated. According to the invention, the accuracy and logic consistency of multi-target image generation are improved.

Description

Technical Field

[0001] This disclosure relates to the field of text-to-image technology, specifically to a method and system for generating multiple targets from text to images based on an evidence diffusion model. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] In multi-objective co-generation of text-to-image (T2I) diffusion models, the consistency constraint between text prompts and image content is insufficient, especially during denoising. Existing token-supervised diffusion models, while learning token-level consistency through segmentation maps, suffer from segmentation map bias and semantic overlap conflicts (particularly prominent in multi-objective scenarios), leading to a decline in image generation quality.

[0004] Existing methods primarily attempt to address the consistency issue between text and images through two main approaches. One approach intervenes during the inference stage. For example, Composable Diffusion uses multiple diffusion models to generate different objects to construct complex scenes, while MagicMix leverages a pre-trained text conditional diffusion model to fuse semantic concepts. Alternatively, it may intervene in the cross-attention map using Gaussian kernels or linguistic features. The other approach, exemplified by Token Compose, optimizes the cross-attention map based on the segmentation graph, providing dense consistency constraints without increasing inference time.

[0005] However, the existing methods described above still have the following problems: 1) Methods that intervene during the inference stage generally require additional operations, which significantly increases the generation time of a single image and has obvious shortcomings in efficiency.

[0006] 2) The method of optimizing cross-attention maps based on segmentation maps is highly dependent on the accuracy of the segmentation maps. When there is a deviation in the segmentation maps, it will introduce incorrect constraints. Furthermore, when processing multiple targets, optimizing each object individually is prone to semantic overlap and conflict, resulting in semantic confusion of the generated images and seriously affecting the generation quality.

[0007] 3) Existing solutions are often limited in practical application due to efficiency issues, or are limited by the quality of segmentation maps, making it difficult to guarantee generation results in complex scenarios. Summary of the Invention

[0008] To address the aforementioned issues, this disclosure proposes a text-to-image multi-target generation method and system based on an evidence diffusion model. It introduces evidence-based deep learning and Dempster-Shafer theory into the T2I diffusion model to construct a novel evidence diffusion model. Simultaneously, it establishes a dual-dimensional loss function system, fusing classification prediction and uncertainty estimation with pixel evidence loss to suppress overconfidence in unreliable labels. Furthermore, it measures and optimizes semantic conflict factors through token conflict loss, weakening semantic contradictions in overlapping multi-target regions and resolving the problem of poor image generation quality caused by existing methods relying on hard labels from segmentation maps.

[0009] According to some embodiments, the present disclosure adopts the following technical solutions: Text-to-image multi-target generation methods based on evidence diffusion models include: Obtain the text prompt and preprocess it; The preprocessed text prompt is input into the evidence diffusion model to generate and output an image containing semantic information. The preprocessed text prompts are input into the evidence diffusion model. First, the text prompts are denoised and noun cross-attention maps are extracted using a basic U-Net network. Then, the noun cross-attention maps are input into a multilayer perceptron evidence network, which maps them to the evidence space through a nonlinear transformation. The evidence value of each pixel with respect to each semantic category is output. A Dirichlet distribution is constructed using the evidence values, and pixel evidence loss is introduced to optimize the Dirichlet distribution. Then, DS evidence theory is used to convert all noun cross-attention maps into basic belief assignment functions. The conflict coefficient of the overlapping area of ​​the noun cross-attention maps is calculated based on the basic belief assignment functions. The conflict coefficient is used to construct a token conflict loss. The pixel evidence loss, token conflict loss, and the evidence diffusion model's own loss are jointly optimized and calculated until an image containing semantic information is generated.

[0010] According to some embodiments, the present disclosure adopts the following technical solutions: A text-to-image multi-object generation system based on an evidence diffusion model includes: The text acquisition module is used to acquire text prompts and perform preprocessing. The image generation module is used to input the preprocessed text prompts into the evidence diffusion model, generate an image containing semantic information, and output it. The preprocessed text prompts are input into the evidence diffusion model. First, the text prompts are denoised and noun cross-attention maps are extracted using a basic U-Net network. Then, the noun cross-attention maps are input into a multilayer perceptron evidence network, which maps them to the evidence space through a nonlinear transformation. The evidence value of each pixel with respect to each semantic category is output. A Dirichlet distribution is constructed using the evidence values, and pixel evidence loss is introduced to optimize the Dirichlet distribution. Then, DS evidence theory is used to convert all noun cross-attention maps into basic belief assignment functions. The conflict coefficient of the overlapping area of ​​the noun cross-attention maps is calculated based on the basic belief assignment functions. The conflict coefficient is used to construct a token conflict loss. The pixel evidence loss, token conflict loss, and the evidence diffusion model's own loss are jointly optimized and calculated until an image containing semantic information is generated.

[0011] According to some embodiments, the present disclosure adopts the following technical solutions: A computer program product includes a computer program that, when executed by a processor, implements the text-to-image multi-target generation method based on the evidence diffusion model.

[0012] According to some embodiments, the present disclosure adopts the following technical solutions: A non-transitory computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the text-to-image multi-target generation method based on the evidence diffusion model.

[0013] According to some embodiments, the present disclosure adopts the following technical solutions: An electronic device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the text-to-image multi-target generation method based on the evidence diffusion model.

[0014] Compared with the prior art, the beneficial effects of this disclosure are as follows: This disclosed text-to-image multi-target generation method based on the evidence diffusion model introduces evidence deep learning and Dempster-Shafer theory into the T2I diffusion model. Through the framework design of pixel-level uncertainty measurement and token-level conflict quantification, the EviDiff evidence diffusion model is constructed to achieve end-to-end optimization without additional inference overhead. At the same time, a two-dimensional loss function system is constructed. Pixel evidence loss is used to fuse classification prediction and uncertainty estimation to suppress the model's overconfidence in unreliable labels. Token conflict loss is used to measure and optimize the semantic conflict factor based on Dempster combination rules to weaken the semantic contradictions in the overlapping regions of multiple targets. This method solves the technical bottleneck caused by the reliance on hard labels in segmentation maps in existing methods from two dimensions: architectural innovation and loss function design.

[0015] This disclosure presents a text-to-image multi-objective generation method based on an evidence diffusion model. It proposes a pixel evidence loss method, which inputs a noun cross-attention map into a multilayer perceptron evidence network and maps it to the evidence space through a nonlinear transformation. The method outputs the evidence value of each pixel with respect to each semantic category, constructs a Dirichlet distribution using the evidence values, and introduces pixel evidence loss to optimize the Dirichlet distribution. By modeling pixel-level uncertainty (based on the evidence network and the Dirichlet distribution), the method suppresses the model's overconfidence in unreliable segmentation map labels and solves the pixel-level semantic bias problem when supervising image content and segmentation maps.

[0016] This disclosed text-to-image multi-objective generation method based on an evidence diffusion model designs a token conflict loss. It uses DS evidence theory to transform all noun cross-attention maps into a basic belief assignment function. This transformation process preserves the evidence information in the original cross-attention maps while satisfying the mathematical constraints of the BBA function (basic belief assignment function). Based on the basic belief assignment function, the conflict coefficient of the overlapping region of the noun cross-attention map is calculated. The conflict coefficient is used to construct the token conflict loss. By optimizing the conflict factor quantified based on DS evidence theory, the method weakens the semantic overlap contradictions of multi-objectives, solves the semantic overlap conflict problem in multi-objective scenarios, and achieves semantically decoupled attention region learning.

[0017] This disclosed text-to-image multi-target generation method based on the evidence diffusion model further defines two types of loss functions in its token conflict loss module. The intra-object consistency loss uses a spatial attention mechanism to focus the activation energy of the cross-attention map onto sub-regions of the target region, enhancing the relevance and integrity within the same semantic object. The inter-object conflict loss directly penalizes conflict activations in overlapping regions. For each pair of conflicting nouns, the difference in activation values ​​within their overlapping regions is calculated, and semantic competition is resolved by maximizing this difference.

[0018] This disclosed text-to-image multi-target generation method based on an evidence diffusion model uses a token conflict loss derived from a weighted sum of intra-object consistency loss and inter-object conflict loss. By adjusting the weight coefficients, the importance of internal consistency and external conflict resolution can be balanced. During backpropagation, this loss function guides the model to learn semantically decoupled attention regions, ensuring that the cross-attention maps corresponding to different nouns are spatially separated as much as possible. This effectively weakens the semantic contradictions in overlapping regions of multiple targets, improving the accuracy and logical consistency of text-to-image generation.

[0019] This disclosed text-to-image multi-target generation method, based on an evidence diffusion model, employs a loss mechanism to ensure noise reduction and initial text association. It then utilizes pixel evidence loss to enhance pixel-level semantic consistency and token conflict loss to resolve semantic contradictions among multiple targets. The output is an image that integrates the semantic information of multiple targets from the text prompt, achieving high-quality image generation from a noisy initial state to one that is consistent with the text, with a reasonable multi-target layout and resolved semantic conflicts. A full-process evidence supervision mechanism is constructed, combining pixel evidence loss and token conflict loss to form a complete training framework from pixel-level uncertainty constraints to cross-modal semantic conflict resolution, improving the accuracy and logical consistency of multi-target image generation. Attached Figure Description

[0020] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0021] Figure 1 This is a flowchart of a text-to-image multi-target generation method based on an evidence diffusion model according to an embodiment of the present disclosure; Figure 2 This is an overall framework diagram of the text-to-image multi-target generation method based on the evidence diffusion model according to an embodiment of this disclosure; Figure 3 This is a detailed schematic diagram illustrating pixel evidence loss according to an embodiment of the present disclosure; Figure 4 This is a detailed schematic diagram illustrating the token conflict loss in an embodiment of this disclosure. Detailed Implementation

[0022] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0023] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0024] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0025] Terminology Explanation Token level: A token, translated as "feature" in Chinese, is the "smallest semantic unit" when a model processes text; Dempster's combinatorial rule: a key "rule source" in evidence theory, responsible for integrating uncertain information and resolving semantic conflicts; Evidence: "Evidence" refers to the reliability assessment of the model's prediction results after the model is mapped from the input data (noun cross-attention map) through a multilayer perceptron evidence network.

[0026] Example 1 One embodiment of this disclosure provides a text-to-image multi-target generation method based on an evidence diffusion model, the steps of which include: Step 1: Obtain the text prompt and preprocess it; Step 2: Input the preprocessed text prompt into the evidence diffusion model to generate and output an image containing semantic information; The preprocessed text prompts are input into the evidence diffusion model. First, the text prompts are denoised and noun cross-attention maps are extracted using a basic U-Net network. Then, the noun cross-attention maps are input into a multilayer perceptron evidence network, which maps them to the evidence space through a nonlinear transformation. The evidence value of each pixel with respect to each semantic category is output. A Dirichlet distribution is constructed using the evidence values, and pixel evidence loss is introduced to optimize the Dirichlet distribution. Then, DS evidence theory is used to convert all noun cross-attention maps into basic belief assignment functions. The conflict coefficient of the overlapping area of ​​the noun cross-attention maps is calculated based on the basic belief assignment functions. The conflict coefficient is used to construct a token conflict loss. The pixel evidence loss, token conflict loss, and the evidence diffusion model's own loss are jointly optimized and calculated until an image containing semantic information is generated.

[0027] As one embodiment, the text-to-image multi-objective generation method based on the evidence diffusion model disclosed herein inputs random noise and text prompts into the evidence diffusion model, which then undergoes a basic U-Net network feedforward process. The U-Net network, as the core of the diffusion model, gradually restores the noise to a clear image through a denoising process, initially completing the text-to-image generation mapping. However, the generated image needs further semantic alignment with the text to address the consistency issue when combining multiple objects. Therefore, the overall training objective of the evidence diffusion model disclosed herein consists of the diffusion model's own loss, pixel evidence loss, and token conflict loss. By jointly optimizing these three losses, the performance of multi-objective combination generation is effectively improved without increasing inference time overhead, solving the problems existing in current methods. The specific implementation process is as follows: Step 1: Obtain the text prompt and preprocess it; Specifically, the text prompt is the text data that needs to be used to generate the image, such as... Figure 2 As shown, for example, “The transplanted brown teddy bear is sitting on a green chair” is a text sentence containing semantic information.

[0028] First, the acquired text prompts are preprocessed by adding random noise and encoding them into a text prompt condition vector using a text encoder, and then extracting each noun token from the text prompts.

[0029] Step 2: Input the preprocessed text prompt into the evidence diffusion model to generate and output an image containing semantic information; Specifically, the evidence diffusion model is a neural network model based on the U-Net network architecture. The evidence diffusion model includes its own loss, pixel evidence loss, and token conflict loss. After the preprocessed text prompt is input into the evidence diffusion model, the specific processing procedure is as follows: Step 21: Input each noun token from the extracted text prompt into the evidence diffusion model. First, use the basic U-Net network to denoise the text prompt and extract the noun cross-attention map of each noun token to reflect the association strength between the image content and the text noun token. Then, extract the binary segmentation map corresponding to each noun as a pixel-level semantic supervision reference.

[0030] The basic U-Net network serves as the foundational module of the evidence diffusion model, progressively inversely transforming initial random noise into a clear image. In this denoising process, the goal is to minimize the difference between the predicted noise distribution and the actual noise distribution, thereby constructing the evidence diffusion model's own loss. This loss forms the basis for the process of generating a clear image from a disordered noise state to one that is initially associated with the text.

[0031] Step 22: Input the noun cross-attention map into the multilayer perceptron evidence network, map it to the evidence space through nonlinear transformation, output the evidence value of each pixel with respect to each semantic category, construct the Dirichlet distribution using the evidence value, and introduce pixel evidence loss to optimize the Dirichlet distribution; Specifically, attention modeling is based on evidence. Given a noun cross-attention graph. and the corresponding binary segmentation diagram Noun Cross-Note Diagram The evidence is fed into a multilayer perceptron (MLP) evidence network with an activation function (Softplus). To cross-note the nouns Mapped to the evidence space, thus outputting evidence values. For noun cross-attention diagrams Each pixel in Its evidence is defined as Thus, the Dirichlet distribution is obtained. for Dirichlet distribution It is considered the conjugate of the multinomial distribution. It provides the predicted distribution of the segmentation results, defined as follows: (1) in, These are the parameters of the multinomial distribution. It is a high-dimensional polynomial function, and yes The 1-dimensional unit simplex is defined as follows: (2) Subsequently, subjective logic was applied to the optimization of the Dirichlet distribution parameters. A theoretical foundation was established for the quantification of Dirichlet distribution parameters, confidence levels, and uncertainty. Specifically, this was applied to the prediction of noun cross-attention maps. It provides a wealth of beliefs and uncertainties, satisfying: (3) in, and These are the pixels of class c. Cross-attention map probability and pixels Uncertainty regarding pixels. The confidence level and the quality of uncertainty can be expressed as follows: (4) in, , is the Dirichlet intensity. It describes the assigned confidence quality, the higher the value for a pixel. The more evidence you obtain, the better. Conversely, the less evidence you obtain, the fewer pixels you get. The greater the overall uncertainty of the segmentation.

[0032] Considering that, on the simplex, an ideal Dirichlet distribution should concentrate its mass on the vertices corresponding to the true class labels, the distribution parameter should be close to 1 for all incorrect classes and much larger for correct classes. Therefore, it is necessary to design a loss function to guide the model to optimize the Dirichlet parameters in a way that minimizes segmentation uncertainty; that is, to introduce a pixel evidence loss to optimize the Dirichlet distribution.

[0033] Specifically, this disclosure uses cross-entropy loss. Linking the Dirichlet distribution to belief representation in subjective logic.

[0034] Based on the evidence, this cross-entropy loss can be reformulated to reflect the expected prediction under the Dirichlet distribution, enabling the model to learn accurate and uncertainty-aware segmentation, as defined below:

[0035] in, It is a ground truth label, and It is a gamma function. This disclosure also introduces a KullbackLeibler (KL) divergent loss to suppress evidence of incorrect classes while preventing the Dirichlet parameter of the ground fact class from being forced to 1, defined as:

[0036] in, and It is a gamma function.

[0037] Ultimately, pixel evidence loss Cross-entropy loss and The components are as follows: (7) Where n represents each text tag belonging to the nouns within the text prompt, and and These are hyperparameters used to balance these three losses. Cross-entropy loss is employed. To maximize the consistency between the noun cross-attention map and the binary segmentation map, it is defined as follows: (8) in, and These represent the width and height of the noun cross-attention map, respectively.

[0038] As one implementation, the noun cross-attention map reflects the U-Net network's attention regions for different nouns during the generation process. These cross-attention maps are then fed into a multilayer perceptron (MLP) evidence network with a Softplus activation function. A nonlinear transformation maps these maps to the evidence space, outputting the evidence value for each pixel with respect to its semantic category. The choice of the Softplus activation function ensures that the evidence value is always positive, conforming to the physical meaning of the probability distribution.

[0039] For each pixel, its evidence value is used to construct a Dirichlet distribution. Specifically, the parameters of the Dirichlet distribution are obtained by adding 1 to the evidence value for each semantic category. This approach ensures that the distribution parameters incorporate evidence information while preserving a certain degree of prior smoothness. Using subjective logic theory, pixel-level probabilities and uncertainties can be calculated from the Dirichlet distribution: probability represents the confidence that the pixel belongs to a specific semantic category, while uncertainty quantifies the model's certainty about the prediction. Both are closely related to the evidence value; more evidence results in higher confidence and lower uncertainty.

[0040] To guide model optimization, the pixel evidence loss module is designed with a three-part loss structure. The first part is the standard cross-entropy loss, used to constrain the model's prediction accuracy for pixel semantic categories. The second part is the Dirichlet-based cross-entropy loss, which takes into account the difference between the predicted and target distributions, paying particular attention to regions with high uncertainty. The third part is the Kullback-Leibler (KL) divergence loss, which regularizes the evidence distribution by minimizing the KL divergence between the predicted Dirichlet distribution and the uniform distribution, preventing the model from over-confiding in unreliable labels. By balancing these three losses, the model can effectively suppress dependence on low-quality segmentation map labels while maintaining prediction accuracy, achieving more reliable pixel-level consistency constraints.

[0041] Step 23: Then, using DS evidence theory, all noun cross-attention maps are converted into basic belief assignment functions. Based on these basic belief assignment functions, the conflict coefficients of the overlapping regions of the noun cross-attention maps are calculated. The token conflict loss is then constructed using these conflict coefficients, including: All noun cross-attention maps are aggregated using DS evidence theory to quantify inter-noun conflicts and learn semantically decoupled attention regions. During the aggregation process, given a set of noun cross-attention maps, a basic belief assignment function is constructed based on DS evidence theory. For two nouns, the conflict coefficient within the overlapping region is calculated based on this function. An intra-object consistency loss is defined to aggregate the activations of their noun cross-attention maps to a subset of their target region. An inter-object conflict loss is also defined to penalize conflict activations in the overlapping region. Specifically: This disclosure uses Dempster-Shafer Evidence Theory (DST) to aggregate all noun-level cross-attention maps to explicitly quantify inter-noun conflicts and guide the model in learning semantically decoupled attention regions. First, the cross-attention maps corresponding to each noun are normalized using a Softmax function, converting them into Basic Belief Assignment (BBA) functions. Each BBA function represents the confidence distribution of each region in the image belonging to a specific noun semantics, including: In the process of cross-attention graph aggregation, given a set of noun cross-attention graphs Construct the basic belief assignment function based on DST: (9) in, It is confidence evidence belonging to the nth noun category. This represents a learnable factor. This corresponds to the binary segmentation map. For each pair of nouns, the conflict coefficient of the overlapping region of their cross-attention maps is calculated. Specifically, within the overlapping region, the BBA functions of the two nouns are multiplied pixel by pixel, and the resulting product value represents the strength of evidence that the pixel simultaneously supports two semantic categories. These product values ​​are summed over the entire overlapping region to obtain the conflict coefficient of the pair of nouns. The higher the conflict coefficient, the more intense the competition between the two semantic categories in that region, and the more severe the semantic overlap contradiction.

[0042] For two nouns Conflict coefficient in overlapping regions Calculated by the following formula: (10) Furthermore, define an intra-object consistency loss. To aggregate the activations of its cross-attention map to certain sub-regions of its target region, an inter-object conflict loss is defined. To penalize conflict activation in overlapping regions, as follows:

[0043]

[0044] in It is the Sigmoid function. This is the constraint threshold. The final token conflict loss is... and The weighted sum is defined as:

[0045] As one implementation, a conflict quantification and resolution mechanism is constructed based on Dempster-Shafer (DS) evidence theory. First, the cross-attention maps corresponding to each noun are normalized using a Softmax function, transforming them into Basic Belief Assignment (BBA) functions. Each BBA function represents the confidence distribution of each region in the image belonging to the semantics of a specific noun. This transformation process preserves the evidence information in the original cross-attention maps while satisfying the mathematical constraints of the BBA functions.

[0046] For each pair of nouns, the system calculates the conflict coefficient of the overlapping region of their attention maps. Specifically, within the overlapping region, the BBA functions of the two nouns are multiplied pixel by pixel, and the product value represents the strength of evidence that the pixel simultaneously supports two semantic categories. These product values ​​are summed over the entire overlapping region to obtain the conflict coefficient of the pair of nouns. The higher the conflict coefficient, the more intense the competition between the two semantic categories in that region, and the more serious the semantic overlap contradiction.

[0047] To address this issue, the token conflict loss module further defines two types of loss functions. Intra-object consistency loss utilizes a spatial attention mechanism to focus the activation energy of the cross-attention map onto sub-regions of the target region, enhancing the relevance and integrity within the same semantic object. Specifically, the cross-attention map of each noun is segmented into regions, and a loss term is designed to ensure consistent activation values ​​within the same region and differentiate activation values ​​between different regions. Inter-object conflict loss directly penalizes conflicting activations in overlapping regions. For each pair of conflicting nouns, the difference in activation values ​​within their overlapping regions is calculated, and semantic competition is mitigated by maximizing this difference. Specifically, the activation values ​​of the overlapping regions are projected onto the feature subspaces of the two nouns, and the Euclidean distance after projection is calculated as the basis for conflict penalty.

[0048] The final token conflict loss is obtained by weighted summation of intra-object consistency loss and inter-object conflict loss. By adjusting the weight coefficients, the importance of internal consistency and external conflict resolution can be balanced. During backpropagation, this loss function guides the model to learn semantically decoupled attention regions, making the cross-attention maps corresponding to different nouns as spatially separated as possible. This effectively weakens the semantic contradictions in overlapping regions of multiple targets and improves the accuracy and logic of text-to-image generation.

[0049] Step 24: After denoising and initial text association based on the loss guarantee of the diffusion model itself, pixel evidence loss to enhance pixel-level semantic consistency, and token conflict loss to resolve semantic contradictions of multiple targets, output an image that integrates the semantic information of multiple targets in the text prompt, thus completing the generation of a high-quality image from a noisy initial state to one that fits the text, has a reasonable layout of multiple targets, and resolves semantic conflicts.

[0050] The total loss of the evidence diffusion model includes pixel evidence loss, token conflict loss, and the loss of the evidence diffusion model itself. The evidence diffusion model EviDiff is composed of... and Joint optimization, with the following optimization objectives: (14) The evidence diffusion model's own loss is the loss incurred during the denoising process as the model continuously optimizes parameter settings to minimize the difference between the predicted and actual noise distributions. This loss, stemming from minimizing the difference between predicted and actual noise distributions, is the diffusion model's own loss. This loss provides a fundamental guarantee for generating a clear image that is initially associated with the text, transforming the image from a state of disordered noise. Furthermore, it lays a basic foundation for denoising quality by fusing pixel evidence loss and token conflict loss to further optimize multi-target generation.

[0051] As one embodiment, this disclosure proposes a novel diffusion model paradigm—the evidence-driven diffusion model EviDiff—which for the first time integrates evidence-based deep learning and Dempster-Shafer theory into a text-to-image (T2I) diffusion model. Through an innovative framework design of pixel-level uncertainty measurement and token-level conflict quantization, it achieves an end-to-end optimization process without additional inference overhead. This paradigm, by constructing a complete chain system encompassing data acquisition, model training, and image generation, transforms multi-source data such as random noise and text prompts into effective support for model training, enabling the model to accurately generate images in complex semantic scenarios. This provides a new theoretical and practical direction for text-to-image generation technology.

[0052] This disclosure proposes a dual-dimensional loss function system to address image generation challenges at both the pixel and token levels. The pixel evidence loss module integrates classification prediction and uncertainty estimation. Utilizing an MLP evidence network with Softplus activation, it encodes the noun cross-attention map as a visual evidence embedding. A Dirichlet distribution is constructed to obtain pixel-level probabilities and uncertainties, and various loss functions, such as cross-entropy, are combined to effectively suppress the model's over-reliance on unreliable segmentation map labels. The token conflict loss module, based on Dempster's combination rule, transforms the cross-attention map into a basic belief assignment function, quantifies the conflict coefficient between nouns, and significantly weakens semantic contradictions in overlapping multi-object regions through collaborative optimization of intra-object and inter-object losses, thereby improving the logical consistency of image semantics.

[0053] This disclosure proposes an evidence-supervised diffusion model training mechanism that redefines the overall loss composition of the image generation process. By organically combining the diffusion model's own loss, pixel evidence loss, and token conflict loss, a complete evidence learning algorithm is formed. During training, the U-Net network calculates the basic denoising loss based on the difference between predicted and actual noise distributions, ensuring the basic quality of image generation. Pixel evidence loss and token conflict loss strengthen model constraints at the semantic level, enabling generated images to guarantee pixel-level semantic consistency and achieve accurate fusion of multi-target semantics. The synergistic effect of these three mechanisms overcomes the technical bottleneck of traditional methods relying on hard labels in segmentation maps through both architectural innovation and loss function design, driving a comprehensive improvement in the performance of text-to-image diffusion models.

[0054] Example 2 One embodiment of this disclosure provides a text-to-image multi-target generation system based on an evidence diffusion model, comprising: The text acquisition module is used to acquire text prompts and perform preprocessing. The image generation module is used to input the preprocessed text prompts into the evidence diffusion model, generate an image containing semantic information, and output it. The preprocessed text prompts are input into the evidence diffusion model. First, the text prompts are denoised and noun cross-attention maps are extracted using a basic U-Net network. Then, the noun cross-attention maps are input into a multilayer perceptron evidence network, which maps them to the evidence space through a nonlinear transformation. The evidence value of each pixel with respect to each semantic category is output. A Dirichlet distribution is constructed using the evidence values, and pixel evidence loss is introduced to optimize the Dirichlet distribution. Then, DS evidence theory is used to convert all noun cross-attention maps into basic belief assignment functions. The conflict coefficient of the overlapping area of ​​the noun cross-attention maps is calculated based on the basic belief assignment functions. The conflict coefficient is used to construct a token conflict loss. The pixel evidence loss, token conflict loss, and the evidence diffusion model's own loss are jointly optimized and calculated until an image containing semantic information is generated.

[0055] Example 3 One embodiment of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the text-to-image multi-target generation method based on the evidence diffusion model.

[0056] Example 4 One embodiment of this disclosure provides a non-transitory computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the text-to-image multi-target generation method based on the evidence diffusion model.

[0057] Example 5 One embodiment of this disclosure provides an electronic device, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the text-to-image multi-target generation method based on the evidence diffusion model.

[0058] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0059] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0060] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A text-to-image multi-target generation method based on an evidence diffusion model, characterized in that, include: Obtain the text prompt and preprocess it; The preprocessed text prompt is input into the evidence diffusion model to generate and output an image containing semantic information. The preprocessed text prompts are input into the evidence diffusion model. First, the text prompts are denoised and noun cross-attention maps are extracted using a basic U-Net network. Then, the noun cross-attention maps are input into a multilayer perceptron evidence network, which maps them to the evidence space through a nonlinear transformation. The evidence value of each pixel with respect to each semantic category is output. A Dirichlet distribution is constructed using the evidence values, and pixel evidence loss is introduced to optimize the Dirichlet distribution. Then, DS evidence theory is used to convert all noun cross-attention maps into basic belief assignment functions. The conflict coefficient of the overlapping area of ​​the noun cross-attention maps is calculated based on the basic belief assignment functions. The conflict coefficient is used to construct a token conflict loss. The pixel evidence loss, token conflict loss, and the evidence diffusion model's own loss are jointly optimized and calculated until an image containing semantic information is generated.

2. The text-to-image multi-target generation method based on the evidence diffusion model as described in claim 1, characterized in that, The preprocessing process includes adding random noise to the text prompts and encoding it into a text prompt condition vector. The U-Net network, as the basic module of the evidence diffusion model, gradually transforms the initial random noise into a clear image. In this denoising process, the goal is to minimize the difference between the predicted noise distribution and the actual noise distribution. The diffusion model loss itself is constructed, which forms the basis for the process of generating a clear image that is initially associated with the text from a state of disordered noise.

3. The text-to-image multi-target generation method based on the evidence diffusion model as described in claim 1, characterized in that, The text prompts are denoised using a basic U-Net network, and noun cross-attention maps are extracted. These maps are then fed into a multilayer perceptron evidence network with activation functions, mapping them to the evidence space to output evidence values. For each pixel in the noun cross-attention map, the evidence is defined as follows: Thus, the Dirichlet distribution is obtained as follows: for Dirichlet distribution The Dirichlet distribution provides a predicted distribution of the segmentation results, serving as the conjugate of the multinomial distribution.

4. The text-to-image multi-target generation method based on the evidence diffusion model as described in claim 3, characterized in that, The Dirichlet distribution concentrates its quality on the corresponding true class label vertices. For all incorrect classes, the distribution parameter is close to 1, while for correct classes, the distribution parameter is larger. Therefore, pixel evidence loss is introduced to guide the model to optimize the Dirichlet parameters in a way that minimizes segmentation uncertainty. The pixel evidence loss includes cross-entropy loss and KL divergence loss.

5. The text-to-image multi-target generation method based on the evidence diffusion model as described in claim 1, characterized in that, All noun cross-attention maps are aggregated using DS evidence theory to quantify inter-noun conflicts and learn semantically decoupled attention regions. During the noun cross-attention map aggregation process, given a set of noun cross-attention maps, a basic belief assignment function is constructed based on DS evidence theory. For two nouns, the conflict coefficient in the overlapping region is calculated based on the basic belief assignment function, and an intra-object consistency loss is defined to aggregate the activation of their noun cross-attention maps to a partial sub-region of their target region. An inter-object conflict loss is also defined to penalize conflict activation in the overlapping region.

6. The text-to-image multi-target generation method based on the evidence diffusion model as described in claim 1, characterized in that, The token conflict loss is obtained by weighting the intra-object consistency loss and inter-object conflict loss. The total loss of the evidence diffusion model includes pixel evidence loss, token conflict loss and the loss of the evidence diffusion model itself.

7. A text-to-image multi-target generation system based on an evidence diffusion model, characterized in that, include: The text acquisition module is used to acquire text prompts and perform preprocessing. The image generation module is used to input the preprocessed text prompts into the evidence diffusion model, generate an image containing semantic information, and output it. The preprocessed text prompts are input into the evidence diffusion model. First, the text prompts are denoised and noun cross-attention maps are extracted using a basic U-Net network. Then, the noun cross-attention maps are input into a multilayer perceptron evidence network, which maps them to the evidence space through a nonlinear transformation. The evidence value of each pixel with respect to each semantic category is output. A Dirichlet distribution is constructed using the evidence values, and pixel evidence loss is introduced to optimize the Dirichlet distribution. Then, DS evidence theory is used to convert all noun cross-attention maps into basic belief assignment functions. The conflict coefficient of the overlapping area of ​​the noun cross-attention maps is calculated based on the basic belief assignment functions. The conflict coefficient is used to construct a token conflict loss. The pixel evidence loss, token conflict loss, and the evidence diffusion model's own loss are jointly optimized and calculated until an image containing semantic information is generated.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the text-to-image multi-target generation method based on the evidence diffusion model as described in any one of claims 1-6.

9. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the text-to-image multi-target generation method based on the evidence diffusion model as described in any one of claims 1-6.

10. An electronic device, characterized in that, include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the text-to-image multi-target generation method based on the evidence diffusion model as described in any one of claims 1-6.

Images