A method and apparatus for generating target integrity based on a diffusion model with boundary activation penalty.

Abstract

This application relates to a method and apparatus for generating complete targets using a diffusion model based on boundary activation penalty. The method includes: acquiring at least one text prompt and at least one latent variable for the target object based on a pre-trained diffusion model; extracting an attention map during the diffusion generation process; constructing a boundary activation penalty loss function based on the attention map; applying boundary activation penalty to the activation values ​​of image boundary regions to calculate the boundary penalty loss; updating at least one latent variable based on the boundary penalty loss; inputting the at least one updated latent variable into a decoder to generate an output image that completely contains the target object, and obtaining completeness and semantic consistency evaluation results. This solves the problems in related technologies where, due to the failure to fundamentally suppress erroneous responses in boundary regions at the attention activation level, the generated object is truncated or missing at the edges, resulting in incomplete target objects and abnormal boundary region activation when the diffusion model generates images.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and image generation, and particularly relates to a diffusion model target integrity generation method and device based on boundary activation punishment. BACKGROUND

[0002] In recent years, diffusion models have been widely applied in image generation tasks. Such models can generate high-quality natural images by gradually denoising in the latent space, and have shown strong semantic control ability in the text-to-image generation task. The framework represented by the stable diffusion model enables the generation of image content with complex semantics and structures through only text prompts. However, despite the continuous improvement in generation quality, models still generally have the problem of "target incompleteness" in practical applications, that is, the generated subject objects may be missing, broken, cut or incorrectly placed in the edge area. This defect significantly reduces the usability and structural integrity of the generated images.

[0003] In related technologies, there are mainly two improvement paths: one is the layout generation control method, which introduces explicit spatial layout information such as bounding boxes, semantic segmentation maps or sketches, and trains models such as ControlNet, LayoutDiffusion or additional network branches thereof to guide the model to generate corresponding semantic content in the specified area; the second is the improvement of the model design level, such as the cutting coordinate embedding mechanism adopted by Stable Diffusion XL, which encodes the coordinate information of the image cutting area to the model input in the training stage, aiming to make the model learn the corresponding relationship between content and spatial position.

[0004] However, in related technologies, since the layout control needs to retrain the model or extend additional branches, and highly depends on the external input of layout information, it increases the training cost and complexity, and lacks universality and flexibility. Since the cutting coordinate embedding mechanism is limited in the inference stage if the coordinate information is not provided, and it only provides a global position reference, it cannot constrain the attention distribution in the boundary area in a fine-grained manner, and cannot fundamentally suppress the abnormal response of the boundary area from the attention activation level, resulting in the generated objects being truncated or missing at the edge. The target object is not complete, the boundary area activation is abnormal, and it needs to be improved. SUMMARY

[0005] This application provides a method and apparatus for generating object integrity based on a diffusion model with boundary activation penalty, in order to solve the problems in related technologies, such as incomplete target objects and abnormal boundary region activation when generating images, because layout control depends on input layout information and requires model retraining or expansion, and because the cropping coordinate embedding mechanism cannot finely constrain the distribution of attention in the boundary region and fails to fundamentally suppress erroneous responses in the boundary region from the attention activation level, resulting in the generated object being truncated or missing at the edge.

[0006] The first aspect of this application provides a method for generating target integrity based on a diffusion model with boundary activation penalty, comprising the following steps: based on a pre-trained diffusion model, obtaining at least one text prompt and at least one latent variable of the target object, and extracting an attention map in the diffusion generation process according to the at least one text prompt and the at least one latent variable; based on the attention map, constructing a boundary activation penalty loss function, and applying boundary activation penalty to the activation values ​​of the image boundary regions of the attention map based on the boundary activation penalty loss function to calculate a boundary penalty loss; updating the at least one latent variable based on the boundary penalty loss to obtain at least one updated latent variable; inputting the at least one updated latent variable into the decoder of the diffusion model to generate an output image that fully contains the target object, and obtaining an integrity and semantic consistency evaluation result based on the output image.

[0007] Through the above-mentioned technical means, the embodiments of this application can extract the attention map of the generation process from the text prompts and latent variables based on the pre-trained diffusion model. By using the boundary activation penalty loss function, the activation values ​​of the boundary regions of the attention map are targeted to calculate the loss, and the latent variables are updated accordingly. The decoder generates an output image that fully contains the target object. Thus, without changing the diffusion model structure and training weights, by introducing a boundary activation penalty term in the inference stage of the pre-trained diffusion model, the model can effectively suppress erroneous activation in the boundary regions, thereby reducing the situation of target truncation or missing. This achieves a significant improvement in the completeness of target generation, with low computational cost, good compatibility, and strong scalability.

[0008] Optionally, in one embodiment of this application, the expression for the boundary activation penalty loss function is: , in, The average activation value of the image boundary region of the attention map. α is the average activation value of the image interior region of the attention map, and β is the weighting coefficient.

[0009] Through the above-mentioned technical means, the embodiments of this application can construct a boundary activation penalty loss function by comparing the attention activation values ​​of the region within the image and the boundary region. The boundary activation penalty loss function can dynamically suppress boundary activation and strengthen center activation during training or inference, thereby guiding the diffusion model to pay more attention to the main body region of the image.

[0010] Optionally, in one embodiment of this application, the attention map includes a cross-attention map and a self-attention map, and the expression for updating the latent variables is: , in, For the latent variable, For the updated latent variables, For time-related learning rate, For cross-attention-based penalty loss, This is a penalty loss based on self-attention.

[0011] Through the above-mentioned technical means, the embodiments of this application use an expression that includes the original latent variables, time-related learning rate, and penalty loss corresponding to cross attention map and self-attention map to update latent variables. This can comprehensively cover the attention mechanism of semantic association and feature self-dependency in the diffusion generation process, avoid the limitations of single attention dimension constraints, and thus establish semantically consistent and spatially reasonable penalty constraints to maintain the semantic integrity and spatial rationality of the target at the same time, improve the accuracy of latent variable updates, and provide stronger support for generating complete target objects.

[0012] Optionally, in one embodiment of this application, the step of obtaining at least one text prompt and at least one latent variable of the target object based on a pre-trained diffusion model, and extracting an attention map in the diffusion generation process based on the at least one text prompt and the at least one latent variable, includes: encoding the text prompt to obtain a semantic embedding vector of the text prompt; initializing the latent variable in the diffusion generation process and denoising the latent variable based on the semantic embedding vector to generate an intermediate latent variable feature map; and extracting the cross attention map and the self attention map based on the latent variable feature map.

[0013] Through the above-mentioned technical means, the embodiments of this application can encode text prompts to obtain semantic embedding vectors, initialize latent variables in the diffusion generation process, and combine the vectors to denoise and generate intermediate latent variable feature maps. From these, cross-attention maps and self-attention maps are extracted, thereby providing a high-quality data foundation for subsequent accurate identification and punishment of invalid activations in boundary regions. By utilizing the forward computation process and intermediate features of the pre-trained diffusion model, without modifying the model architecture or training process, real-time monitoring and intervention of the generation process are achieved, ensuring the versatility and deployment efficiency of the solution.

[0014] Optionally, in one embodiment of this application, obtaining the integrity and semantic consistency evaluation result based on the output image includes: obtaining the integrity index of the target object based on the output image; and / or, obtaining the generation efficiency index of the target object based on the output image; and / or, obtaining the semantic matching degree index of the target object based on the output image; and / or, obtaining the perceptual quality index of the target object based on the output image; and / or, obtaining the image quality index of the target object based on the output image.

[0015] Through the above-mentioned technical means, the embodiments of this application can provide a multi-dimensional and quantitative evaluation system to comprehensively evaluate the generation performance of the diffusion model from one or more aspects such as target integrity, semantic consistency, visual subjective quality and model inference efficiency. This ensures that the boundary activation penalty mechanism improves integrity without negatively affecting other key performance indicators, thus fully demonstrating the effectiveness and practicality of the method.

[0016] A second aspect of this application provides a target integrity generation apparatus based on a diffusion model with boundary activation penalty, comprising: an acquisition module, configured to acquire at least one text prompt and at least one latent variable of a target object based on a pre-trained diffusion model, and to extract an attention map in the diffusion generation process based on the at least one text prompt and the at least one latent variable; a penalty module, configured to construct a boundary activation penalty loss function based on the attention map, and to apply boundary activation penalty to the activation values ​​of the image boundary regions of the attention map based on the boundary activation penalty loss function to calculate a boundary penalty loss, and to update the at least one latent variable based on the boundary penalty loss to obtain at least one updated latent variable; and a generation module, configured to input the at least one updated latent variable into the decoder of the diffusion model to generate an output image that fully contains the target object, and to obtain an integrity and semantic consistency evaluation result based on the output image.

[0017] Through the above-mentioned technical means, the embodiments of this application can extract the attention map of the generation process from the text prompts and latent variables based on the pre-trained diffusion model. By using the boundary activation penalty loss function, the activation values ​​of the boundary regions of the attention map are targeted to calculate the loss, and the latent variables are updated accordingly. The decoder generates an output image that fully contains the target object. Thus, without changing the diffusion model structure and training weights, by introducing a boundary activation penalty term in the inference stage of the pre-trained diffusion model, the model can effectively suppress erroneous activation in the boundary regions, thereby reducing the situation of target truncation or missing. This achieves a significant improvement in the completeness of target generation, with low computational cost, good compatibility, and strong scalability.

[0018] Optionally, in one embodiment of this application, the expression for the boundary activation penalty loss function is: , in, The average activation value of the image boundary region of the attention map. α is the average activation value of the image interior region of the attention map, and β is the weighting coefficient.

[0019] Through the above-mentioned technical means, the embodiments of this application can construct a boundary activation penalty loss function by comparing the attention activation values ​​of the region within the image and the boundary region. The boundary activation penalty loss function can dynamically suppress boundary activation and strengthen center activation during training or inference, thereby guiding the diffusion model to pay more attention to the main body region of the image.

[0020] Optionally, in one embodiment of this application, the attention map includes a cross-attention map and a self-attention map, and the expression for updating the latent variables is: , in, For the latent variable, For the updated latent variables, For time-related learning rate, For cross-attention-based penalty loss, This is a penalty loss based on self-attention.

[0021] Through the above-mentioned technical means, the embodiments of this application use an expression that includes the original latent variables, time-related learning rate, and penalty loss corresponding to cross attention map and self-attention map to update latent variables. This can comprehensively cover the attention mechanism of semantic association and feature self-dependency in the diffusion generation process, avoid the limitations of single attention dimension constraints, and thus establish semantically consistent and spatially reasonable penalty constraints to maintain the semantic integrity and spatial rationality of the target at the same time, improve the accuracy of latent variable updates, and provide stronger support for generating complete target objects.

[0022] Optionally, in one embodiment of this application, the acquisition module includes: an encoding unit, configured to encode the text prompt to obtain a semantic embedding vector of the text prompt; a denoising unit, configured to initialize latent variables during the diffusion generation process and denoise the latent variables based on the semantic embedding vector to generate an intermediate latent variable feature map; and an extraction unit, configured to extract the cross attention map and the self attention map based on the latent variable feature map.

[0023] Through the above-mentioned technical means, the embodiments of this application can encode text prompts to obtain semantic embedding vectors, initialize latent variables in the diffusion generation process, and combine the vectors to denoise and generate intermediate latent variable feature maps. From these, cross-attention maps and self-attention maps are extracted, thereby providing a high-quality data foundation for subsequent accurate identification and punishment of invalid activations in boundary regions. By utilizing the forward computation process and intermediate features of the pre-trained diffusion model, without modifying the model architecture or training process, real-time monitoring and intervention of the generation process are achieved, ensuring the versatility and deployment efficiency of the solution.

[0024] Optionally, in one embodiment of this application, the generation module includes: an integrity index generation unit, configured to obtain an integrity index of the target object based on the output image; and / or, a generation efficiency index generation unit, configured to obtain a generation efficiency index of the target object based on the output image; and / or, a semantic matching degree index generation unit, configured to obtain a semantic matching degree index of the target object based on the output image; and / or, a perceptual quality index generation unit, configured to obtain a perceptual quality index of the target object based on the output image; and / or, an image quality index generation unit, configured to obtain an image quality index of the target object based on the output image.

[0025] Through the above-mentioned technical means, the embodiments of this application can provide a multi-dimensional and quantitative evaluation system to comprehensively evaluate the generation performance of the diffusion model from one or more aspects such as target integrity, semantic consistency, visual subjective quality and model inference efficiency. This ensures that the boundary activation penalty mechanism improves integrity without negatively affecting other key performance indicators, thus fully demonstrating the effectiveness and practicality of the method.

[0026] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the diffusion model target integrity generation method based on boundary activation penalty as described in the above embodiments.

[0027] A fourth aspect of this application provides a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described diffusion model target integrity generation method based on boundary activation penalty.

[0028] A fifth aspect of this application provides a computer program product that stores a computer program that, when executed by a processor, implements the above-described method for generating target integrity based on a diffusion model with boundary activation penalty.

[0029] This application embodiment can extract the attention map of the generation process from the text prompt and latent variables based on a pre-trained diffusion model. It uses a boundary activation penalty loss function to specifically penalize the activation values ​​of the boundary regions of the attention map to calculate the loss, and updates the latent variables accordingly. The decoder then generates an output image that fully contains the target object. Thus, without changing the diffusion model structure and training weights, by introducing a boundary activation penalty term during the inference stage of the pre-trained diffusion model, it effectively suppresses erroneous activations in the boundary regions, thereby reducing the occurrence of truncated or missing targets. This significantly improves the completeness of target generation, while also offering low computational overhead, good compatibility, and strong scalability. Therefore, it solves the problems in related technologies where layout control depends on input layout information, requiring model retraining or expansion, and where the cropping coordinate embedding mechanism cannot finely constrain the distribution of attention in the boundary regions, failing to fundamentally suppress erroneous responses in the boundary regions at the attention activation level, leading to truncated or missing generated objects and incomplete target objects and abnormal boundary region activation when the diffusion model generates images.

[0030] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0031] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating a method for generating target integrity based on a boundary activation penalty in a diffusion model, according to an embodiment of this application. Figure 2 This is a flowchart of a diffusion model target integrity generation method based on boundary activation penalty according to an embodiment of this application; Figure 3 This is a schematic diagram of a target integrity generation device based on a boundary activation penalty diffusion model according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application.

[0032] Figure label: 10-Target integrity generation device based on boundary activation penalty diffusion model; 100-Acquisition module, 200-Penalty module, 300-Generation module; 401-Memory, 402-Processor, 403-Communication interface. Detailed Implementation

[0033] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0034] The following describes, with reference to the accompanying drawings, a method and apparatus for generating target integrity based on a diffusion model with boundary activation penalty according to embodiments of this application. In response to the aforementioned related technologies, layout control relies on input layout information, requiring model retraining or expansion. Furthermore, the cropping coordinate embedding mechanism cannot finely constrain the distribution of attention in boundary regions, failing to fundamentally suppress erroneous responses in boundary regions at the attention activation level. This results in truncated or missing objects at the edges, leading to incomplete target objects and abnormal boundary region activation when the diffusion model generates images. This application provides a diffusion model target integrity generation method based on boundary activation penalty. In this method, an attention map of the generation process is extracted from text prompts and latent variables based on a pre-trained diffusion model. A boundary activation penalty loss function is used to specifically penalize the boundary region activation values ​​of the attention map to calculate the loss, and the latent variables are updated accordingly. The decoder then generates an output image that fully contains the target object. Thus, without changing the diffusion model structure and training weights, by introducing a boundary activation penalty term during the inference stage of the pre-trained diffusion model, erroneous activation in boundary regions is effectively suppressed, thereby reducing the occurrence of truncated or missing targets. This significantly improves the completeness of target generation, while also offering low computational overhead, good compatibility, and strong scalability. This solves the problems in related technologies, such as the need for retraining or expanding the model due to the dependence of layout control on input layout information, the inability of the cropping coordinate embedding mechanism to finely constrain the distribution of attention in the boundary region, and the failure to fundamentally suppress erroneous responses in the boundary region at the attention activation level, resulting in the generated object being truncated or missing at the edge, and the incomplete target object and abnormal activation in the boundary region when the diffusion model generates images.

[0035] Specifically, Figure 1 This is a flowchart illustrating a diffusion model target integrity generation method based on boundary activation penalty, provided in an embodiment of this application.

[0036] like Figure 1 As shown, the diffusion model target integrity generation method based on boundary activation penalty includes the following steps: In step S101, based on the pre-trained diffusion model, at least one text prompt and at least one latent variable of the target object are obtained, so as to extract the attention map in the diffusion generation process according to the at least one text prompt and at least one latent variable.

[0037] It is understood that the pre-trained diffusion model in the embodiments of this application can be a publicly available pre-trained stable diffusion model, such as Stable Diffusion v1.5, v2.1 or SDXL. The pre-trained diffusion model can be set by those skilled in the art according to the actual situation, and no specific restrictions are imposed here. Latent variables can be understood as the representation of the image in the compressed space. Attention map can be understood as the spatial activation distribution map calculated by the internal attention mechanism of the model during the generation (denoising) process.

[0038] In actual implementation, the embodiments of this application can obtain at least one text prompt and at least one latent variable input based on a pre-trained text-to-image diffusion model, extract the attention map in the generation process, and directly apply it to the pre-trained stable diffusion model without retraining the diffusion model. The goal integrity can be improved through forward inference without changing the original parameters.

[0039] The embodiments of this application can be based on the process of pre-training a diffusion model. By obtaining text prompts and initializing latent variables, attention maps reflecting semantic associations are extracted, thereby non-intrusively obtaining key intermediate representations in the generation process. The structure is simple and does not require retraining of the diffusion model. The attention map directly associates text semantics with the spatial features of latent variables, ensuring that subsequent boundary constraints always revolve around the generation requirements of the target object.

[0040] Optionally, in one embodiment of this application, based on a pre-trained diffusion model, at least one text prompt and at least one latent variable of the target object are obtained, so as to extract an attention map in the diffusion generation process based on at least one text prompt and at least one latent variable, including: encoding the text prompt to obtain a semantic embedding vector of the text prompt; initializing the latent variable in the diffusion generation process, and denoising the latent variable based on the semantic embedding vector to generate an intermediate latent variable feature map; and extracting a cross attention map and a self attention map based on the latent variable feature map.

[0041] It is understood that the semantic embedding vector in the embodiments of this application can be understood as a high-density feature vector formed after the text prompt is encoded; the intermediate latent variable feature map can be understood as the latent space representation containing the image information of the current stage after each step of the denoising process.

[0042] In actual implementation, embodiments of this application can load a publicly available pre-trained stable diffusion model (such as Stable Diffusion v1.5, v2.1 or SDXL) as the generation framework, input text prompts and extract text semantic embedding vectors through CLIP text encoder.

[0043] During the diffusion generation process, latent variables are initialized and denoised based on semantic embedding vectors to generate intermediate latent variable feature maps. Based on these latent variable feature maps, cross-attention maps and self-attention maps are extracted. During the diffusion denoising process, cross-attention maps and self-attention maps at different time steps are extracted to describe the correspondence and spatial distribution between text semantics and image space.

[0044] For example, embodiments of this application can encode the input text prompt. The input text is fed into a CLIP encoder to obtain the corresponding semantic embedding vector e, which has a dimension of d=768. e is then fed into the U-Net network of the diffusion model as a conditional control signal to guide the denoising process of the latent variables.

[0045] During the generation phase, latent variables are randomly initialized. The image was made to conform to a standard normal distribution N(0,I), and the diffusion step count T=50 was set. Then, a stepwise denoising process was performed using the DDIM sampler to gradually reconstruct a high-quality latent image representation from z0. At each time step t (t=1,2,…,T), an intermediate latent variable feature map is generated to extract attention information.

[0046] At each diffusion time step, cross-attention maps are extracted from the intermediate layers of the U-Net network. and self-attention map Cross-attention maps reflect the correspondence between semantic words in text and latent spatial locations in images, while self-attention maps describe the spatial dependencies between latent variables.

[0047] This application embodiment can encode text prompts to obtain semantic embedding vectors, initialize latent variables in the diffusion generation process, and combine the vectors to denoise and generate intermediate latent variable feature maps. From these, cross-attention maps and self-attention maps are extracted, thus providing a high-quality data foundation for subsequent accurate identification and penalty of invalid activations in boundary regions. By utilizing the forward computation process and intermediate features of the pre-trained diffusion model, without modifying the model architecture or training process, real-time monitoring and intervention of the generation process are achieved, ensuring the versatility and deployment efficiency of the solution.

[0048] In step S102, a boundary activation penalty loss function is constructed based on the attention map, and the activation values ​​of the image boundary regions of the attention map are penalized by the boundary activation penalty function to calculate the boundary penalty loss. At least one latent variable is updated based on the boundary penalty loss to obtain at least one updated latent variable.

[0049] It is understood that the boundary activation penalty loss function in this application embodiment can be understood as a tool to quantify the difference in attention activation between the boundary region and the interior region. By penalizing the excessive activation of the boundary region, the model is forced to focus its attention on the target object inside the image. The boundary penalty loss can be understood as the calculation result of the boundary activation penalty loss function, which directly reflects the degree of unreasonableness of the boundary activation. The latent variable update can be understood as the process of adjusting the latent variables using loss backpropagation. By correcting the latent variable features, the subsequent generation process is guided to avoid target incompleteness.

[0050] In practical implementation, this application embodiment can identify semantic regions related to the target object through cross-attention maps, and select key points in the smoothed attention map using clustering algorithms, thereby locating self-attention features related to the spatial layout of the target object. This application embodiment can also impose penalty constraints on the attention activation values ​​of image edge regions in the early stages of image generation. By constructing a boundary penalty loss function, the probability of the model generating targets at image edges is reduced, guiding the generated targets to concentrate in the image center region.

[0051] Furthermore, in each diffusion time step, the boundary activation penalty loss can be calculated and the latent variable representation optimized through backpropagation to obtain an updated latent space representation. The latent space representation under the penalty constraint is then updated through gradient optimization, achieving spatial correction of the target without retraining the model. The penalty constraint is applied only in the first T1 early time steps of the total number of diffusion steps T, where T1 satisfies T>T1>0, preferably T1=45, and when T=50.

[0052] For example, in embodiments of this application, after extracting the cross-attention map, it can be Gaussian smoothed to eliminate extreme noise, and K key point locations can be selected based on a clustering algorithm, and their corresponding self-attention maps can be averaged to form a unified spatial reference map, wherein K is preferably 10.

[0053] Specifically, based on the cross-attention map and self-attention map of the diffusion model at the current time step, the average activation values ​​of the image boundary region and the internal region are calculated respectively, and a boundary activation penalty loss function is constructed.

[0054] The formula for calculating cross-attention is: , The formula for calculating self-attention is: , right Gaussian smoothing is used to reduce noise interference, and K activation centers (K=10) are selected by K-Means clustering to extract the corresponding activation centers. The fusion attention map is obtained by averaging the regions. .

[0055] based on Construct a boundary activation penalty term. Divide the latent space into boundary regions. With internal area Calculate the average activation value of both. and .

[0056] Furthermore, a boundary activation penalty loss function is defined. This function dynamically suppresses boundary activation and strengthens center activation during training or inference, thereby guiding the model to focus more on the main image region. This application's embodiments are based on the expression for latent variable updates. Gradient updates are performed, latent variable optimization is executed in the early stages of diffusion (the first 5 steps), and the remaining variables are kept fixed in the last 45 steps to ensure image detail stability.

[0057] The embodiments of this application can construct a boundary activation penalty loss function based on the attention map, penalize the activation values ​​of the image boundary region, reduce the probability of generating objects in the boundary region, calculate the penalty loss and update the latent variables. The latent variable update directly affects the core carrier of the generation, which can directly and continuously correct the model's tendency to generate target objects on the edge, promote the integrity of objects from the root of the generation process, and achieve lightweight optimization without retraining the model parameters.

[0058] Optionally, in one embodiment of this application, the expression for the boundary activation penalty loss function is: , in, The average activation value of the image boundary region in the attention map. α represents the average activation value of the image's internal region in the attention map, and β represents the weighting coefficients.

[0059] In actual implementation, the embodiments of this application can calculate the average activation values ​​of the image boundary region and the internal region based on the cross-attention map and self-attention map of the diffusion model at the current time step, and construct the boundary activation penalty loss function.

[0060] Specifically, a boundary activation penalty loss function is constructed by comparing the attention activation values ​​of regions within the image with those of the boundary regions. The loss function L is defined as follows: ， in, The average activation value of the image boundary region. α represents the average activation value of the region inside the image, and β are weighting coefficients, preferably α=1.0 and β=2.0. This penalty term suppresses high activation in the boundary regions, encouraging the model to generate complete objects in the center of the image.

[0061] This application embodiment can construct a boundary activation penalty loss function by comparing the attention activation values ​​of regions within an image with those of boundary regions. Through the boundary activation penalty loss function, boundary activation is dynamically suppressed and center activation is strengthened during training or inference, thereby guiding the diffusion model to pay more attention to the main body region of the image.

[0062] Optionally, in one embodiment of this application, the attention map includes a cross-attention map and a self-attention map, and the expression for updating the latent variables is: , in, As a latent variable, For the updated latent variables, For time-related learning rate, For cross-attention-based penalty loss, This is a penalty loss based on self-attention.

[0063] In practical implementation, this embodiment of the application can suppress high activation in boundary regions through boundary penalty loss, encouraging the model to generate complete objects in the image center. The latent variable update rule is defined as: , in, For time-related learning rate, and The penalties are based on cross-attention and self-attention, respectively. These two methods are combined to establish semantically consistent and spatially reasonable penalty constraints, simultaneously maintaining the semantic integrity and spatial rationality of the target. Latent variables are adjusted through gradient descent. Suppress unreasonable boundary activation.

[0064] The embodiments of this application use an expression that includes the original latent variables, time-related learning rate, and penalty loss corresponding to cross-attention maps and self-attention maps to update latent variables. This can comprehensively cover the attention mechanism of semantic association and feature self-dependency in the diffusion generation process, avoid the limitations of single attention dimension constraints, and thus establish semantically consistent and spatially reasonable penalty constraints to maintain the semantic integrity and spatial rationality of the target at the same time, improve the accuracy of latent variable updates, and provide stronger support for generating complete target objects.

[0065] In step S103, at least one updated latent variable is input into the decoder of the diffusion model to generate an output image that fully contains the target object, and the completeness and semantic consistency evaluation results are obtained based on the output image.

[0066] It is understood that the decoder of the diffusion model in the embodiments of this application can be understood as the component in the diffusion model responsible for mapping the optimized low-dimensional latent variables back to the high-dimensional pixel space, such as the VAE decoder.

[0067] In actual implementation, the embodiments of this application can input the updated latent variables into the decoder part of the diffusion model to generate an output image containing the complete target object, evaluate the completeness and semantic consistency of the generated result, and obtain an image output with target completeness optimized.

[0068] For example, embodiments of this application can, after completing all diffusion steps, determine the final latent variable. The input is processed by the VAE decoder to generate the corresponding image output. In the experiment, the resolution was set to 512×512, the batch size to 1, and the sampler to DDIM mode. The generated image shows that, after boundary penalty constraints, the subject is more complete, and the transition between the background and edges is more natural.

[0069] In the embodiments of this application, the calculation process is implemented in the PyTorch framework, and the functions and calculation processes used are all publicly available deep learning standard modules.

[0070] The image output obtained through the diffusion model is used as the test result to evaluate the integrity and semantic consistency of the generated image. The latent variables optimized by boundary activation penalty are input into the decoder to generate the final image, and the generated image is compared and analyzed with the result generated by the original diffusion model.

[0071] This application embodiment can input the updated latent variables into the decoder to generate an image, thereby reducing the target incompleteness problem from the root and improving structural integrity. The integrity and semantic consistency evaluation results are obtained through multi-dimensional indicators, intuitively demonstrating the improvement effect brought by the boundary activation penalty and improving the adaptability of the method.

[0072] Optionally, in one embodiment of this application, obtaining the integrity and semantic consistency evaluation results based on the output image includes: obtaining the integrity index of the target object based on the output image; and / or, obtaining the generation efficiency index of the target object based on the output image; and / or, obtaining the semantic matching degree index of the target object based on the output image; and / or, obtaining the perceptual quality index of the target object based on the output image; and / or, obtaining the image quality index of the target object based on the output image.

[0073] It is understood that, in the embodiments of this application, the integrity index can be understood as an index that measures whether the main target in the generated image is cropped, missing, or incomplete; the generation efficiency index can be understood as an index that measures the average inference time required by the model when generating a single image, reflecting the computational complexity and running efficiency of the model; the semantic matching index can be understood as an index that measures the degree of semantic matching between the generated image and the input text description; the perceptual quality index can be understood as an index of the visual quality assessment model trained based on human subjective preferences, used to measure the clarity, lighting naturalness, and content aesthetic features of the generated image; and the image quality index can be understood as an index that measures the overall objective quality of the generated image.

[0074] For example, embodiments of this application can evaluate the performance of the generative model using five mainstream metrics, including Target Integrity Indicator (HOIR), Generation Efficiency Indicator (Time Cost), Semantic Matching Indicator (PickScore), Perceptual Quality Indicator (HPSv2), and Image Quality Indicator (CLIP-IQA). These metrics can comprehensively evaluate the generative performance of the diffusion model from multiple aspects, such as target integrity, semantic consistency, visual subjective quality, and model inference efficiency.

[0075] Among them, the Target Integrity Index (HOIR), judged manually, measures whether the main target in the generated image is cropped, missing, or incomplete. It is the core indicator that best reflects the model's ability to generate structural content. If the generated image contains missing parts of the main target, truncated edges, or morphological distortions, it is considered an incomplete sample. The HOIR is calculated as the proportion of incomplete samples to all generated samples, defined as: , in, This indicates the number of objects that were cropped, missing, or incomplete in the generated result. This indicates the total number of samples generated. The lower the value of this indicator, the more complete the generated target is.

[0076] The generation efficiency metric (Time Cost) measures the average inference time required by the model to generate a single image, reflecting the model's computational complexity and operational efficiency. This metric is closely related to the model's real-time performance during deployment. A lower value indicates faster inference speed and higher utilization of computational resources. In practical experiments, a stable time estimate is obtained by averaging multiple samples.

[0077] The semantic matching metric (PickScore) measures the degree of semantic matching between the generated image and the input text description. It calculates the similarity between image and text features based on a multimodal semantic embedding model (such as CLIP), reflecting whether the model correctly understands and implements the semantic information of the input prompt. A higher PickScore indicates a more consistent generated image with the text content and a stronger semantic reconstruction capability.

[0078] The Perceptual Quality Metric (HPSv2) is a visual quality assessment model trained based on human subjective preferences. It measures the sharpness, lighting naturalness, and content aesthetic features of generated images. This metric reflects the realism and naturalness of the model's generated images at the visual level; a higher score indicates that the generated image is closer to a high-quality image as perceived by humans.

[0079] The CLIP-IQA (Collision Image Quality Index) is based on the CLIP model of image feature representation and is used to measure the overall objective quality of generated images. It models the consistency of image feature distribution, reflecting the comprehensive performance of the generated image in terms of sharpness, detail preservation, and structural stability. A higher value for this index indicates better overall visual quality.

[0080] In practice, an automated software evaluation process can be used to programmatically calculate the average value of the above indicators for each sample.

[0081] To verify the beneficial effects of the embodiments of this application, Stable Diffusion v1.5, Stable Diffusion v2.1, SDXL, and the LayoutDiffusion series of layout generation models were selected as comparison objects. Images were generated under the same set of prompts, and the generation results were quantitatively compared. The experiment used MS-COCO, CelebA-HQ, and a subset of OpenImages as benchmark datasets. 1000 samples were generated for each model for statistical analysis. The experimental results are shown in Table 1, where Table 1 is the dataset attribute table.

[0082] Table 1 Dataset Attribute Table

[0083] As can be seen from Table 1, the embodiments of this application (Ours) significantly outperform other methods in the Target Integrity Index (HOIR), which is only 17.3%, about 28.4 percentage points lower than the basic Stable Diffusion, and nearly 10 percentage points lower than the current best layout constraint method SLD. This shows that the boundary activation penalty mechanism can effectively reduce the situation where the target is truncated or incomplete.

[0084] In terms of generation efficiency (Time Cost), the average inference time of the embodiments of this application is 5.75 s, which is similar to the 5.51 s of the base model and significantly better than most models that require layout constraints or additional inputs (such as Layout Guidance's 24.02 s). This indicates that the boundary activation constraints introduced in the inference stage of the embodiments of this application add almost no additional computational burden.

[0085] In terms of semantic consistency (PickScore) and perceptual quality (HPSv2) metrics, the embodiments of this application also demonstrate overall advantages, with a PickScore of 23.41 and an HPSv2 score of 25.66, both being the highest values ​​among all comparison methods. This indicates that the boundary penalty term not only improves the structural integrity of the target but also maintains the naturalness and semantic coherence of the image.

[0086] On the image quality index (CLIP-IQA), the embodiment of this application achieved a score of 0.703, which is close to the baseline of StableDiffusion (0.714) and slightly better than other complex structure models (such as GLIGEN's 0.672 and LayoutLLM's 0.594), indicating that the embodiment of this application improves the integrity of the target without sacrificing the overall image quality.

[0087] Further experiments show that when statistics are performed on different cue types (such as people, animals, vehicles, and indoor scenes), the average target completeness rate of the embodiments in this application is improved by about 26% compared with the base model, and the effect is more significant in semantically complex scenes. This indicates that the boundary activation penalty can adaptively adjust the model's attention distribution, making the spatial information in the diffusion process more balanced.

[0088] The embodiments of this application can provide a multi-dimensional and quantitative evaluation system to comprehensively evaluate the generation performance of the diffusion model from one or more aspects such as target integrity, semantic consistency, visual subjective quality and model inference efficiency. This ensures that the boundary activation penalty mechanism improves integrity without negatively impacting other key performance indicators, thus fully demonstrating the effectiveness and practicality of the method.

[0089] The embodiments of this application can provide a multi-dimensional and quantitative evaluation system to comprehensively evaluate the generation performance of the diffusion model from one or more aspects such as target integrity, semantic consistency, visual subjective quality and model inference efficiency. This ensures that the boundary activation penalty mechanism improves integrity without negatively impacting other key performance indicators, thus fully demonstrating the effectiveness and practicality of the method.

[0090] This application's embodiments improve target integrity while maintaining the semantic consistency and detail fidelity of the original image, with a computational cost of less than 8 seconds per image. It can be used in conjunction with different versions of stable diffusion models (including SDv1.5, SDv2.1, and SDXL), reducing the target incompleteness rate from 45.7% to 17.3% under the same generation conditions. It can be applied to multimodal generation scenarios such as text-to-image generation, layout-controlled generation, and video frame generation to improve the integrity and structural rationality of the generated target.

[0091] This application embodiment employs the DDIM sampler throughout the inference process to improve generation stability and repeatability. This application embodiment can be implemented using the Python platform based on the PyTorch deep learning framework, with the Stable Diffusion model serving as the foundational structure of the generative network. The U-Net network is the core denoising module of the diffusion model, the CLIP text encoder is responsible for encoding the input text prompts into semantic vectors, and the VAE decoder is used to decode the latent space vectors into images.

[0092] Specifically, it can be combined with Figure 2 As shown, the working principle of the diffusion model target integrity generation method based on boundary activation penalty in this application is explained in detail with a specific embodiment.

[0093] like Figure 2 As shown, this embodiment of the application can extract attention maps during the diffusion generation process based on the latent variables of the image at time t. These attention maps include self-attention maps and cross-attention maps. The cross-attention map reflects the correspondence between semantic words in the text and latent spatial locations in the image, while the self-attention map describes the spatial dependencies between latent variables. The two are combined to establish semantically consistent and spatially reasonable penalty constraints. The cross-attention map identifies semantic regions related to the target object, and a clustering algorithm is used to filter key points in the smoothed attention map, thereby locating self-attention features related to the spatial layout of the target object.

[0094] Furthermore, in each diffusion time step, a penalty constraint is applied to the attention activation values ​​of the image edge region. The average activation values ​​of the boundary and internal regions in the two types of attention maps are calculated and substituted into the boundary activation penalty loss function to aggregate and calculate the boundary penalty loss. Based on the boundary penalty loss, the latent variable representation is updated by gradient through backpropagation to obtain the updated latent space representation, i.e., the image latent variables at time t-1.

[0095] The target integrity generation method based on boundary activation penalty proposed in this application can extract the attention map of the generation process from the text prompt and latent variables based on the pre-trained diffusion model. A boundary activation penalty loss function is used to apply targeted penalties to the activation values ​​of the boundary regions of the attention map to calculate the loss, and the latent variables are updated accordingly. The decoder then generates an output image that fully contains the target object. Thus, without changing the diffusion model structure and training weights, by introducing a boundary activation penalty term during the inference stage of the pre-trained diffusion model, erroneous activation in the boundary regions is effectively suppressed, thereby reducing the occurrence of truncated or missing targets. This significantly improves the completeness of target generation, and features low computational cost, good compatibility, and strong scalability. Therefore, it solves the problems in related technologies where layout control depends on input layout information, requiring model retraining or expansion, and where the cropping coordinate embedding mechanism cannot finely constrain the distribution of attention in the boundary regions, failing to fundamentally suppress erroneous responses in the boundary regions at the attention activation level, resulting in truncated or missing generated objects and incomplete target objects and abnormal boundary region activation when the diffusion model generates images.

[0096] Next, referring to the accompanying drawings, a diffusion model target integrity generation apparatus based on boundary activation penalty proposed according to an embodiment of this application is described.

[0097] Figure 3 This is a schematic diagram of the structure of the diffusion model target integrity generation device based on boundary activation penalty according to an embodiment of this application.

[0098] like Figure 3 As shown, the diffusion model target integrity generation device 10 based on boundary activation penalty includes: an acquisition module 100, a penalty module 200, and a generation module 300.

[0099] The acquisition module 100 is used to acquire at least one text prompt and at least one latent variable of the target object based on a pre-trained diffusion model, so as to extract the attention map in the diffusion generation process based on the at least one text prompt and at least one latent variable.

[0100] The penalty module 200 is used to construct a boundary activation penalty loss function based on the attention map, and to apply boundary activation penalty to the activation values ​​of the image boundary regions of the attention map based on the boundary activation penalty loss function to calculate the boundary penalty loss. Based on the boundary penalty loss, at least one latent variable is updated to obtain at least one updated latent variable.

[0101] The generation module 300 is used to input at least one updated latent variable into the decoder of the diffusion model to generate a complete output image containing the target object, and to obtain the completeness and semantic consistency evaluation results based on the output image.

[0102] Optionally, in one embodiment of this application, the expression for the boundary activation penalty loss function is: , in, The average activation value of the image boundary region in the attention map. α represents the average activation value of the image's internal region in the attention map, and β represents the weighting coefficients.

[0103] Optionally, in one embodiment of this application, the attention map includes a cross-attention map and a self-attention map, and the expression for updating the latent variables is: , in, As a latent variable, For the updated latent variables, For time-related learning rate, For cross-attention-based penalty loss, This is a penalty loss based on self-attention.

[0104] Optionally, in one embodiment of this application, the acquisition module 100 includes an encoding unit, a denoising unit, and an extraction unit.

[0105] The encoding unit is used to encode the text prompt to obtain the semantic embedding vector of the text prompt.

[0106] The denoising unit is used to initialize latent variables during the diffusion generation process and denoise the latent variables based on the semantic embedding vector to generate intermediate latent variable feature maps.

[0107] The extraction unit is used to extract cross-attention maps and self-attention maps based on latent variable feature maps.

[0108] Optionally, in one embodiment of this application, the generation module 300 includes: an integrity index generation unit, a generation efficiency index generation unit, a semantic matching degree index generation unit, a fourth evaluation unit, and a fifth evaluation unit.

[0109] The integrity index generation unit is used to obtain the integrity index of the target object based on the output image.

[0110] And / or, a generation efficiency index generation unit, used to obtain the generation efficiency index of the target object based on the output image.

[0111] And / or, a semantic matching index generation unit, used to obtain the semantic matching index of the target object based on the output image.

[0112] And / or, a perceptual quality index generation unit, used to obtain the perceptual quality index of the target object based on the output image.

[0113] And / or, an image quality index generation unit is used to obtain the image quality index of the target object based on the output image. It should be noted that the foregoing explanation of the embodiment of the diffusion model target integrity generation method based on boundary activation penalty also applies to the diffusion model target integrity generation apparatus based on boundary activation penalty of this embodiment, and will not be repeated here.

[0114] The target integrity generation device based on boundary activation penalty proposed in this application can extract the attention map of the generation process from text prompts and latent variables based on a pre-trained diffusion model. It uses a boundary activation penalty loss function to apply targeted penalties to the activation values ​​of the boundary regions of the attention map to calculate the loss, and updates the latent variables accordingly. The decoder then generates an output image that fully contains the target object. Thus, without changing the diffusion model structure and training weights, by introducing a boundary activation penalty term during the inference stage of the pre-trained diffusion model, it effectively suppresses erroneous activations in the boundary regions, thereby reducing the occurrence of truncated or missing targets. This significantly improves the completeness of target generation, while also offering low computational overhead, good compatibility, and strong scalability. Therefore, it solves the problems in related technologies where layout control depends on input layout information, requiring model retraining or expansion, and where the cropping coordinate embedding mechanism cannot finely constrain the distribution of attention in the boundary regions, failing to fundamentally suppress erroneous responses in the boundary regions at the attention activation level. This results in truncated or missing generated objects, leading to incomplete target objects and abnormal boundary region activation when the diffusion model generates images.

[0115] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0116] When the processor 402 executes the program, it implements the diffusion model target integrity generation method based on boundary activation penalty provided in the above embodiments.

[0117] Furthermore, electronic devices also include: Communication interface 403 is used for communication between memory 401 and processor 402.

[0118] The memory 401 is used to store computer programs that can run on the processor 402.

[0119] Memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0120] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0121] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0122] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0123] This application also provides a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described diffusion model target integrity generation method based on boundary activation penalty.

[0124] This application also provides a computer program product on which a computer program is stored, which, when executed by a processor, implements the above-described method for generating target integrity based on a diffusion model with boundary activation penalty.

[0125] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0126] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0127] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0128] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0129] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0130] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0131] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0132] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for generating target integrity based on a diffusion model with boundary activation penalty, characterized in that, Includes the following steps: Based on a pre-trained diffusion model, at least one text prompt and at least one latent variable of the target object are obtained, so as to extract an attention map in the diffusion generation process according to the at least one text prompt and the at least one latent variable; Based on the attention map, a boundary activation penalty loss function is constructed, and based on the boundary activation penalty loss function, the activation values ​​of the image boundary regions of the attention map are penalized by boundary activation to calculate the boundary penalty loss. Based on the boundary penalty loss, the at least one latent variable is updated to obtain at least one updated latent variable. The at least one updated latent variable is input into the decoder of the diffusion model to generate an output image that fully contains the target object, and the completeness and semantic consistency evaluation results are obtained based on the output image.

2. The method according to claim 1, characterized in that, The expression for the boundary activation penalty loss function is: ， in, The average activation value of the image boundary region of the attention map. α is the average activation value of the image interior region of the attention map, and β is the weighting coefficient.

3. The method according to claim 1, characterized in that, in, The attention map includes a cross-attention map and a self-attention map, and the expression for updating the latent variables is: ， in, For the latent variable, For the updated latent variables, For time-related learning rate, For cross-attention-based penalty loss, This is a penalty loss based on self-attention.

4. The method according to claim 3, characterized in that, The pre-trained diffusion model acquires at least one text prompt and at least one latent variable for the target object, and extracts an attention map in the diffusion generation process based on the at least one text prompt and the at least one latent variable, including: The text prompt is encoded to obtain its semantic embedding vector; During the diffusion generation process, latent variables are initialized and denoised based on the semantic embedding vector to generate intermediate latent variable feature maps; Based on the latent variable feature map, the cross attention map and the self attention map are extracted.

5. The method according to claim 1, characterized in that, The completeness and semantic consistency evaluation results obtained based on the output image include: The integrity index of the target object is obtained based on the output image; And / or, obtain the generation efficiency index of the target object based on the output image; And / or, obtain the semantic matching degree index of the target object based on the output image; And / or, obtain the perceived quality index of the target object based on the output image; And / or, obtain the image quality index of the target object based on the output image.

6. A diffusion model target integrity generation device based on boundary activation penalty, characterized in that, include: The acquisition module is used to acquire at least one text prompt and at least one latent variable of the target object based on a pre-trained diffusion model, so as to extract the attention map in the diffusion generation process according to the at least one text prompt and the at least one latent variable; The penalty module is used to construct a boundary activation penalty loss function based on the attention map, and to apply boundary activation penalty to the activation values ​​of the image boundary regions of the attention map based on the boundary activation penalty loss function to calculate the boundary penalty loss, and to update the at least one latent variable based on the boundary penalty loss to obtain at least one updated latent variable. A generation module is used to input the at least one updated latent variable into the decoder of the diffusion model to generate a complete output image containing the target object, and to obtain a completeness and semantic consistency evaluation result based on the output image.

7. The apparatus according to claim 6, characterized in that, The expression for the boundary activation penalty loss function is: ， in, The average activation value of the image boundary region of the attention map. α is the average activation value of the image interior region of the attention map, and β is the weighting coefficient.

8. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and capable of running on the processor, the processor executing the program to implement the diffusion model target integrity generation method based on boundary activation penalty as described in any one of claims 1-5.

9. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the diffusion model target integrity generation method based on boundary activation penalty as described in any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that, The computer program is executed to implement the diffusion model target integrity generation method based on boundary activation penalty as described in any one of claims 1-5.

Images