Layout guide-based video generation object quantity control method

Abstract

The application discloses a kind of video generation object quantity control methods based on layout guide without training, the method includes: using the self-attention and cross-attention graph in the video pre-generation process, screening best attention head and constructing initial semantic layout to identify object quantity;According to the difference between current quantity and prompt word target quantity, by minimizing cost function, add and delete correction is carried out to layout graph at instance level;Using the corrected layout graph as guide signal, modulate cross-attention mechanism to regenerate video.The application uses the "first identification-then guide" paradigm without training, effectively solves the counting error problem caused by weak semantics and instance ambiguity of existing text-to-video model, while maintaining video quality and temporal coherence, significantly improves the compliance ability of generated video to quantity instruction.

Description

Technical Field

[0004] This invention belongs to the fields of computer vision and image processing technology, and more specifically, relates to a training-free, layout-guided method for controlling the number of video generated objects. Background Technology

[0006] Text-to-video (T2V) generation models are developing rapidly and are widely used in scenarios such as automatic ad generation, educational video production, and entertainment content creation. Existing mainstream T2V models are generally based on the DiT architecture of diffusion transformers, which can generate coherent and high-quality videos based on text descriptions. However, when dealing with specific numbers of cue words, there is often a problem of "number misalignment".

[0007] Analysis revealed that the model's responses to digit-related semantic tokens in the cross-attention mechanism tend to be quite divergent. Furthermore, DiT models typically operate in a highly downsampled spatiotemporal latent space, limiting the separability of individual object instances. Ultimately, this makes it difficult for the model to accurately capture quantity constraints and perform stable counting control.

[0008] Current solutions typically involve retraining the model, but this requires building a large-scale dataset with precise numerical annotations and can easily compromise the model's original visual quality and motion consistency. Therefore, there is an urgent need for a method that can effectively correct the number of objects in generated videos without retraining, while maintaining video quality and temporal consistency. Summary of the Invention

[0010] To address the aforementioned shortcomings or improvement needs of existing technologies, this invention provides a layout-guided method for aligning the number of objects in video generation. This method employs a two-stage paradigm of "first identification, then guidance," achieving more precise counting control without retraining model parameters, thus alleviating the problems of poor adherence to quantity instructions and unclear instance differentiation in mid-video generation models.

[0011] This invention discloses a method for aligning the number of objects generated in a video, the method comprising:

[0012] Step S1: Obtain input text prompts and sampled noise, perform the pre-generation process for video generation, and extract self-attention maps and cross-attention maps in the intermediate layer of the pre-generated trajectory;

[0013] Step S2: Based on the instance separability index and semantic response strength, select the best self-attention head and the best cross-attention head respectively;

[0014] Step S3: Merge the data of the selected best self-attention head and the best cross-attention head to construct an initial semantic layout graph, and identify the number of object instances generated at the moment based on the layout graph;

[0015] Step S4: Calculate the difference between the number of currently generated object instances and the number of targets in the text prompt words, and based on the difference, perform instance-level region addition or region removal operations on the initial semantic layout graph by minimizing the cost function to obtain the corrected target layout graph;

[0016] Step S5: During the video regeneration process, the cross-attention mechanism is spatially modulated and guided using the target layout map.

[0017] According to the method of the present invention, in step S2, selecting the optimal self-attention head specifically includes:

[0018] Select the reference time step in the pre-generated trajectory The intermediate layer extracts each self-attention head. Attention map;

[0019] The self-attention map is projected onto the first three components using PCA (Principal Component Analysis) and converted to grayscale to obtain the final evaluation map. ;

[0020] Design three complementary scoring indicators to Evaluation: Foreground / Background Separation Structural richness and edge sharpness The head with the highest weighted sum is selected as the best self-attention head. The three indicators are calculated as follows:

[0021] Foreground / Background Separation Self-attention map The standard deviation of the intensity of all pixels in the range is used to measure global contrast.

[0022] Structural richness Self-attention graph The map is divided into multiple non-overlapping blocks. The sum of pixel features within each block is calculated, and then the variance of these block feature sums is calculated. This metric is used to reward attention maps with medium-scale spatial quality and to penalize overly smoothed or degraded maps.

[0023] Edge sharpness Self-attention maps Edge detection is performed using the Sobel operator, and the average gradient magnitude of all pixels on the resulting gradient map is calculated; this metric is used to highlight clear object outlines that support instance separation.

[0024] According to the method of the present invention, in step S2, selecting the optimal cross-attention head specifically includes:

[0025] For each target noun tag T in the text prompt, obtain its cross-attention map in each head. ;

[0026] Calculate the maximum activation value for each cross-attention map. ;

[0027] Select The largest head is the best cross-attention head for this marker. .

[0028] According to the method of the present invention, in step S3, constructing the initial semantic layout graph specifically includes:

[0029] Clustering and segmenting the optimal self-attention map generates a set of spatial candidate regions. ;

[0030] Threshold filtering is applied to the optimal cross-attention map to suppress values ​​below the peak-to-peak ratio threshold, and density-based clustering is applied to form a focus mask F;

[0031] Calculate each candidate region The semantic overlap score with the focus mask F is calculated using the following formula: , where |⋅| represents the number of pixels in the region;

[0032] reserve Using the region as a valid instance region, construct a semantic layout graph containing non-intersecting foreground regions. Pixels within the valid area are assigned category labels. The remaining pixels were assigned background labels. .

[0033] According to the method of the present invention, in step S4, performing instance-level region removal on the initial semantic layout graph specifically includes:

[0034] The number of object instances currently generated More than the target number At that time, determine whether to delete an instance;

[0035] Identify layout diagram For all connected regions belonging to category T, calculate the area of ​​each region and select the region with the smallest area as the region to be removed.

[0036] Reset the labels of all pixels within this area to the background labels. To preserve the main layout structure to the greatest extent possible.

[0037] According to the method of the present invention, in step S4, performing instance-level region addition on the initial semantic layout graph specifically includes:

[0038] The number of object instances currently generated Less than the target number At that time, determine the template for inserting the new instance. ;

[0039] If a region of the same category T already exists, copy the region with the smallest area as a template;

[0040] If it does not exist, a circle with a preset radius will be constructed as a template;

[0041] In each frame f, the positions c for inserting the template are enumerated, and the optimal insertion position of the template is determined in the candidate center grid by minimizing the cost function C(c). The cost function formula is as follows:

[0042]

[0043] in:

[0044] For overlapping items, used to penalize collisions with existing layouts;

[0045] The central term is used to encourage proximity to existing spatial distributions;

[0046] These are timing terms used to ensure stability across frames;

[0047] λ is the balance coefficient. As the center of the candidate insertion position, As the geometric center of the image, This is the center position of the instance in the previous frame.

[0048] According to the method of the present invention, in step S5, the cross-attention mechanism is spatially modulated and guided using the target layout diagram in the following manner:

[0049] Define a guiding intensity function δ(t) that decreases monotonically with the denoising time step t;

[0050] For the cross-attention calculation formula According to the layout diagram The types are adjusted accordingly:

[0051] For the object removal region, the bias term B is set to a negative constant to suppress attention weights;

[0052] When adding a region to an object, if a circular template is used, the bias term B is set to k⋅δ(t), where k is a scalar coefficient;

[0053] When adding a region to an object, if a copy template is used, the reference region is calculated. Average pre-Softmax score and will add areas Overwrite ⋅δ(t).

[0054] In summary, the proposed solution adopts a two-stage paradigm of "identification-guidance." First, it accurately identifies the existing layout and quantity using the characteristics of self-attention and cross-attention. Then, it addresses the quantity misalignment problem through a layout correction strategy based on minimizing the cost function. Finally, it guides the regeneration process through training-free attention modulation. This invention significantly improves the ability of textual video models to follow object quantity instructions without compromising the prior knowledge of the pre-trained model or the video's visual quality, solving the problems of quantity alignment difficulties and instance sticking in existing technologies. Attached Figure Description

[0056] Figure 1 This is a flowchart of a method for aligning the number of video generated objects without training and based on layout guidance, as proposed in an embodiment of the present invention.

[0057] Figure 2 This is a schematic diagram of the overall framework of a training-free, layout-guided video generation object number alignment method proposed in an embodiment of the present invention;

[0058] Figure 3 This is a visual diagram illustrating the sorting and filtering of self-attention heads based on instance separability scores in an embodiment of the present invention;

[0059] Figure 4 This is a qualitative comparison chart of the video frames generated by the embodiments of the present invention and the baseline model in terms of quantity alignment and image quality. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] Example 1:

[0063] This embodiment utilizes a pre-trained video generation model, and after intervention and inference using the training-free layout guidance method proposed in this invention, obtains a number-aligned generated video, and then tests the generated result. This embodiment mainly illustrates how to control the number alignment of generated videos.

[0064] Combination Figure 1 and Figure 2 This invention discloses a method for aligning the number of video generation objects based on layout guidance. Figure 1 This is a flowchart of a method for aligning the number of video generated objects based on layout guidance according to an embodiment of the present invention. The method includes:

[0065] Step S1: Obtain input text prompts and sampled noise, perform the pre-generation process for video generation, and extract self-attention maps and cross-attention maps in the intermediate layer of the pre-generated trajectory;

[0066] Step S2: Based on the instance separability index and semantic response strength, select the best self-attention head and the best cross-attention head respectively;

[0067] Step S3: Merge the data of the selected best self-attention head and the best cross-attention head to construct an initial semantic layout graph, and identify the number of object instances generated at the moment based on the layout graph;

[0068] Step S4: Calculate the difference between the number of currently generated object instances and the number of targets in the text prompt words, and based on the difference, perform instance-level region addition or region removal operations on the initial semantic layout graph by minimizing the cost function to obtain the corrected target layout graph;

[0069] Step S5: During the video regeneration process, the cross-attention mechanism is spatially modulated and guided using the target layout map.

[0070] In step S1, the input text prompts and sampled noise are obtained, the pre-generation process of video generation is performed, and the self-attention map and cross-attention map are extracted in the middle layer of the pre-generated trajectory;

[0071] In this embodiment, the processing of input data and pre-generation process in step S1 mainly includes:

[0072] Receive user input including a text prompt word describing a specific quantity (e.g., "3 cats") and a randomly sampled noise vector;

[0073] We initiated the pre-generation process of the DiT video generation model. Specifically, we used the Wan2.1-1.3B model as the pre-trained base model. The Wan2.1-1.3B model adopts a diffusion model framework, has 30 layers, 12 attention heads, and uses 50 time steps for sampling.

[0074] Select a reference time step in the pre-generated trajectory =20 and intermediate layer =15, extract the self-attention tensor and cross-attention tensor of the output of this layer.

[0075] In step S2, the best self-attention head and the best cross-attention head are selected based on the instance separability index and semantic response strength, respectively.

[0076] In this embodiment, step S2 specifically includes:

[0077] First, principal component analysis (PCA) is performed on the attention map of each extracted self-attention head h, projected onto the first three components, and converted into a grayscale image. ;

[0078] Design three complementary scoring indicators to Evaluation: Foreground / Background Separation Structural richness and edge sharpness The head with the highest weighted sum is selected as the best self-attention head. The three indicators are calculated as follows:

[0079] Foreground / Background Separation Self-attention map The standard deviation of the intensity of all pixels in the range is used to measure global contrast.

[0080] Structural richness Self-attention graph The map is divided into multiple non-overlapping blocks. The sum of pixel features within each block is calculated, and then the variance of these block feature sums is calculated. This metric is used to reward attention maps with medium-scale spatial quality and to penalize overly smoothed or degraded maps.

[0081] Edge sharpness Self-attention maps Edge detection is performed using the Sobel operator, and the average gradient magnitude of all pixels on the resulting gradient map is calculated; this metric is used to highlight clear object outlines that support instance separation.

[0082] The overall score S for each self-attention map is calculated using the following weighted formula. ):

[0083]

[0084] In this embodiment, the weighting coefficients are set to α=1, β=1, and γ=0.1;

[0085] like Figure 3 The heads shown are sorted, and the head with the highest overall score is selected as the best self-attention head. ;

[0086] For each target noun tag T in the text prompt, obtain its cross-attention map in each head. ;

[0087] Calculate the maximum activation value for each cross-attention map. ;

[0088] Select The largest head is the best cross-attention head for this marker. .

[0089] In step S3, the data of the selected best self-attention head and the best cross-attention head are fused to construct an initial semantic layout graph, and the number of currently generated object instances is identified based on the layout graph;

[0090] In this embodiment, step S3 specifically includes:

[0091] For the best self-attention map Clustering and segmentation are performed; specifically, the MeanShift algorithm is used to generate a set of spatial candidate regions. ;

[0092] For the optimal cross-attention map Threshold filtering is performed to suppress values ​​below the peak-to-peak ratio threshold, and density-based clustering is applied. Specifically, the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) method is used to form a focus mask F.

[0093] Calculate each candidate region The semantic overlap score with the focus mask F is calculated using the following formula: , where |⋅| represents the number of pixels in the region;

[0094] reserve Using the region as a valid instance region, construct a semantic layout graph containing non-intersecting foreground regions. Pixels within the valid area are assigned category labels. The remaining pixels were assigned background labels. .

[0095] In this embodiment, Set it to 0.2.

[0096] In step S4, the difference between the number of currently generated object instances and the number of targets in the text prompt words is calculated, and based on the difference, instance-level region addition or region removal operations are performed on the initial semantic layout graph by minimizing the cost function to obtain the corrected target layout graph;

[0097] In this embodiment, step S4 specifically includes:

[0098] Parse text prompts to obtain the target number ;

[0099] The number of object instances T generated in the current frame f More than the target number At that time, determine whether to delete an instance;

[0100] Identify layout diagram For all connected regions belonging to category T, calculate the area of ​​each region and select the region with the smallest area as the region to be removed.

[0101] Reset the labels of all pixels within this area to the background labels. To preserve the main layout structure to the greatest extent possible.

[0102] The number of object instances T generated in the current frame f Less than the target number At that time, determine the template for inserting the new instance. ;

[0103] If a region of the same category T already exists, copy the region with the smallest area as a template;

[0104] If it does not exist, a circle with a preset radius r is constructed as a template;

[0105] In each frame f, the positions c for inserting the template are enumerated, and the optimal insertion position of the template is determined in the candidate center grid by minimizing the cost function C(c). The cost function formula is as follows:

[0106] in:

[0107] For overlapping items, used to penalize collisions with existing layouts;

[0108] The central term is used to encourage proximity to existing spatial distributions;

[0109] These are timing terms used to ensure stability across frames;

[0110] λ is the balance coefficient. As the center of the candidate insertion position, As the geometric center of the image, This is the center position of the instance in the previous frame.

[0111] In this embodiment, the preset radius r=5 and the balance coefficient λ=8. The above operation is repeated until the quantity... equal The corrected target layout diagram is obtained. .

[0112] In step S5, during the video regeneration process, the target layout map is used to spatially modulate and guide the cross-attention mechanism to generate a video with the same number of objects as the text prompts.

[0113] In this embodiment, step S5 specifically includes:

[0114] Define a guiding intensity function δ(t) that decreases monotonically with the denoising time step t;

[0115] For the cross-attention calculation formula According to the layout diagram The types are adjusted accordingly:

[0116] For the object removal region, the bias term B is set to a negative constant to suppress attention weights;

[0117] When adding a region to an object, if a circular template is used, the bias term B is set to k⋅δ(t), where k is a scalar coefficient;

[0118] When adding a region to an object, if a copy template is used, the reference region is calculated. Average pre-Softmax score and will add areas Overwrite ⋅δ(t).

[0119] Specifically, in this embodiment, the following is selected: t is the diffusion time step; the negative constant is set to -1e4; k is 0.8.

[0120] The final output video contains the same number of objects as the prompt words.

[0121] Example 2:

[0122] Based on the layout-guided video generation object number alignment method described in Embodiment 1 above, this embodiment illustrates the performance of this method on a specially constructed benchmark test set, and conducts quantitative and qualitative comparative analysis with existing mainstream video generation models.

[0123] To comprehensively evaluate numerical alignment capabilities in Text-to-Video (T2V), this embodiment uses a specially constructed benchmark dataset, CountBench. Existing T2V benchmarks such as EvalCrafter and T2VCompBench often focus on visual quality or general text alignment, neglecting precise numerical generation. In contrast, CountBench contains 210 carefully crafted cue words for complex scenes, designed to evaluate numerical fidelity. These cue words cover a wide range of test conditions, including: generation instructions covering 1 to 8 object instances; and combinations involving 1 to 3 different object categories. The dataset was constructed by combining automated generation of GPT-5 with manual review, ensuring the dynamic nature of the cue word descriptions and eliminating duplicate or illogical entries, enabling a systematic evaluation of the T2V model's ability to handle multiple numerical constraints.

[0124] This embodiment uses five complementary metrics to quantitatively evaluate the method's quantitative alignment capability and generation quality:

[0125] Counting Accuracy (CountAcc): Measures how well the generated results adhere to the quantity instruction. The GroundingDINO model is used to detect objects in each frame. If the number of detected objects of a certain class matches the prompt, it is scored as 1 point; otherwise, it is scored as 0 points. The final score is the average of all frames and all classes.

[0126] Temporal Consistency (TC): This measures the stability of the number of generated objects. For each pair of adjacent frames, if the number of objects of a certain type remains consistent, it is scored as 1 point; otherwise, it is scored as 0 points. The final score is the average of all frame pairs and categories.

[0127] CLIP Score: Used to evaluate the overall semantic alignment between the generated video and the text prompts. It is calculated as the average of the CLIP scores frame by frame.

[0128] Aesthetic Score (VBench): Evaluates the visual appeal and attractiveness of the generated image, taking into account high-level visual attributes such as composition, color, and style.

[0129] Image Quality Rating (VBench-Imaging Quality): Evaluates the basic technical quality of the generated image by quantifying the image's performance in low-level visual defects such as sharpness, noise, and artifacts using a dedicated model and taking the average value of each frame.

[0130] In this embodiment, the Wan2.1-1.3B text-generated video model was selected as the baseline model, and the method of the present invention was compared with the baseline model. The quantitative results are shown in Table 1. The method of the present invention achieves a significant improvement in counting accuracy while maintaining highly competitive overall generation quality.

[0131] Table 1. Results of the baseline model and this embodiment on the benchmark set.

[0132] method CountAcc% TC% CLIP Score Imaging Aesthetic Baseline model 42.3 81.2 33.9 71.3 61.5 This embodiment 49.7 83.4 35.6 70.9 63.5

[0133] also, Figure 4 The visualization results further qualitatively demonstrate the effectiveness of this method. Compared with the baseline model, the video generated by the method of this invention can reliably present the exact number of prompts required, while maintaining a natural scene layout and smooth temporal coherence.

[0134] Example 3:

[0135] To further verify the effectiveness of the key modules and strategies in the layout-guided video generation object number alignment method proposed in this invention, this embodiment conducted a series of ablation experiments based on the CountBench benchmark dataset. This embodiment mainly explores the impact of different cost function terms during layout correction, as well as the specific contributions of object addition and removal operations to the final performance. The results show that each module strategy can improve performance, with optimal performance achieved when the entire method is used.

[0136] When a new object instance needs to be added, this invention designs a heuristic cost function C(c) to determine the optimal insertion position. This cost function consists of overlap cost. Center distance cost and the cost of timing consistency It consists of three parts. To evaluate the contribution of each part, this embodiment tested the counting accuracy under different combinations, and the results are shown in Table 2.

[0137] This invention guides the generation process by modifying the layout diagram, specifically including operations in two dimensions: object addition and object removal. To investigate the specific contribution of these two operations to the final quantity alignment accuracy, this embodiment compares the performance of performing only the addition operation, performing only the removal operation, and performing both operations simultaneously. The experimental results are shown in Table 3.

[0138] As shown in Table 3, the performance improvement brought by the addition operation alone is significant, indicating that the main bottleneck of the existing model lies in the omission of objects. The removal operation alone also brings positive benefits, effectively correcting the error of redundant generation. When both operations are enabled simultaneously, the system performance reaches its peak and is better than the linear sum of their individual effects. This result confirms that the object addition and removal strategies have a significant synergistic effect in dealing with complex quantity misalignment problems. Both are indispensable and together ensure the robustness of the method of this invention in diverse scenarios.

[0139] Table 2 Comparison of the effects of the three cost functions

[0140]

[0141] Table 3 Comparison of the effects of adding and removing operations

[0142]

[0143] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A training-free, layout-guided method for controlling the number of objects generated in video, characterized in that, The method includes: Step S1, acquiring input text prompts and sampled noise, performing a pre-generation process for video generation, and extracting self-attention maps and cross-attention maps in the intermediate layer of the pre-generated trajectory; Step S2, selecting the best self-attention head and the best cross-attention head based on instance separability index and semantic response strength; Step S3, fusing the data of the selected best self-attention head and the best cross-attention head to construct an initial semantic layout map, and identifying the number of currently generated object instances based on the layout map; Step S4, calculating the difference between the number of currently generated object instances and the target number in the text prompts, and performing instance-level region addition or removal operations on the initial semantic layout map by minimizing the cost function according to the difference, to obtain a corrected target layout map; Step S5, during the video regeneration process, using the target layout map to spatially modulate and guide the cross-attention mechanism.

2. The method according to claim 1, characterized in that, In step S2, the selection of the best self-attention head includes: performing feature dimensionality reduction and grayscale processing on the attention map of each self-attention head at the reference time step in the pre-generated trajectory; calculating the separation score S for each self-attention map, wherein the score S is based on the foreground-background separation. Structural richness and edge sharpness The weighted sum is calculated; the head with the highest score S is selected as the best self-attention head.

3. The method according to claim 2, characterized in that, The formula for calculating the score S is: .in, This represents the self-attention map of the h-th head. The standard deviation of intensity measures the separation between the foreground and background. The feature variance of the blocks is used to measure structural richness; α represents the average magnitude of the gradient, which measures edge sharpness; α, β, and γ are positive weighting coefficients.

4. The method according to claim 1, characterized in that, In step S2, the selection of the optimal cross-attention head specifically includes: for each target noun tag T in the text prompt, obtaining its cross-attention map across all heads. ; Calculate the maximum activation value for each cross-attention map : Select the maximum activation value The highest head is taken as the best cross-attention head for the target noun label T.

5. The method according to claim 1, characterized in that, In step S3, constructing the initial semantic layout graph includes: performing clustering segmentation on the best self-attention graph to obtain candidate regions; performing threshold filtering and density clustering on the best cross-attention graph to obtain a focus mask; calculating the overlap score between the candidate regions and the focus mask, and retaining regions with an overlap score greater than a preset threshold τ as valid object instance regions.

6. The method according to claim 1, characterized in that, In step S4, the instance-level region addition or removal operation is specifically as follows: if the current number is greater than the target number, remove the smallest instance region of the same category and reset its pixel label to the background; if the current number is less than the target number, determine the insertion position and insert a new instance template. The new instance template prioritizes copying the smallest instance region among the currently existing instances of the same category; If no instance of the same type exists, a preset geometry will be used as the template.

7. The method according to claim 5, characterized in that, The insertion position is determined by minimizing a cost function, which comprehensively considers: the degree of overlap between the new instance and the existing layout; the distance between the center of the new instance and the center of the existing spatial distribution; and (when the number of frames is greater than 1) the distance between the center of the new instance and the center of the same instance in the previous frame, to maintain temporal continuity. In each frame f, the position c of the insertion template is enumerated, and the optimal insertion position of the template is determined in the candidate center grid by minimizing the cost function C(c). The cost function formula is as follows: .in: For overlapping items, used to penalize collisions with existing layouts; The central term is used to encourage proximity to existing spatial distributions; The timing term is used to ensure stability across frames; λ is the balance coefficient. As the center of the candidate insertion position, As the geometric center of the image, This is the center position of the instance in the previous frame.

8. The method according to claim 1, characterized in that, In step S5, the spatial modulation guidance of the cross-attention mechanism using the target layout map includes: defining a guidance intensity function δ(t), which monotonically decreases with the denoising time step t; and the cross-attention calculation formula. For regions to be removed, the bias term B in the cross-attention mechanism is modified to a negative value to suppress generation; for regions to be added, if a preset geometry template is used, the bias term B is set to k⋅δ(t), where k is a scalar coefficient; if a copy template is used, the reference region is calculated. Average pre-Softmax score and will add areas Overwrite ⋅δ(t) increases the value of the bias term B.

9. A layout-guided video generation object number alignment system, characterized in that, include: The pre-generation module is used to perform video pre-generation and extract attention maps based on text prompts; The recognition module is used to filter attention heads and construct an initial layout map to identify the current number of objects; The layout correction module is used to perform instance-level addition and deletion corrections to the layout diagram based on quantity differences; the guided generation module is used to modulate the attention mechanism using the corrected layout diagram to regenerate the video.

Images