Video generation method and system based on entropy-guided k-guardrail sampling

Abstract

The invention discloses a video generation method and system based on entropy guide k-guardrail sampling, and relates to computer vision. Each frame of a video is expressed as a discrete token sequence; predicting the token at the location (t, i) using an autoregression model; calculating Shannon entropy for the token prediction category distribution; performing affine transformation and cutting on the normalized entropy value to obtain a self-adaptive cumulative probability threshold value; performing truncation sorting on the token prediction category distribution by using a cumulative probability threshold to obtain a truncation point c; performing lower limit constraint on the cut-off point c by adopting a k-guardrail mechanism to obtain an adjusted cut-off point, and forming a final candidate token set based on the adjusted cut-off point; performing probability redistribution on the final candidate token set by using normalization operation; performing token selection on the renormalized distribution by adopting random sampling to obtain a generated token; according to the invention, entropy collapse and error accumulation are effectively inhibited without modifying the model or retraining, and the quality and consistency of video generation are improved.

Description

Technical Field

[0001] This application relates to the field of computer vision, and more specifically, to a video generation method and system based on entropy-guided k-barrier sampling. Background Technology

[0002] In recent years, the field of video-based world models has experienced explosive growth, making significant progress in generating high-fidelity, temporally coherent, and physically consistent video sequences. These models aim to establish an internal representation of the dynamics of the world, thereby supporting a variety of applications ranging from realistic robotic simulation to high-quality content creation. This trend has driven the emergence of video generation models capable of understanding and simulating interactive environments, not only synthesizing videos from text but also performing world reasoning and environmental interaction to a certain extent.

[0003] Among numerous generative paradigms, the Autoregressive (AR) architecture, after achieving significant success in Large Language Models (LLMs), has been widely adopted in the field of video generation. The basic idea of ​​the autoregressive model is to decompose the joint probability distribution of video frames into a product of conditional probabilities, thereby modeling temporal causality frame by frame. This sequential generation approach is highly flexible, supporting both variable-length video generation and naturally integrating with scalable Transformer architectures. By decomposing the joint distribution of video tokens or frames into a product of conditional probabilities, the autoregressive model can generate coherent long temporal videos frame by frame or token by token, thus supporting fine-grained temporal control and interactive generation.

[0004] However, this frame-by-frame dependency of autoregressive models also introduces serious problems: error accumulation and exposure bias. During generation, even a slight inaccuracy or suboptimal choice in a single frame can amplify over time, leading to quality degradation, structural misalignment, or semantic drift in long video sequences. In the inference phase, the prediction of each frame or token serves as input for subsequent steps, so any small deviation propagates and is amplified over time, ultimately manifesting as flickering, unnatural motion, or deviation from the expected scene. Error accumulation has been theoretically considered one of the core challenges of autoregressive video generation, often coexisting with problems such as memory bottlenecks.

[0005] To address the aforementioned issues, various mitigation strategies have been proposed in existing research. For example, some studies simulate the inference process during the training phase, allowing the model to learn to self-correct during inference by using its own historical predictions as input. Other studies introduce noisy contexts or masking mechanisms to make the model robust to imperfect inputs. Although these methods improve temporal consistency to some extent, they often require modifications to the model structure or retraining, increasing system complexity and deployment costs. Summary of the Invention

[0006] To address the error accumulation in long-term autoregressive video generation, this application provides a video generation method and system based on entropy-guided k-barrier sampling. It can adaptively adjust the size of the candidate set according to the uncertainty of token-level prediction and prevent excessive greed through the k-barrier mechanism. It can effectively suppress entropy collapse and error accumulation without modifying the model or retraining, thereby improving the quality and consistency of video generation.

[0007] One aspect of this application provides a video generation method based on entropy-guided k-fence sampling, comprising: S1, acquiring video data to be generated, and representing each frame of the video as a discrete token sequence. Where t is the time step index, i∈{1,...,m} is the intra-frame spatial location index, and V is the discrete codebook; S2, using an autoregressive model to assign tokens at position (t,i) Perform predictions and generate token prediction category distributions. S3, calculate the predicted category distribution of tokens respectively. Shannon entropy And according to Shannon entropy Calculate the adaptive cumulative probability threshold S4, based on the cumulative probability threshold Determining the cutoff point using the k-barrier mechanism And based on the cutoff point Determine the final candidate token set S5, based on random sampling method for the final candidate token set S6. Perform sampling to obtain the target token; S7. Traverse each video frame and decode the video according to the target token to obtain the complete video frame sequence.

[0008] In this context, the token is the basic representation unit of a video frame after discretization encoding. Discrete coding techniques such as VQ-VAE (Video Quantization Variational Autoencoder) convert the continuous pixel representation of video frames into a discrete sequence of symbols. Each token... This represents the discrete encoded value at spatial location i in the t-th frame of the video, with values ​​ranging from a predefined codebook V. For example, if the codebook size |V| = 8192, then each token can take an integer value between 0 and 8191. This discretization reduces the data dimensionality, allowing autoregressive models to transform the video generation problem into a sequence prediction problem, similar to word prediction in language models.

[0009] Token Prediction Category Distribution This is the probability prediction of an autoregressive model for all possible token values ​​at position (t, i). When the autoregressive model predicts what should be displayed at position i in frame t, it outputs the probability distribution of all possible visual representations at that position. Prediction target: What visual features (color, texture, edges, etc.) should the pixel block at this location exhibit? Distribution meaning: Each v value in the codebook represents a predefined visual pattern. This indicates the probability that a given visual pattern will be present at that location. For example, when predicting the location of a road centerline: the probability of a white marker token might be 0.6, the probability of a gray pavement token might be 0.3, and the probabilities of other textures would be very low. When predicting a foliage area: the probabilities of various shades of green texture tokens might be quite dispersed (0.15, 0.12, 0.18...). This distribution depends not only on the portion already generated in the current frame but also on the content of historical frames. and action information This ensures temporal continuity. For example, if the location in the previous frame was the front of the vehicle, the current frame is more likely to predict it as the middle or rear of the vehicle.

[0010] Shannon entropy It quantifies the degree of certainty of the model for the visual content at the i-th position in the t-th frame, and directly reflects the visual complexity at that position; Cumulative probability threshold Based on the visual uncertainty at that location, the number of possible visual representations to be considered is dynamically determined; a fixed sampling strategy (such as always top-k=30) will result in: introducing too many noisy candidates at the vehicle edges, causing structural blurring; insufficient candidates in the leaf area, resulting in monotonous texture; while the adaptive threshold adjusts the number of candidates according to the visual characteristics of each pixel block.

[0011] The k-fence mechanism is a lower bound constraint strategy that guarantees minimum exploration degree by forcibly retaining the highest probability frontier. Tokens are used to prevent the candidate set from shrinking excessively. This prevents frames from degenerating into cartoonish, simple color blocks, for example, when... Even if a nucleus sample at a certain position selects only one token, the k-fence mechanism will force the candidate set to be expanded to include at least the three tokens with the highest probability, thereby maintaining structural stability while avoiding the degradation problem caused by complete determinism.

[0012] Furthermore, S2 utilizes an autoregressive model to evaluate the token at position (t, i). Perform predictions and generate token prediction category distributions. This includes: the previously decoded token of the current frame. Historical context and historical actions The conditional input is used as the input to the autoregressive model; the conditional probability of each candidate token v∈V at position (t, i) is calculated using the autoregressive model. .

[0013] Furthermore, S3, the token prediction category distribution obtained from S2. Calculate Shannon entropy and Shannon entropy Normalization is performed to obtain the normalized entropy value. This includes: predicting category distribution based on tokens. Calculate the Shannon entropy at position (t, i). Obtain the size |V| of the codebook V, and calculate the theoretical maximum value of the Shannon entropy. According to the calculated Shannon entropy The theoretical maximum value obtained Calculate the normalized entropy value .

[0014] Furthermore, S4, for the normalized entropy value Perform affine transformation and clipping to obtain an adaptive cumulative probability threshold. This includes: obtaining the lower bound of the preset normalized entropy value. and the Upper Realm and the lower bound of the cumulative probability threshold. and the Upper Realm According to the lower bound of the normalized entropy value and the Upper Realm and the lower bound of the cumulative probability threshold. and the Upper Realm Calculate the mapping parameters α and β of the affine transformation; based on the mapping parameters α and β of the affine transformation, calculate the normalized entropy value. Perform an affine transformation to obtain the mapped cumulative probability threshold; then trim the mapped cumulative probability threshold to obtain an adaptive cumulative probability threshold. .

[0015] Furthermore, S5 utilizes a cumulative probability threshold. Predicting the distribution of token categories Perform truncation sorting to obtain cutoff point c, including: the token prediction class distribution obtained from S2. All tokens v in the codebook V are assigned their corresponding probabilities Sort in descending order to obtain the sorted token index sequence. Based on the sorted token index sequence Construct the sorted probability sequence ; for the sorted probability sequence Accumulate sequentially to calculate the cumulative probability sequence; obtain an adaptive cumulative probability threshold based on S4. Based on the calculated cumulative probability sequence, determine the initial cutoff point; based on the determined initial cutoff point c, extract the first c tokens from the sorted token index sequence to form an initial candidate token set; output the determined initial cutoff point c as the input for S6 to perform lower bound constraints using the k-barrier mechanism.

[0016] Furthermore, in S6, a k-barrier mechanism is used to apply a lower limit constraint to the cutoff point c, resulting in the adjusted cutoff point. Based on the adjusted cutoff point c', the final candidate token set is formed. This includes: obtaining preset k-fence parameters. The initial cutoff point c obtained from S5 is compared with the k-guardrail parameters. Compare the two values ​​and take the maximum value as the adjusted cutoff point. Based on the cutoff point From the sorted token index sequence Extracting the first 1 set of tokens is generated to form a k-fence candidate token set; the resulting initial candidate token set is... With the obtained set of k-fence candidate tokens Perform a union operation to form the final candidate token set. .

[0017] Furthermore, S7 utilizes a normalization operation on the final candidate token set. Probability redistribution is performed to obtain a renormalized distribution, including: the final candidate token set formed from S6. Extract all candidate tokens and obtain the token prediction category distribution for each candidate token in S2. The corresponding original probability value The final candidate token set obtained by calculation The sum of the original probabilities of all candidate tokens The final candidate token set obtained from S6 For each candidate token v, predict the category distribution based on the corresponding token. The original probability values ​​and normalization factors in the data. Calculate the corresponding renormalized probability. Based on the renormalized probability Construct a renormalized distribution .

[0018] Furthermore, in S8, token selection is performed on the renormalized distribution using random sampling to obtain the generated token, including: based on the renormalized distribution... Construct the cumulative distribution function Generate a random number that is uniformly distributed in the interval [0, 1]. Based on random numbers and cumulative distribution function The target token is determined by the inverse transform sampling method. ; Another aspect of this application provides a video generation system based on entropy-guided k-barrier sampling, comprising: a video data acquisition module for acquiring video data to be generated and representing each frame of the video as a discrete token sequence. The token prediction module uses an autoregressive model to predict the token at position (t, i). Perform predictions and generate token prediction category distributions. The entropy calculation and normalization module processes the token prediction category distribution obtained from the token prediction module. Calculate Shannon entropy and Shannon entropy Normalization is performed to obtain the normalized entropy value. The threshold adaptive adjustment module adjusts the normalized entropy value obtained from the entropy calculation and normalization modules. Perform affine transformation and clipping to obtain an adaptive cumulative probability threshold. The truncation and sorting module utilizes the cumulative probability threshold obtained from the threshold adaptive adjustment module. The token prediction category distribution obtained from the token prediction module The truncation sorting process yields the truncation point c. The k-barrier constraint module applies a lower bound constraint to the truncation point c obtained from the truncation sorting module using the k-barrier mechanism, resulting in the adjusted truncation point. Based on the adjusted cutoff point Forming the final candidate token set The probability renormalization module uses normalization operations to apply to the final candidate token set formed by the k-fence constraint module. The system performs probability redistribution to obtain a renormalized distribution; the random sampling module uses random sampling to select tokens from the renormalized distribution obtained by the probability redistribution module to generate tokens; the iterative control module controls the iterative execution of the video data acquisition module, token prediction module, entropy calculation and normalization module, threshold adaptive adjustment module, truncation and sorting module, k-barrier constraint module, probability redistribution module and random sampling module until a complete video frame sequence is generated. Compared to existing technologies, the advantages of this application are: To address the structural drift caused by error accumulation during long-term autoregressive video generation, the inability of fixed sampling strategies to adapt to the spatiotemporal heterogeneity of token distribution, and the texture degradation caused by entropy collapse in existing technologies, this application provides a video generation method based on entropy-guided k-barrier sampling. It can dynamically adjust the size of the candidate token set by quantifying the uncertainty of token-level prediction and mapping it to an adaptive cumulative probability threshold without modifying the model architecture or retraining. Meanwhile, the k-barrier mechanism is introduced to ensure minimum exploration degree to avoid excessive greedy sampling, thereby maintaining structural stability in high confidence regions and enhancing texture diversity in high uncertainty regions. This effectively suppresses long-term temporal error accumulation and entropy collapse, and comprehensively improves the spatiotemporal consistency, structural fidelity and detail richness of the generated video. Attached Figure Description

[0019] Figure 1 This is an example flowchart of a video generation method based on entropy-guided k-fence sampling according to this application; Figure 2 This is a visual comparison of the effects of applying the strategy of this application before and after applying it to the DrivingWorld and VaVIM models; Figure 3 The diagram shows the before-and-after comparison of the strategy applied to the Cosmos model. Figure 4 This is a visualization of the entropy collapse phenomenon in the AR decoding of this application; Figure 5 This is a diagram illustrating the effect of entropy adaptive guidance in this application. Figure 6 The image shows the rendering of the k-guardrail design for this application. Detailed Implementation

[0020] The present application will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0021] Example 1 like Figure 1As shown, a video generation method based on entropy-guided k-fence sampling includes: S1, acquiring the video data to be generated, and representing each frame of the video as a discrete token sequence. Where t is the time step index, i∈{1,...,m} is the intra-frame spatial location index, and V is a discrete codebook of size V; S2, using an autoregressive model to assign tokens at position (t,i) Perform predictions and generate token prediction category distributions. The token prediction category distribution condition is based on the previously decoded tokens in the current frame. Historical context and historical actions S3, the predicted category distribution of tokens obtained from S2. Calculate Shannon entropy and Shannon entropy Normalization is performed to obtain the normalized entropy value. : Normalize the Shannon entropy to the unit interval [0, 1] to obtain the normalized entropy value: Among them, the normalized entropy value Characterizes the degree of uncertainty in the distribution of the token prediction categories obtained from S2.

[0022] S4. Perform an affine transformation and clipping on the normalized entropy value to obtain an adaptive cumulative probability threshold. ; The normalized entropy value obtained from S3 is obtained through affine transformation. Mapped to cumulative probability threshold : ;in, For the clipping function, These are the lower and upper bounds of the cumulative probability threshold, respectively. The mapping parameters α and β of the affine transformation are determined by the following formula: ; ;in, These are the lower and upper bounds of the normalized entropy value, respectively. Cumulative probability threshold The normalized entropy value obtained from S3 changes dynamically. S5, use the cumulative probability threshold to truncate and sort the predicted category distribution to obtain the cutoff point c; Predict the category distribution of the tokens obtained in S2. The tokens are sorted in descending order of probability to obtain the sorted probability sequence. ; The cumulative probability threshold obtained based on S4 Calculate the minimum cutoff point that satisfies the cumulative probability condition: The initial cutoff point c determines the size of the initial candidate token set. S6, using the k-barrier mechanism to apply a lower limit constraint to the cutoff point, obtains the adjusted cutoff point. ; Applying the k-barrier mechanism to the initial cutoff point c obtained in S5, we calculate the adjusted cutoff point: ;in, This is the preset minimum guardrail value; Based on the adjusted cutoff point This forms the final candidate token set. Final candidate token set Includes S5 sorted probability sequence The front of the middle One token; The k-barrier mechanism ensures that the final candidate token set is no smaller than [a certain size]. ; S7. The probability of the final candidate token set is redistributed using a normalization operation to obtain a renormalized distribution. The final candidate token set obtained from S6 The token probabilities in the data are renormalized: ; ;in, The adjusted cutoff point obtained from S6, This is the i-th probability in the sorted probability sequence obtained in S5; Renormalized distribution Satisfying the probability normalization condition ; S8, using random sampling to select tokens from the renormalized distribution, yields the generated tokens. The renormalized distribution obtained from S7 Random sampling token index: ; Use the token corresponding to the sampled token index y as the generated token for position (t, i); S9, iterate through S1 to S8 until a complete video frame sequence is generated. Add the generated token obtained in S8 to the already generated discrete token sequence, and update the decoded preceding token of the current frame. ; Repeat steps S2 to S8 to generate subsequent position tokens for the current frame one by one until the generation of m position tokens for the current frame is completed; Repeat the above process to generate subsequent frames until a complete video frame sequence is generated. Specifically, S2 uses an autoregressive model to evaluate the token at position (t, i). Perform predictions and generate token prediction category distributions. This includes: the previously decoded token of the current frame. Historical context and historical movement As a conditional input to the autoregressive model; where the previous token of the current frame has been decoded. For discrete token sequences in S1 The set of generated tokens at position i in the current frame t. Historical context Includes token information for all generated frames prior to time step t; historical actions. Includes video generation control action information prior to time step t; The conditional probability of each candidate token v∈V at position (t, i) is calculated using an autoregressive model:

[0023] Obtain the probability distribution for all V discrete tokens in the codebook V, and the token prediction class distribution. Satisfying the probability normalization condition ; Predict the category distribution of the obtained tokens. The output is used to calculate the Shannon entropy in S3. Input; Token Prediction Category Distribution The shape reflects the prediction confidence of the autoregressive model at the current position (t, i). When the distribution is highly concentrated in a few tokens, it indicates a high confidence prediction. When the distribution is relatively uniform, it indicates a high prediction uncertainty.

[0024] Specifically, S3 is the token prediction category distribution obtained from S2. Calculate Shannon entropy and Shannon entropy Normalization is performed to obtain the normalized entropy value. ,include: Token prediction category distribution obtained based on S2 Calculate the Shannon entropy at position (t, i):

[0025] Among them, Shannon entropy By considering the probabilities of all tokens v in the codebook V The summation of the negative products of the values ​​yields the quantified token prediction class distribution obtained from S2. The degree of uncertainty; Obtain the size |V| of the codebook V, and calculate the theoretical maximum value of the Shannon entropy. ; Theoretical maximum value Corresponding token prediction category distribution The Shannon entropy value is the value when the distribution is uniform, i.e., when The entropy value that holds for all v∈V; According to Shannon entropy The theoretical maximum value log|V| is obtained, and the normalized entropy value is obtained from this: Normalized entropy value It is mapped to the unit interval [0, 1]; Normalized entropy A value close to 0 indicates the token prediction category distribution obtained by S2. Highly concentrated, the autoregressive model predicts with high confidence; normalized entropy value A value close to 1 indicates the token prediction category distribution obtained by S2. The distribution is close to uniform, and the autoregressive model has high prediction uncertainty.

[0026] Specifically, S4, for the normalized entropy value Perform affine transformation and clipping to obtain an adaptive cumulative probability threshold. ,include: Obtain the lower bound of the preset normalized entropy value and the Upper Realm and the lower bound of the cumulative probability threshold. and the Upper Realm ; Lower bound of normalized entropy and the Upper Realm The normalized entropy value obtained from S3 is defined. The effective mapping interval, where, ; Lower bound of cumulative probability threshold and the Upper Realm Define the cumulative probability threshold The target value range, where, ; Calculate the mapping parameters of the affine transformation based on the normalized entropy boundary and the cumulative probability threshold boundary: ; The mapping parameter α determines the normalized entropy value. To the cumulative probability threshold The slope of the linear mapping, and the mapping parameter β determines the intercept of the mapping; The normalized entropy value obtained from S3 Affine transformation is performed using the mapping parameters α and β calculated in S4.2: ; Obtain the cumulative probability threshold after initial mapping The cumulative probability threshold after initial mapping The normalized entropy value obtained from S3 The relationship is linear; Cumulative probability threshold after initial mapping By applying the pruning function, an adaptive cumulative probability threshold is obtained: The pruning function will set the cumulative probability threshold after initial mapping. Limiting the target value range set in S4.1 Inside; Adaptive cumulative probability threshold The output is the S5 token prediction category distribution. Input for truncated sorting; Adaptive cumulative probability threshold The normalized entropy value obtained with S3 Dynamic adjustment: When the normalized entropy value A lower value corresponds to a smaller cumulative probability threshold. It tends to retain fewer high-probability candidate tokens; when the normalized entropy value A higher value corresponds to a larger cumulative probability threshold. They tend to retain more candidate tokens to increase sampling diversity.

[0027] Specifically, S5 utilizes a cumulative probability threshold. Predicting the distribution of token categories Perform a truncation sort to obtain the cutoff point c, including: Predict the category distribution of the tokens obtained in S2. All tokens v in the codebook V are assigned their corresponding probabilities Sort in descending order; Obtain the sorted token index sequence ,in, The token with the highest probability The token with the lowest probability satisfies ; Based on the obtained sorted token index sequence Construct the sorted probability sequence ;in, This represents the probability value corresponding to the j-th token after sorting, and the sorted probability sequence. The token prediction category distribution obtained for S2 The descending probability representation; The resulting sorted probability sequence Calculate the cumulative probability sequence by summing sequentially: Among them, cumulative probability This represents the sum of probabilities of the first j tokens after sorting, and the cumulative probability sequence. Satisfies the property of monotonically increasing ; Adaptive cumulative probability threshold obtained based on S4 and the calculated cumulative probability sequence Determine the initial cutoff point: The initial cutoff point c satisfies the cumulative probability. The cumulative probability threshold obtained from S4 is reached or exceeded for the first time. The minimum token position index; Based on the determined initial cutoff point c, the first c tokens are extracted from the obtained sorted token index sequence to form an initial candidate token set: ; Initial candidate token set Including cumulative probability reaching the cumulative probability threshold The minimum number of tokens required achieves the normalized entropy value obtained from S3. A nucleus sampling mechanism that adaptively adjusts the size of the candidate token set; The determined initial cutoff point c is output as the input for S6 to perform lower bound constraints using the k-barrier mechanism.

[0028] Specifically, in S6, the k-barrier mechanism is used to impose a lower limit constraint on the cutoff point c, resulting in the adjusted cutoff point. Based on the adjusted cutoff point This forms the final candidate token set. ,include: Get the preset k-fence parameters k-guardrail parameters A minimum bound on the size of the candidate token set is defined; where It is a positive integer, guaranteed to be within the range of low normalized entropy values. Even when the initial cutoff point c obtained by S5 is too small, it can still maintain a minimum level of sampling exploration and avoid texture degradation caused by excessive greedy sampling. The initial cutoff point c obtained in S5 is compared with the k-fence parameters obtained in S6.1. Compare the two values ​​and take the maximum value as the adjusted cutoff point: Adjusted cutoff point Ensure that the size of the final candidate token set is not less than This achieves a lower bound constraint on the initial cutoff point c of S5; When the initial cutoff point c is less than k - guardrail parameter At that time, that is From the obtained sorted token index sequence Extracting the first 1 set of tokens, forming a k-fence candidate token set: ; k-Fence Candidate Token Set The token prediction category distribution obtained from S2 The highest probability One token; The obtained initial candidate token set With the obtained set of k-fence candidate tokens Perform a union operation to form the final candidate token set: ; When S5 obtains the initial cutoff point When the union operation does not change the contents of the set, that is... When the initial cutoff point At that time, the union operation expands the initial candidate token set to include at least the first few tokens. A high-probability token; The final candidate token set formed by statistics The number of tokens, denoted as Token quantity Equal to the adjusted cutoff point ; The final candidate token set will be formed The output serves as the input for probability redistribution in S7; Final candidate token set It takes into account the adaptive cumulative probability threshold obtained by S4. The adaptiveness of the guidance and the set k-barrier parameters Guaranteed minimum exploratory power: at high normalized entropy values The region allows for a larger candidate set to increase sampling diversity at low normalized entropy values. The region uses a k-barrier mechanism to prevent excessive greedy sampling caused by excessive shrinkage of the candidate set, thereby achieving a balance between structural stability and texture richness.

[0029] Specifically, S7 uses a normalization operation on the final candidate token set. Probability redistribution yields a renormalized distribution, including: The final candidate token set formed from S6 Extract all candidate tokens and obtain the token prediction category distribution for each candidate token in S2. The corresponding original probability value; For the final candidate token set Each token in Its original probability value is ; The final candidate token set obtained by calculation The sum of the original probabilities of all candidate tokens: ; Original probability sum The normalization factor is used for the normalization operation. The probabilities of the final candidate token set S_{t, i} obtained by S6 are redistributed and their sum is 1. The final candidate token set obtained from S6 For each candidate token v in S2, the predicted token category distribution is obtained. The original probability value divided by the calculated normalization factor The normalized probability is obtained as follows: For ; Normalized probability For candidate token v in the final candidate token set The relative probability within; For the final candidate token set not obtained in S6 The token in the middle is renormalized to have a probability of 0: ; This enables the prediction of the token category distribution obtained from S2. The truncation operation retains only the final set of candidate tokens. Candidate tokens in; Based on the obtained renormalized probability Construct a renormalized distribution ; Renormalized distribution Satisfying the normalization condition of the probability distribution And only the final candidate token set obtained in S6 The tokens inside have a non-zero probability; Verify the constructed renormalized distribution The sum of probabilities: ; It was confirmed that the renormalized distribution satisfies the basic requirements of a probability distribution. The constructed renormalized distribution The output serves as the input for S8 to perform random sampling; Renormalized distribution The token prediction class distribution obtained in S2 is then normalized. The final candidate token set obtained in S6 The redistribution maintained the relative probability relationships between candidate tokens while excluding those not included in the final candidate token set. The low-probability tokens in the S8 provide a probability distribution for random sampling after entropy guidance and k-barrier constraints.

[0030] Specifically, S8 uses random sampling to select tokens from the renormalized distribution to obtain generated tokens, including: Obtain the renormalized distribution of the S7 output. normalized distribution The sampling probability of each candidate token at position (t, i) is defined; Among them, the final candidate token set obtained by S6 The tokens in the token have a non-zero probability Not in the final candidate token set The probability of obtaining the token is 0. Based on the obtained renormalized distribution Construct the cumulative distribution function ; For the final candidate token set obtained in S6 The j-th token sorted in descending order according to S5.1 Its cumulative distribution function value is: ; Cumulative distribution function Renormalization distribution Mapping to the [0, 1] interval, and implementing random sampling; Generate a random number that is uniformly distributed in the interval [0, 1]. ; random numbers It follows a uniform distribution U(0,1) and satisfies Random token selection is performed based on the renormalized distribution; Based on the generated random number and the constructed cumulative distribution function The target token is determined by the inverse transform sampling method: , in, Generate tokens The final candidate token set obtained by S6 The cumulative distribution function value first reaches or exceeds the random number. The token; Verify the selected generated token The final candidate token set obtained from S6 ,Right now ; Because the S7 normalization operation ensures that only the final candidate token set is used. The tokens in the set have a non-zero probability, therefore random sampling must select tokens from this set; Record the generated token and its index information at position (t, i); Generate tokens The selection probability and the renormalized distribution obtained from S7 The corresponding probability It is proportional to the probability distribution, thus enabling random sampling. The generated token Output the final token generated at position (t, i); The output generated token Add to the sequence of decoded tokens in the current frame t, and update the previous decoded tokens in the current frame. ; Updated decoded token sequence This will be used as a conditional input for the autoregressive model in S2 to predict the token at the next position (t, i+1); The renormalized distribution obtained through the random sampling process using S7 To select a token, the renormalized distribution combines the token prediction class distribution obtained from S2. The original probability information, S3 to S4 based on normalized entropy values The adaptive adjustment and S6's k-barrier mechanism constraint, while maintaining sampling randomness, dynamically balance structural stability and texture diversity based on the uncertainty of model prediction.

[0031] Example 2 A video generation system based on entropy-guided k-fence sampling includes: The video data acquisition module acquires the video data to be generated and represents each frame of the video as a discrete token sequence. , where t is the time step index, i∈{1,...,m} is the intra-frame spatial location index, and V is a discrete codebook of size V.

[0032] The token prediction module uses an autoregressive model to predict the token at position (t, i). Perform predictions and generate token prediction category distributions. The token prediction module will determine the preceding tokens that have been decoded in the current frame. Historical context and historical actions As a conditional input to the autoregressive model, the conditional probability of each candidate token v∈V at position (t, i) is calculated through the autoregressive model. .

[0033] The entropy calculation and normalization module processes the token prediction category distribution obtained by the token prediction module. Calculate Shannon entropy and Shannon entropy Normalization is performed to obtain the normalized entropy value. The entropy calculation and normalization module predicts category distribution based on tokens. Calculate the Shannon entropy at position (t, i). Obtain the size |V| of the codebook V and calculate the theoretical maximum value of Shannon entropy log|V|. Based on the calculated Shannon entropy... and theoretical maximum value Calculate the normalized entropy value .

[0034] The threshold adaptive adjustment module adjusts the normalized entropy value obtained from the entropy calculation and normalization modules. Perform affine transformation and clipping to obtain an adaptive cumulative probability threshold. The threshold adaptive adjustment module obtains the lower bound of the preset normalized entropy value. and the Upper Realm and the lower bound of the cumulative probability threshold. and the Upper Realm Based on these boundary values, calculate the affine transformation mapping parameters α and β, and then adjust the normalized entropy value according to the mapping parameters α and β. An affine transformation is performed to obtain the mapped cumulative probability threshold, and the mapped cumulative probability threshold is then clipped to obtain an adaptive cumulative probability threshold. .

[0035] The truncation and sorting module utilizes the cumulative probability threshold obtained by the threshold adaptive adjustment module. The token prediction category distribution obtained from the token prediction module Perform truncation sorting to obtain truncation point c; the truncation sorting module predicts the class distribution of tokens. All tokens v in the codebook V are assigned their corresponding probabilities The token index sequence is obtained by sorting in descending order. Construct the sorted probability sequence The sorted probability sequences are summed sequentially to calculate the cumulative probability sequence, based on the cumulative probability threshold. The initial cutoff point c is determined by the cumulative probability sequence, and the first c tokens are extracted from the sorted token index sequence to form the initial candidate token set.

[0036] The k-barrier constraint module uses the k-barrier mechanism to apply a lower bound constraint to the cutoff point c obtained by the cutoff sorting module, resulting in the adjusted cutoff point. The final candidate token set is formed based on the adjusted cutoff point c'. The k-fence constraint module obtains preset k-fence parameters. The initial cutoff point c and the k-guardrail parameters are set together. The maximum value of the two values ​​is used as the adjusted cutoff point. Extract the first c' tokens from the sorted token index sequence to form a k-barrier candidate token set. Perform a union operation between the initial candidate token set and the k-barrier candidate token set to form the final candidate token set. .

[0037] The probability renormalization module uses normalization operations to form the final candidate token set of the k-fence constraint module. Probability redistribution is performed to obtain a renormalized distribution; the probability renormalization module selects from the final candidate token set. Extract all candidate tokens and obtain the distribution of each candidate token in the token prediction category. The original probability values ​​are used to calculate the final candidate token set. The sum of the original probabilities of all candidate tokens As a normalization factor, for the final candidate token set Each candidate token v in the dataset is determined based on its original probability value and normalization factor. Calculate the renormalized probability And construct a normalized distribution based on the normalized probabilities. .

[0038] The random sampling module uses random sampling to select tokens from the renormalized distribution obtained by the probability renormalization module, thus generating tokens; the random sampling module selects tokens based on the renormalized distribution. Constructing the cumulative distribution function Generate a random number that is uniformly distributed in the interval [0, 1]. According to random numbers and cumulative distribution function The target token to be generated is determined using the inverse transform sampling method. And will generate tokens Add to the sequence of decoded tokens in the current frame.

[0039] The iterative control module controls the iterative execution of the video data acquisition module, token prediction module, entropy calculation and normalization module, threshold adaptive adjustment module, truncation sorting module, k-barrier constraint module, probability renormalization module, and random sampling module until a complete video frame sequence is generated.

[0040] Example 3 Since ENKG is a plug-and-play inference strategy, this application integrates it into several existing autoregressive visual world models for experimental purposes, including DrivingWorld (Hu et al., 2024), VaVIM (Bartoccionie et al., 2025), and Cosmos (NVIDIA, 2025). To ensure fair comparison, the remaining parameters in the generation phase remain consistent with the original models. Specifically, DrivingWorld uses top-k (k = 30), VaVIM uses greedy sampling, and Cosmos uses top-p (p = 0.8).

[0041] As shown in Tables 1 and 2, this application conducted two sets of comparisons on the self-built dataset of DrivingWorld: (1) Component ablation (Table 1): Removing entropy mapping (w / o Entropy) or k-guard (w / o k-Guard) will significantly worsen FVD, FID, and LPIPS, or decrease PSNR, indicating that both components are indispensable; the full strategy achieves the best results in all four indicators. (2) Comparison with common decoding strategies (Table 2): Compared with Greedy / Top-k, Top-p, and PK (hybrid strategy), the entropy sampling method of this application is significantly better in the three perceptual indicators of FVD, FID, and LPIPS, and also has the highest PSNR, demonstrating the advantages of stability and detail fidelity in long-term generation.

[0042] Table 1 Ablation study of the DrivingWorld model on a self-built dataset (comparing the effects of each component of ENKG)

[0043] Table 2 Comparison of different decoding strategies on DrivingWorld (self-built dataset)

[0044] This application evaluates its model on two datasets: DiverseDrive and nuPlan. DiverseDrive is a high-quality driving video dataset built in this application, containing 50 video clips. Compared to nuPlan, DiverseDrive covers more scenes and contains richer vegetation and details. These characteristics help evaluate the model's generalization ability in a wider range of scenarios, thus more closely reflecting the actual needs of real-world model evaluation.

[0045] To measure the quality of the generated video, this application reports video-level Fréchet Video Distance (FVD) to evaluate video realism and frame-level Fréchet Inception Distance (FID) to evaluate single-frame image fidelity. In addition, this application also reports lower-level metrics such as LPIPS, PSNR, and SSIM as supplementary evaluations (note that these lower-level metrics are not entirely suitable for video generation tasks).

[0046] As shown in Table 3, integrating ENKG into different architectures yields robust and significant improvements. On DiverseDrive, the method in this application reduces FVD and FID by an average of approximately 22.8% and 36.5%, respectively, while reducing LPIPS and improving PSNR / SSIM, indicating that the method benefits both perceptual quality and structure preservation. Even for DrivingWorld, which is already fully trained on NuPlan, ENKG still brings performance improvements. Even on Cosmos, where the autoregressive backbone is relatively weak, this application also observes a modest improvement on DiverseDrive. It should be noted that VaVIM has a tendency to frame-freezing in the baseline settings, which artificially lowers FVD; in contrast, the strategy in this application effectively alleviates this problem, resulting in more realistic dynamic performance.

[0047] Table 3: Quantitative results for Saturn and Nuplan. * Cosmos uses a fixed 33-frame generation window, therefore its metrics are calculated based on the first 33 frames of each sequence (other models use 75 frames).

[0048] like Figure 2 and Figure 3As shown, existing sampling techniques often lead to texture degradation: key details such as road markings, zebra crossings, and surrounding vegetation become blurry and unclear; at the same time, they are prone to color distortion or color dilution, destroying the realism of the scene. VaVIM may even collapse into static or near-static frames at the baseline, failing to reflect the actual dynamics in the driving environment, and the vehicle appears to be stationary. In contrast, the entropy-guided strategy of this application estimates the Shannon entropy of the prediction distribution for each location (token) in real time, and monotonically maps this entropy to a local nucleus (cumulative probability) threshold pt,ip_{t,i}pt,i, which is then joined with the top-kkk guardrail set to form the final sampling candidate set. This mechanism allows high-uncertainty regions to retain a larger candidate range to maintain diversity, while low-uncertainty regions tighten the candidates to stabilize the structure, thereby significantly reducing degradation phenomena such as texture smoothing, color drift, and frame skipping in long-term generation, while maintaining controllable computational overhead.

[0049] The core of entropy adaptive guidance lies in dynamically adjusting the sampling kernel based on the uncertainty of model predictions. For example... Figure 4 As shown in Figure 4, this mechanism effectively mitigates the texture degradation and color drift problems commonly found in baseline methods: it allows for a wider range of token candidates when uncertainty is high, and narrows the candidate set to avoid redundant noise when uncertainty is low, thereby preserving high-frequency details (such as road markings and vegetation textures). This location-adjusted diversity control yields direct benefits in frame-level visual fidelity, reflected in a decrease in FID and LPIPS.

[0050] The k-barrier mechanism ensures minimal diversity even in high-confidence (low-entropy) regions. Removing the k-barrier, even with entropy-adaptive guidance, can lead to overconfidence in certain situations, resulting in a lack of temporal consistency (e.g., vehicles should be moving, but in the generated sequence, vehicles are approximately stationary). Figure 6 As shown, the introduction of k-barriers can directly alleviate such failure modes, making the motion dynamics smoother and more physically plausible, which is also quantitatively reflected in the FVD metric. Therefore, k-barriers are crucial for maintaining temporal continuity and preventing the generation of degenerate sequences into static sequences.

[0051] In summary, this application proposes Uncertainty-aware Adaptive Sampling, a simple yet effective inference-stage strategy based on prediction entropy supplemented by minimum k-barriers. Unlike previous methods that require modifications to the network structure or retraining, this method operates only during inference, thus allowing for seamless application to existing large-scale video generation models. Extensive experiments demonstrate that ENKG significantly improves temporal consistency, preserves fine-grained texture, and extends the effective generation temporal domain. The results show that the inference-time uncertainty-based strategy provides a practical and general path for building more robust and high-fidelity video world models.

[0052] The foregoing illustrative description of the present application and its embodiments is not restrictive and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. The accompanying drawings are only one embodiment of the present application, and the actual structure is not limited thereto. Therefore, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the present application, such designs should fall within the scope of protection of this application. Furthermore, the use of the word "include" does not exclude other elements or steps, and a single word preceding an element does not exclude the inclusion of multiple such elements. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A video generation method based on entropy-guided k-fence sampling, characterized in that, include: S1, Obtain the video data to be generated, and represent each frame of the video as a discrete token sequence. Where t is the time step index, i∈{1,...,m} is the intra-frame spatial location index, and V is the discrete codebook; S2, using an autoregressive model to evaluate the token at position (t, i). Perform predictions and generate token prediction category distributions. ; S3, calculate the token prediction category distribution respectively. Shannon entropy And according to Shannon entropy Calculate the adaptive cumulative probability threshold ; S4, based on the cumulative probability threshold Determining the cutoff point using the k-barrier mechanism And based on the cutoff point Determine the final candidate token set ; S5, based on random sampling method for the final candidate token set Sampling is performed to obtain the target token; S6: Iterate through each video frame and decode the video according to the target token to obtain the complete video frame sequence.

2. The video generation method based on entropy-guided k-fence sampling according to claim 1, characterized in that: S2, using an autoregressive model to evaluate the token at position (t, i). Making predictions includes: The decoded preceding token of the current frame Historical context and historical actions As a conditional input to the autoregressive model; The conditional probability of each candidate token v∈V at position (t, i) is calculated using an autoregressive model. .

3. The video generation method based on entropy-guided k-fence sampling according to claim 1, characterized in that: According to Shannon entropy Calculate the adaptive cumulative probability threshold ,include: Obtain the size |V| of the codebook V, and calculate the theoretical maximum value of the Shannon entropy. ; Based on the calculated Shannon entropy The theoretical maximum value obtained Calculate the normalized entropy value ; Obtain the lower bound of the preset normalized entropy value and the Upper Realm and the lower bound of the cumulative probability threshold. and the Upper Realm ; Based on the lower bound of the normalized entropy value and the Upper Realm and the lower bound of the cumulative probability threshold. and the Upper Realm Calculate the mapping parameters α and β of the affine transformation; Based on the mapping parameters α and β of the affine transformation, the normalized entropy value... Perform an affine transformation to obtain the cumulative probability threshold after mapping; The mapped cumulative probability threshold is then pruned to obtain an adaptive cumulative probability threshold. .

4. The video generation method based on entropy-guided k-fence sampling according to claim 1, characterized in that: Based on the cumulative probability threshold Determining the cutoff point using the k-barrier mechanism ,include: Predict the category distribution of the tokens obtained in S2. All tokens v in the codebook V are assigned their corresponding probabilities Sort in descending order to obtain the sorted token index sequence. And construct the sorted probability sequence ; For the sorted probability sequence Accumulate sequentially to calculate the cumulative probability sequence; Adaptive cumulative probability threshold obtained based on S4 Based on the calculated cumulative probability sequence, determine the initial cutoff point; Based on the determined initial cutoff point c, the first c tokens are extracted from the sorted token index sequence to form an initial candidate token set; Get the preset k-fence parameters ; The initial cutoff point c obtained from S5 and the k-guardrail parameters are compared. Compare the two values ​​and take the maximum value as the adjusted cutoff point. .

5. The video generation method based on entropy-guided k-fence sampling according to claim 4, characterized in that: Obtain the target token, including: According to the cutoff point From the sorted token index sequence Extracting the first Each token forms a k-barrier candidate token set; The obtained initial candidate token set With the obtained set of k-fence candidate tokens Perform a union operation to form the final candidate token set. .

6. The video generation method based on entropy-guided k-fence sampling according to claim 5, characterized in that: Obtaining the target token also includes: The final candidate token set formed from S6 Extract all candidate tokens and obtain the token prediction category distribution for each candidate token in S2. The corresponding original probability value ; Based on the final candidate token set Calculate the sum of the original probabilities of all candidate tokens. ; The final candidate token set obtained from S6 For each candidate token v, predict the category distribution based on the corresponding token. The original probability values ​​and normalization factors in the data. Calculate the corresponding renormalized probability. ; Based on the renormalized probability Construct a renormalized distribution .

7. The video generation method based on entropy-guided k-fence sampling according to claim 6, characterized in that: Obtaining the target token also includes: According to the renormalization distribution Construct the cumulative distribution function ; Generate a random number that is uniformly distributed in the interval [0, 1]. ; Based on random numbers and cumulative distribution function The target token is determined by the inverse transform sampling method. .

8. A video generation system based on entropy-guided k-fence sampling, characterized in that, include: The video data acquisition module acquires the video data to be generated and represents each frame of the video as a discrete token sequence. ; The token prediction module uses an autoregressive model to predict the token at position (t, i). Perform predictions and generate token prediction category distributions. ; The entropy calculation and normalization module processes the token prediction category distribution obtained by the token prediction module. Calculate Shannon entropy and Shannon entropy Normalization is performed to obtain the normalized entropy value. ; The threshold adaptive adjustment module adjusts the normalized entropy value obtained from the entropy calculation and normalization modules. Perform affine transformation and clipping to obtain an adaptive cumulative probability threshold. ; The truncation and sorting module utilizes the cumulative probability threshold obtained by the threshold adaptive adjustment module. The token prediction category distribution obtained from the token prediction module Perform a truncation sort to obtain the cutoff point c; The k-barrier constraint module uses the k-barrier mechanism to apply a lower bound constraint to the cutoff point c obtained by the cutoff sorting module, resulting in the adjusted cutoff point. Based on the adjusted cutoff point Forming the final candidate token set ; The probability renormalization module uses normalization operations to form the final candidate token set of the k-fence constraint module. Probability redistribution is performed to obtain a renormalized distribution; The random sampling module uses random sampling to select tokens from the renormalized distribution obtained by the probability renormalization module, thus generating tokens. The iterative control module controls the iterative execution of the video data acquisition module, token prediction module, entropy calculation and normalization module, threshold adaptive adjustment module, truncation sorting module, k-barrier constraint module, probability renormalization module, and random sampling module until a complete video frame sequence is generated.

9. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 7.

Images