Method, device, and storage medium for generating first and last frame videos based on anchor point frame bridging

Abstract

This application discloses a method, device, and storage medium for generating first and last frame videos based on anchor frame bridging, relating to the field of video generation technology. The method includes: performing a frame order reversal operation based on the input first and last frames to generate a reverse video sequence; evaluating the quality of each frame in the reverse video sequence; determining the optimal anchor frame from the reverse video sequence based on the quality evaluation results; and inputting the first frame, the last frame, and the optimal anchor frame as joint conditions, combined with text prompts, into a video generation model to generate a target video. This application constructs a semantic transmission bridge from the first frame to the last frame by adaptively inserting high-quality anchor frames at key temporal positions, effectively solving the problems of semantic distortion and abrupt transitions in intermediate frames caused by attention decay, and significantly improving the temporal coherence and overall visual quality of the generated video.

Description

Technical Field

[0001] This application relates to the field of video generation technology, and in particular to a method, device and storage medium for generating first and last frame videos based on anchor frame bridging. Background Technology

[0002] First and last frame video generation aims to generate dynamic videos with coherent beginning and end semantics and reasonable content in the middle frames, based on a given start frame, end frame, and text prompts. It is widely used in video editing, dynamic content creation, and physical modeling. Currently, first and last frame video generation is implemented by reusing the image-to-video (I2V) model, using the first and last frames as control inputs to guide the generation of the video sequence.

[0003] Because I2V models generally rely on self-attention mechanisms to achieve inter-frame information exchange, this mechanism, when modeling long sequences, only assigns high attention weights to adjacent or local frames, making it difficult to establish effective cross-frame semantic relationships between the first and last frames. In the task of generating first and last frame videos, the deterministic semantic information of the first and last frames decays rapidly with increasing temporal distance as it propagates to the intermediate frames. This leads to problems such as scene distortion, subject distortion, or abrupt content changes in the intermediate frames, severely damaging the temporal consistency and visual coherence of the video, and failing to meet the basic requirements of accuracy and smooth transition of intermediate frames for high-quality first and last frame video generation.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a method, device and storage medium for generating first and last frame videos based on anchor frame bridging, which aims to solve the technical problem of distortion of intermediate frame content caused by semantic attenuation in the first and last frame video generation method.

[0006] To achieve the above objectives, embodiments of this application provide a method for generating first and last frame videos based on anchor frame bridging, the method comprising: Perform a frame order reversal operation based on the first and last input frames to generate a reverse video sequence; The quality of each frame in the reverse video sequence is evaluated, and the optimal anchor frame is determined from the reverse video sequence based on the quality evaluation results. The first frame, the last frame, and the optimal anchor frame are used as joint conditions, combined with text prompts, and input into the video generation model to generate the target video.

[0007] In one embodiment, the step of performing a frame order reversal operation based on the input first and last frames to generate a reverse video sequence includes: The timing positions of the first frame and the last frame are swapped to form a reverse input frame pair; Generate a reverse text prompt based on the reverse input frame pair; The reverse input frame pair and the reverse text prompt are input into the video generation model to generate the reverse video sequence.

[0008] In one embodiment, the step of performing quality assessment on each frame in the reverse video sequence and determining the optimal anchor frame from the reverse video sequence based on the quality assessment results includes: For each frame in the reverse video sequence, calculate the perceptual similarity value between the frame and its preceding and following adjacent frames to obtain the quality assessment value of each frame. Based on the quality assessment value, determine the frame with the lowest quality assessment value from the reverse video sequence; Based on the temporal position corresponding to the frame with the lowest quality assessment value, the target temporal position corresponding to the temporal position in the reverse video sequence is determined by mirror mapping. The frame at the target temporal position in the reverse video sequence is determined as the optimal anchor frame.

[0009] In one embodiment, the step of calculating the perceptual similarity value between each frame and its preceding and following adjacent frames to obtain a quality assessment value for each frame in the reverse video sequence includes: By calculating the first perceptual similarity value between the frame and the previous adjacent frame, and the second perceptual similarity value between the frame and the next adjacent frame; Based on the first perceptual similarity value and the second perceptual similarity value, a quality assessment value for the frame is obtained, wherein the lower the quality assessment value, the worse the semantic continuity between the frame and its adjacent frames.

[0010] In one embodiment, after the step of determining the frame at the target temporal position in the reverse video sequence as the optimal anchor frame, the method for generating the first and last frame videos based on anchor frame bridging further includes: When constructing the guide frame sequence, the target temporal position is determined as the target insertion position of the optimal anchor frame.

[0011] In one embodiment, the step of using the first frame, the last frame, and the optimal anchor frame as joint conditions, combined with text prompts, as input to the video generation model to generate the target video includes: A positive text prompt is generated based on the first frame and the last frame, and the positive text prompt is semantically encoded to obtain text features; Extract the visual features of the first frame and the last frame, and then concatenate the visual features corresponding to the first frame and the last frame to obtain joint visual features; Based on the temporal positions corresponding to the first frame, the last frame, and the optimal anchor frame, a binary mask is constructed. The binary mask is used to identify known frames and intermediate frames to be generated. The first frame, the last frame, the optimal anchor frame, and the zero-fill frame are concatenated in chronological order to form a guide frame sequence, and the guide frame sequence is subjected to latent spatial encoding to obtain latent features; The text features, the joint visual features, the latent features, and the binary mask are input into the video generation model to generate the target video.

[0012] In one embodiment, the step of inputting the text features, the joint visual features, the latent features, and the binary mask into the video generation model to generate the target video includes: The text features, the joint visual features, the latent features, and the binary mask are input into the diffusion Transformer architecture of the video generation model; In the diffusion Transformer architecture, the latent features are used as the denoising object, and based on the binary mask, the text features and the joint visual features are used as conditional information through a decoupled cross-attention mechanism to guide the multi-round iterative denoising process of the latent features. The multi-round iterative denoising process is executed to gradually eliminate noise in the latent features and generate clean latent features. The clean latent features are decoded to obtain the target video.

[0013] In one embodiment, the step of generating a positive text prompt based on the first frame and the last frame includes: Input the first frame and the last frame into the visual language model; The visual semantic content of the first frame and the last frame is analyzed by the visual language model, and the semantic evolution relationship from the first frame to the last frame is inferred based on the visual semantic content. Based on the semantic evolution relationship, positive text prompts are generated to represent the video content and the inter-frame action transition process.

[0014] This application embodiment also provides a first and last frame video generation device based on anchor frame bridging. The first and last frame video generation device based on anchor frame bridging includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the first and last frame video generation method based on anchor frame bridging as described above.

[0015] This application embodiment also provides a storage medium, which is a computer-readable storage medium, and stores a computer program on the storage medium. When the computer program is executed by a processor, it implements the steps of the first and last frame video generation method based on anchor frame bridging as described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: This application generates a reverse video sequence by performing a frame order reversal operation on the first and last input frames. Leveraging the strong constraints imposed by the original last frame on the first half of the reverse video sequence, resulting in higher visual quality and semantic certainty, it provides a high-quality candidate basis for selecting the optimal anchor frame, avoiding the semantic decay issue that occurs when directly selecting anchor frames from the forward sequence. By evaluating the quality of each frame in the reverse video sequence and determining the optimal anchor frame, it can accurately locate key frames with strong semantic constraints, effectively bridging the semantic transfer between the first and last frames and solving the semantic constraint problem in the middle region of the forward video sequence. To address the weakness of the model, this paper further addresses the issue by using the first frame, last frame, and optimal anchor frame as joint conditions, combined with text prompts input into the video generation model. This enables multi-frame conditions and text semantics to work together to provide a more comprehensive temporal and semantic reference for the video generation model, effectively suppressing semantic drift and content distortion in intermediate frames. Furthermore, this application achieves this through only the core steps of frame order reversal, quality assessment, and joint condition input, without modifying the video generation model itself. This ensures both the generation effect and the adaptability of the solution, ultimately achieving a significant improvement in the temporal consistency and visual quality of the first and last frame video generation. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an embodiment of the video generation method for first and last frames based on anchor frame bridging in this application. Figure 2 This is a schematic diagram of the architecture provided in Embodiment 1 of the video generation method for first and last frames based on anchor frame bridging in this application; Figure 3 This is a schematic diagram illustrating the effect of the first and last frame video generation method based on anchor frame bridging provided in this application. Figure 4 This is a schematic diagram of the hardware operating environment of the first and last frame video generation device based on anchor frame bridging, which is involved in the first and last frame video generation method based on anchor frame bridging in the embodiments of this application.

[0018] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0020] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0021] First and last frame video generation aims to generate dynamic videos with coherent beginning and end semantics and reasonable content in the middle frames, based on a given start frame, end frame, and text prompts. It is widely used in video editing, dynamic content creation, and physical modeling. Currently, first and last frame video generation is implemented by reusing the image-to-video (I2V) model, using the first and last frames as control inputs to guide the generation of the video sequence.

[0022] Because I2V models generally rely on self-attention mechanisms to achieve inter-frame information exchange, this mechanism, when modeling long sequences, only assigns high attention weights to adjacent or local frames, making it difficult to establish effective cross-frame semantic relationships between the first and last frames. In the task of generating first and last frame videos, the deterministic semantic information of the first and last frames decays rapidly with increasing temporal distance as it propagates to the intermediate frames. This leads to problems such as scene distortion, subject distortion, or abrupt content changes in the intermediate frames, severely damaging the temporal consistency and visual coherence of the video, and failing to meet the basic requirements of accuracy and smooth transition of intermediate frames for high-quality first and last frame video generation.

[0023] In view of the above problems, this application proposes a first and last frame video generation method based on anchor frame bridging. The method generates a reverse video sequence by performing a frame order reversal operation based on the input first and last frames; it then performs quality evaluation on each frame in the reverse video sequence and determines the optimal anchor frame from the reverse video sequence based on the quality evaluation results; finally, it uses the first frame, the last frame, and the optimal anchor frame as joint conditions, combined with text prompts, as input to a video generation model to generate the target video.

[0024] This application provides a solution that generates a reverse video sequence by performing a frame order reversal operation on the first and last input frames. Leveraging the strong constraints of the original last frame on the first half of the reverse video sequence, resulting in higher visual quality and semantic certainty, a high-quality candidate basis is provided for selecting the optimal anchor frame. This avoids the problem of semantic attenuation when directly selecting anchor frames from the forward sequence. By evaluating the quality of each frame in the reverse video sequence and determining the optimal anchor frame, key frames with strong semantic constraints can be accurately located, effectively bridging the semantic transfer between the first and last frames and solving the problem of semantic degradation in the middle section of the forward video sequence. This addresses the issue of weak domain semantic constraints. Furthermore, by using the first frame, last frame, and optimal anchor frame as joint conditions, combined with textual prompts input into the video generation model, multi-frame conditions and textual semantics work together to provide a more comprehensive temporal and semantic reference for the video generation model, effectively suppressing semantic drift and content distortion in intermediate frames. Moreover, this application achieves this only through the core steps of frame order reversal, quality assessment, and joint condition input, without modifying the video generation model itself. This ensures both the generation effect and the adaptability of the solution, ultimately achieving a significant improvement in the temporal consistency and visual quality of the first and last frame video generation.

[0025] It should be noted that the executing entity in this embodiment can be a computing service device with image processing, network communication, and program execution functions, such as a computer, mobile phone, cloud service platform, etc., or an electronic device capable of realizing the above functions, a first- and last-frame video generation system based on anchor frame bridging, etc. The following description uses a first- and last-frame video generation system based on anchor frame bridging as an example to illustrate this embodiment and the following embodiments.

[0026] The first embodiment of the video generation method for first and last frames based on anchor frame bridging proposed in this application is described in reference. Figure 1 The method includes steps S10 to S30: Step S10: Perform a frame order reversal operation based on the input first and last frames to generate a reverse video sequence.

[0027] It should be noted that the first frame refers to the starting frame image of the target video to be generated, carrying the initial visual content and semantic information of the video; the last frame refers to the ending frame image of the target video to be generated, carrying the final visual content and semantic information of the video. Together, they constitute the boundary conditions for video generation. The frame order reversal operation refers to swapping the input order of the first and last frames, transforming the original forward video generation task from the first frame to the last frame into a reverse video generation task from the last frame to the first frame. The reverse video sequence refers to the complete video frame sequence generated by the above frame order reversal operation. The visual content of this video frame sequence is a dynamic transition from the original last frame to the original first frame in the time dimension.

[0028] In video generation tasks based on the first and last frames, current methods often employ image-to-video (I2V) models to directly generate the target video. However, this approach relies on a self-attention mechanism to propagate semantic information, with significant attention weights only existing between adjacent frames. This leads to a gradual attenuation of the deterministic semantics of the first and last frames as it propagates to the middle frames, resulting in issues such as semantic drift in the middle frames, scene distortion, and subject deformation. This step, through frame order reversal, can uncover potentially semantically related frames between the first and last frames, providing candidate samples for subsequent semantic breakpoint location. Simultaneously, the reversed frame sequence after frame order reversal forms a mirror mapping relationship with the original frame sequence (i.e., the forward frame sequence), which helps subsequent screening steps accurately pinpoint key temporal positions. This lays the data foundation for inserting anchor frames at the points of maximum semantic breakpoint, mitigating the risk of semantic disconnect between the middle frames and the first and last frames, and improving the temporal consistency of the generated target video.

[0029] Specifically, step S10 includes steps S110 to S130: Step S110: Swap the timing positions of the first frame and the last frame to form a reverse input frame pair.

[0030] Step S120: Generate a reverse text prompt based on the reverse input frame pair.

[0031] Step S130: Input the reverse input frame pair and the reverse text prompt into the video generation model to generate the reverse video sequence.

[0032] In this embodiment, the first frame is swapped first. With the last frame The position is used to obtain the inverse input frame pair. Then, the Qwen model is used to analyze the visual semantic content and dynamic transition relationship of the inverse input frame pairs to generate inverse text prompts. And encode the reverse text hints as text embeddings. Simultaneously, the inverse input frame pairs are encoded using a VAE encoder, transforming them into conditional inputs. Finally, input the conditions. and text embedding The input video generation model (I2V model) is used to generate an inverse video sequence through iterative denoising using a diffusion model.

[0033] Understandably, because inter-video generation (I2V) models are more sensitive to input conditions at the beginning of a sequence, the generated initial portion is usually of higher quality. By performing a frame reversal operation, using the original high-quality tail frame as the starting condition for the reverse generation task, the video generation model can be prompted to generate a reverse video sequence with a significantly higher initial quality. Based on this characteristic, the first half of this reverse video sequence, due to the strong semantic constraints of the original tail frame (i.e., the reverse first frame), exhibits particularly outstanding generation quality. Therefore, the frames in the first half of the reverse video sequence can be used as a high-quality anchor frame candidate set, so that the key frames that best represent the semantic break regions can be selected as representative anchor frames later.

[0034] Step S20: Perform quality assessment on each frame in the reverse video sequence, and determine the optimal anchor frame from the reverse video sequence based on the quality assessment results.

[0035] It's important to note that quality assessment refers to quantifying the semantic continuity and visual coherence between each frame in the reverse video sequence and its preceding and following frames, thereby measuring the smoothness of the local transition at that frame's location. A lower quality assessment value indicates a less smooth semantic transition and poorer semantic continuity at that frame's location. The optimal anchor frame is a key frame selected from the reverse video sequence using mirror mapping rules that corresponds to the frame with the lowest located quality assessment value. Its role is to serve as an intermediate bridging frame to guide video generation in the forward video generation task.

[0036] Specifically, step S20 includes steps S210 to S240: Step S210: For each frame in the reverse video sequence, calculate the perceptual similarity value between the frame and its preceding and following adjacent frames to obtain the quality assessment value of each frame.

[0037] It should be noted that perceptual similarity is a measure of inter-frame difference calculated in the embedding space based on high-level visual features extracted by deep neural networks. It can effectively assess the semantic consistency of two images at the human perceptual level. Compared to directly calculating differences in pixel space, this measure based on high-level visual features can more stably and accurately reflect the coherence of high-level semantic content such as changes in subject pose, evolution of scene structure, and transfer of object relationships.

[0038] In this step, the perceptual similarity values ​​of the target frame with its preceding and following adjacent frames are calculated separately, and these two perceptual similarity values ​​are fused to generate a comprehensive quality assessment value. This quality assessment value aims to quantify the smoothness of the semantic transition of the target frame within its local temporal context, providing objective data for locating semantic break regions.

[0039] In this embodiment, LPIPS (Learned Perceptual Image Patch Similarity) is used to evaluate the quality of each frame in the reverse video sequence. LPIPS is a perceptual image similarity measurement method based on pre-trained convolutional neural networks (such as VGG or ResNet). By calculating the distance between two frames in a deep feature space, it models the sensitivity of the human visual system to image differences, thereby effectively capturing the semantic discontinuities between frames.

[0040] Specifically, step S210 includes steps S2110 to S2120: Step S2110: Calculate the first perceptual similarity value between the frame and the previous adjacent frame, and the second perceptual similarity value between the frame and the next adjacent frame.

[0041] Step S2120: Based on the first perceptual similarity value and the second perceptual similarity value, obtain the quality assessment value of the frame, wherein the lower the quality assessment value, the worse the semantic continuity between the frame and adjacent frames.

[0042] In this embodiment, a frame quality evaluation function is defined. ,in, The first in the reverse video sequence frame, (Excluding the first and last frames to ensure there are adjacent frames). Since a higher LPIPS value indicates greater semantic differences and poorer semantic continuity between frames, a negative sign is used. The smaller the value, the more severe the semantic break at that frame location.

[0043] In practice, before conducting quality assessment, each frame in the reverse video sequence can be preprocessed, including size normalization and noise removal, to avoid visual interference factors affecting the accuracy of the assessment results.

[0044] Step S220: Based on the quality assessment value, determine the frame with the lowest quality assessment value from the reverse video sequence.

[0045] In this embodiment, the key locations with the worst semantic continuity are identified from the quantitative evaluation results through a positioning operation. The frame with the lowest quality evaluation value has the most significant semantic discontinuity in its local location, and is the most likely point of risk for semantic breaks or content abrupt changes during the semantic transition from the first frame to the last frame.

[0046] Specifically, this is achieved by calculating the quality assessment value of each frame in the reverse video sequence. , find Minimum frame position Normalize the frame position with the lowest quality assessment value; the corresponding normalized position is then... ,in This represents the total number of frames in the reverse video sequence.

[0047] It should be noted that, to facilitate subsequent mirror mapping calculations, the frame position is usually... Convert to normalized position parameters Normalized position This refers to mapping discrete frame numbers to... The continuous-time representation of the interval is used to eliminate scale differences caused by variations in the total number of frames in a video sequence.

[0048] Step S230: Based on the temporal position corresponding to the frame with the lowest quality assessment value, determine the target temporal position in the reverse video sequence through mirror mapping.

[0049] It should be noted that mirror mapping is a pre-defined geometrical symmetry transformation. Its core principle is to map the input temporal position relative to the center point of the reverse video sequence to obtain its mirror position. In this embodiment, the temporal position corresponding to the frame with the lowest quality evaluation value (i.e., the normalized position) is used. ), calculate the relationship between the mirror image and the reverse video sequence. Target temporal position in mirror relationship ,but ,in This represents the total number of frames in the reverse video sequence.

[0050] Step S240: Determine the frame at the target temporal position in the reverse video sequence as the optimal anchor frame.

[0051] In this embodiment, the target temporal position calculated according to step S230 is... Extract the frame located at the target temporal position from the reverse video sequence and determine it as the anchor frame. The anchor frame It has strong semantic consistency with the first and last frames and can serve as an intermediate semantic bridge.

[0052] Understandably, the frame mirror-symmetric to the frame with the lowest quality evaluation value is chosen as the optimal anchor frame. This is because, during the generation of the reverse video sequence, the region near the initial frame (also known as the reverse first frame, or the original last frame) is subject to strong boundary constraints and typically exhibits higher visual fidelity and semantic consistency. Conversely, the most severe semantic breaks often occur in the latter half of the sequence, far from the initial frame. Through mirror mapping, this most severe semantic break is mapped to the first half of the reverse video sequence, near the initial frame, thus ensuring the selected anchor frame possesses high semantic quality. In the subsequent forward video generation task, this anchor frame is placed in the target temporal position slot symmetrical to the location of the most severe semantic break. This introduces a high-confidence intermediate guiding signal in the region most prone to semantic distortion, effectively enhancing the conditional constraint capability of the video generation model in this critical transition region and improving the temporal coherence of the generated target video.

[0053] Optionally, after step S240, when constructing the guide frame sequence, the target temporal position is determined as the target insertion position of the optimal anchor frame. Here, the guide frame sequence is a pre-constructed temporal reference structure for the forward video generation task, including the first frame, the last frame, the optimal anchor frame, and reserved insertion slots for intermediate frames (i.e., zero-fill frames). Using the target temporal position obtained through mirror mapping as the insertion position of the optimal anchor frame enables precise matching between the optimal anchor frame and the region with the most severe semantic breaks in the forward video sequence. Simultaneously, determining this insertion position requires no additional temporal calibration operations and can be directly used as input conditions for the video generation model, providing a clear semantic guidance benchmark for the subsequent generation of coherent and stable target videos.

[0054] In the generation of long video sequences, semantic information exhibits non-uniform attenuation as it propagates from the first frame to the last, typically showing significant semantic discontinuities in key semantic transition intervals such as abrupt action changes, subject interactions, or scene transitions. This embodiment quantifies and evaluates the local semantic continuity between each frame in the reverse video sequence and its adjacent frames, effectively locating the position with the most severe semantic break. Then, based on the mirror symmetry of the reverse video sequence, a higher-quality target temporal position in the reverse video sequence is mapped from this position, and the corresponding frame is selected as the optimal anchor frame. Subsequently, the target temporal position of this optimal anchor frame in the reverse video sequence is mapped to the corresponding temporal interval of the target video to be generated, thus adaptively introducing the optimal anchor frame with strong semantic constraints as an additional condition at the corresponding temporal position in the forward video sequence. This optimal anchor frame, together with the first and last frames, forms a multi-frame joint guiding signal, providing strong semantic constraints for the video generation model during diffusion denoising, thereby suppressing semantic drift, content distortion, or temporal breaks caused by condition sparsity in the middle region, significantly improving the overall coherence and semantic consistency of the generated video.

[0055] Step S30: The first frame, the last frame, and the optimal anchor frame are used as joint conditions, combined with text prompts, and input into the video generation model to generate the target video.

[0056] It should be noted that text prompts are semantic instructions used to describe the video content and the evolution of actions, which can guide the video generation model to generate target video content that meets expectations.

[0057] In this embodiment, a diffusion model with a decoupled Transformer architecture, such as the I2V model, is preferably used as the video generation model.

[0058] Specifically, step S30 includes steps S310 to S350: Step S310: Generate a positive text prompt based on the first frame and the last frame, and perform semantic encoding on the positive text prompt to obtain text features.

[0059] It should be noted that positive text prompts refer to natural language statements that describe the expected dynamic evolution process from the first frame to the last frame, and are used to provide high-level semantic guidance for video generation models.

[0060] As one possible implementation, step S310 includes steps S3110 to S3130: Step S3110: Input the first frame and the last frame into the visual language model.

[0061] In this embodiment, the first and last frames of input need to be normalized in size and standardized in pixel value to ensure that they match the input requirements of the visual language model. The visual language model can be a pre-trained model with image understanding and text generation capabilities, such as Qwen, BLIP-2, and LLaVA. The choice of different models can be determined based on semantic understanding accuracy and generation efficiency.

[0062] In the specific implementation process, the order of the first and last frames must be maintained during input to avoid errors in semantic evolution reasoning due to frame reversal. At the same time, the input image can be enhanced to improve the robustness of the model to complex scenes.

[0063] Step S3120: Analyze the visual semantic content of the first frame and the last frame using the visual language model, and infer the semantic evolution relationship from the first frame to the last frame based on the visual semantic content.

[0064] It should be noted that semantic evolution relationship refers to the logical association such as action changes, state transitions, or scene changes from the first frame to the last frame.

[0065] In this embodiment, the visual language model can parse the visual semantic content of the first and last frames separately through a multi-round attention mechanism, and then establish the temporal relationship between the two through a contextual reasoning module to generate a semantic evolution description that includes the subject of the action, the process of the action, and the state changes. During the reasoning process, it is necessary to ensure the continuity of the subject's actions and the consistency of the scene to avoid generating logically contradictory semantic evolution relationships. At the same time, relevant domain knowledge constraints can be introduced to improve the rationality of the reasoning results.

[0066] Step S3130: Based on the semantic evolution relationship, generate the positive text prompts used to represent the video content and the inter-frame action transition process.

[0067] In this embodiment, the visual language model can be guided to generate positive text prompts that meet the requirements through prompt word engineering. For example, by inputting the prompt instruction "describe the complete action process from the first frame to the last frame", the visual language model can generate coherent text containing the start, development and end of the action. Alternatively, template filling can be used to fill the key elements in the semantic evolution relationship into a preset template to generate standardized positive text prompts.

[0068] As an example, the original first and last frame pairs are first analyzed using the Qwen model. Generate positive text prompts Then, the positive text prompts are encoded into text features. .

[0069] Step S320: Extract the visual features of the first frame and the last frame, and stitch the visual features corresponding to the first frame and the last frame together to obtain joint visual features.

[0070] In this embodiment, the visual features of the first and last frames can be extracted using the CLIP (Contrastive Language-Image Pretraining) image encoder. and Then, the visual features corresponding to the first and last frames are concatenated in dimensional order to obtain the joint visual features. .

[0071] Step S330: Based on the temporal positions corresponding to the first frame, the last frame, and the optimal anchor frame, construct a binary mask, which is used to identify known frames and intermediate frames to be generated.

[0072] It should be noted that a binary mask is a temporal vector composed of 0s and 1s, the length of which is the same as the total number of frames in the target video to be generated. 1s are used to identify the temporal position of known frames (first frame, last frame, optimal anchor frame), and 0s are used to identify the temporal position of intermediate frames to be generated. The video generation model can use this binary mask to accurately distinguish known information from the region to be generated, thereby introducing targeted conditional constraints during the denoising process.

[0073] In the specific implementation process, the total number of frames of the target video is first determined, and then the normalized temporal positions of the first frame, the last frame, and the optimal anchor point frame are mapped to the corresponding frame numbers. Then, an all-zero vector is initialized, and the position of the corresponding frame number is assigned a value of 1 to obtain the binary mask.

[0074] As an example, constructing a binary mask , where 1 indicates the first frame, the last frame and the optimal anchor frame (which must be reserved), and 0 indicates the intermediate frame to be generated.

[0075] Step S340: The first frame, the last frame, the optimal anchor frame, and the zero-fill frame are concatenated in chronological order to form a guide frame sequence, and the guide frame sequence is subjected to latent spatial encoding to obtain latent features.

[0076] It should be noted that a zero-fill frame refers to an image frame with all pixel values ​​being 0. It is used to fill the positions of intermediate frames to be generated in the guide frame sequence, so as to ensure that the length of the guide frame sequence is consistent with the target video.

[0077] In this embodiment, a guide frame sequence is first constructed based on the sequence length of the target video to be generated and the temporal positions of the known frames. Specifically, the first frame, the optimal anchor frame, and the last frame are placed at their respective temporal indices in the sequence, and zero-padding frames are used to fill the other index positions, thus constructing the guide frame sequence. Next, the guide frame sequence is input into the encoder, which maps it from a high-dimensional pixel space to a low-dimensional, dense latent space to obtain the corresponding latent feature representation. This latent feature serves as the initial noise latent state for the subsequent diffusion generation process, and its structure includes the sequence's structural information and the initial encoding of the keyframes.

[0078] As an example, the first frame, last frame, optimal anchor frame, and zero-padding frame are concatenated along the timeline to form a guide frame sequence. Latent features are obtained by encoding with a VAE (Variational Auto-Encoder). .

[0079] Step S350: Input the text features, the joint visual features, the latent features, and the binary mask into the video generation model to generate the target video.

[0080] In this embodiment, text features, joint visual features, latent features, and a binary mask are input into the diffusion Transformer architecture of the video generation model. This video generation model, through its decoupled cross-attention mechanism, treats text features and joint visual features as independent, different forms of conditional information. Under the modulation of the binary mask, these are injected into different levels and time steps of the denoising process to guide the gradual purification of noisy latent features, thereby generating clean latent features that conform to semantic and visual constraints. Finally, the denoised clean latent representation is decoded by a VAE decoder to obtain a temporally coherent target video that is semantically consistent with the first and last frames.

[0081] It should be noted that the noisy latent feature refers to the noisy low-dimensional vector obtained by injecting Gaussian noise into the original latent features in the initial stage of the iterative denoising of the diffusion model. It is the direct processing object of the diffusion model denoising process. In this embodiment, the noisy latent feature is initialized by the latent features obtained by VAE encoding of the guide frame sequence, and is gradually purified into clean latent features in subsequent iterations.

[0082] Specifically, step S350 includes steps S3510 to S3540: Step S3510: Input the text features, the joint visual features, the latent features, and the binary mask into the diffusion Transformer architecture of the video generation model.

[0083] It should be noted that the Diffusion Transformer (DiT) architecture is a diffusion model based on Transformer, which has powerful temporal modeling and feature fusion capabilities and can generate high-quality videos through multi-round iterative denoising.

[0084] In this embodiment, before inputting text features, joint visual features, latent features, and binary masks into the diffusion Transformer architecture of the video generation model, it is necessary to perform dimensional adaptation on each feature so that the dimensions of the text features and joint visual features match the attention input dimensions of the DiT architecture; the binary mask needs to be converted into a temporal attention mask to guide the model to focus on the position of known frames.

[0085] Step S3520: In the diffusion Transformer architecture, the latent features are used as the denoising object, and based on the binary mask, the text features and the joint visual features are used as conditional information through a decoupled cross-attention mechanism to guide the multi-round iterative denoising process of the latent features.

[0086] It should be noted that the decoupled cross-attention mechanism refers to treating text features and joint visual features as independent conditional information, and having them interact with noisy latent features through different attention branches, thereby avoiding cross-modal feature interference and improving the accuracy of conditional constraints.

[0087] In this embodiment, textual features are injected through an independent cross-attention branch, primarily used to globally guide the semantic content and dynamic evolution trends of the video; joint visual features are injected through another independent cross-attention branch, primarily used to provide precise visual references at specific temporal positions. Binary masks are used to spatially selectively modulate the injection of conditional information (especially joint visual features): for known frame positions with a mask value of 1, the video generation model strongly constrains the reference to joint visual features when processing the corresponding latent features, ensuring that the generated frame is highly consistent with the given conditional frame, while also incorporating global guidance from textual features; for intermediate frame positions to be generated with a mask value of 0, the video generation model does not introduce strong visual constraints at that position, relying solely on semantic guidance from textual features, model prior knowledge, and temporal context information passed through self-attention to generate content. The existence of the mask essentially suppresses the direct effect of visual conditions at non-target positions, preserving reasonable creative space for the coherent generation of intermediate frames.

[0088] Step S3530: Perform the multi-round iterative denoising process to gradually eliminate noise in the latent features and generate clean latent features.

[0089] In this embodiment, during each iteration of denoising, the video generation model predicts and updates the noise based on the current noise latent features and conditional information, gradually reducing the noise level in the latent features, and finally obtaining clean latent features without noise.

[0090] Step S3540: Decode the clean latent features to obtain the target video.

[0091] In this embodiment, a VAE decoder is used to decode the clean latent features, restoring the low-dimensional latent features into a high-dimensional image sequence, thus obtaining the final target video.

[0092] As an example, text features Joint visual features Potential characteristics and the rearranged binary mask The DiT architecture of the input I2V model is used to inject conditional information through decoupled cross-attention. The iterative denoising process is as follows: ,in, The velocity field predicted by the model represents the update direction and magnitude of the latent noise features in the denoising iteration, and is used to guide the intensity and trajectory of each denoising step; for Latent noise characteristics at any given time.

[0093] This embodiment introduces a plug-in anchor frame bridging method, significantly optimizing the first and last frame video generation task without altering the core structure of the existing image-to-video (I2V) model. Its technical effects are mainly reflected in the following aspects: First, this method possesses strong adaptability and ease of deployment, and can be directly integrated as a pre-module into various mainstream video generation models such as Wan2.1 and HunyuanVideo without retraining, greatly reducing the engineering threshold for application. Second, by generating high-quality anchor candidate frames through frame order reversal and adaptively locating and bridging the most severe semantic breaks, only a small number (e.g., one) of key anchor frames are needed to effectively construct a stable semantic propagation channel from the first frame to the last frame. This fundamentally alleviates problems such as intermediate frame distortion, scene warping, and abrupt transitions caused by attention decay, significantly improving the temporal consistency of the generated video and enabling a smooth transition of perceptual similarity between frames. Furthermore, the anchor frame selection and guided generation process is computationally efficient, introducing only a small amount of preprocessing overhead without affecting the generation speed of the original model.

[0094] For example, to help understand the implementation process of the anchor-frame bridging-based first and last frame video generation method in this embodiment, please refer to... Figure 2 , Figure 2 A schematic diagram of the architecture of a video generation method for first and last frames based on anchor frame bridging is provided, specifically: This embodiment mainly includes two stages: adaptive anchor frame selection and anchor frame guided generation.

[0095] In the adaptive anchor frame selection phase, the core objective is to generate a high-quality set of anchor frame candidates, select the optimal anchor frame from them, and determine the corresponding insertion position.

[0096] Specifically, firstly, a frame order reversal operation is performed, and a reverse prompt is generated: the positions of the first and last input frames are swapped to obtain a reverse input frame pair; the semantics and dynamic transition relationship of the reverse input frame pair are analyzed using the Qwen model to generate the corresponding reverse text prompt.

[0097] Subsequently, reverse video generation is performed. The reverse input frame pairs are encoded into conditional inputs in the latent space by the encoder. This conditional input, along with the reverse text cue, is then fed into the video generation model. The model uses an iterative denoising process within the diffusion model to generate a complete reverse video sequence. The first half of this reverse video sequence is strongly semantically constrained by the original tail frame, resulting in higher visual quality and semantic consistency. It can be used as a candidate set of anchor frames.

[0098] Next, semantic break location and anchor frame selection are performed. By evaluating the semantic continuity of each frame in the reverse video sequence with its preceding and following adjacent frames, the position of the frame with the most severe semantic break is located. Then, based on the mirror symmetry relationship within the reverse video sequence, the mirror frame corresponding to the position of the frame with the most severe semantic break is selected as the optimal anchor frame. This optimal anchor frame has stronger semantic consistency with the first and last frames and can serve as an intermediate semantic bridge.

[0099] In the anchor frame generation stage, the core objective is to use the first and last frames and the anchor frame as joint conditions to guide the generation of a temporally coherent target video.

[0100] Specifically, the Qwen model is first used to analyze the original first and last frame pairs to generate positive text prompts, clarifying the semantic direction and action transitions of the target video to be generated.

[0101] Subsequently, conditional feature fusion is performed to extract the visual features of the first and last frames and concatenate them into joint visual features. At the same time, a binary mask is constructed to identify the positions of the known frames and the intermediate frames to be generated.

[0102] Next, joint guided generation is performed, concatenating the first and last frames, anchor frames, and zero-padding frames along the timeline to form a guided frame sequence. This sequence is encoded as latent features and then input into the DiT architecture of the video generation model along with joint visual features, text features corresponding to positive text cues, and a binary mask. A decoupled cross-attention mechanism injects multimodal conditional information into the denoising process, iteratively cleaning up noise in the latent features. Finally, the clean, denoised latent features are decoded by a decoder to obtain the temporally coherent and semantically consistent final target video.

[0103] For example, to help understand the implementation process of the anchor-frame bridging-based first and last frame video generation method in this embodiment, please refer to... Figure 3 , Figure 3 A schematic diagram illustrating the effect of a video generation method for first and last frames based on anchor frame bridging is provided. Specifically: Figure (a) visualizes the attention of intermediate frames, intuitively showing the distribution characteristics of attention weights in existing video generation models. It can be seen that there are significant attention connections only between adjacent frames, while the attention weight of the first and last frames to the intermediate frames is extremely low. This phenomenon directly confirms the technical pain point of "semantic decay" and explains why the semantic information of the first and last frames cannot be effectively transmitted to the middle area of ​​the video, thus leading to distortion of the intermediate frames. Figure (b) compares the LPIPS similarity performance between the ordinary method (FL2FV) and the first and last frame video generation method based on Anchor Frame Bridging (AFB) of this application (hereinafter referred to as the AFB method) between the first and last frames. The similarity curve of the ordinary method fluctuates drastically, especially with a significant trough in the middle of the video, reflecting a large semantic deviation between the intermediate frames and the first and last frames, resulting in serious semantic decay. In contrast, the curve of the AFB method of this application is more stable, maintaining a high similarity level throughout, indicating that the anchor frames effectively construct a semantic propagation bridge and greatly alleviate the semantic decay problem of the intermediate frames. Figures (c) and (d) visually demonstrate the difference in technical effects by comparing the complete generation process. Ordinary methods only use the first frame, the last frame, and text prompts as input, resulting in video distortion and abrupt transitions in the middle frames, such as distorted yoga postures. Furthermore, the video ends with abrupt transitions to fit the semantics of the last frame. In contrast, the AFB method of this application adaptively inserts anchor frames at key temporal positions, using them as semantic bridges to guide the generation. In the final generated video, the yoga posture transitions naturally and smoothly from "touching the toes" to "forward bend," with no obvious distortion in the middle frames and smooth transitions, effectively solving the semantic decay and abrupt transition problems existing in the prior art.

[0104] This application provides a first and last frame video generation device based on anchor frame bridging. The first and last frame video generation device based on anchor frame bridging includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the first and last frame video generation method based on anchor frame bridging in the above embodiment 1.

[0105] The following is for reference. Figure 4 The diagram illustrates a structural schematic of a first- and last-frame video generation device suitable for implementing the embodiments of this application based on anchor-frame bridging. The first- and last-frame video generation device based on anchor-frame bridging in the embodiments of this application may include various hardware and software components for implementing the first- and last-frame video generation method based on anchor-frame bridging. Figure 4 The illustrated first and last frame video generation device based on anchor frame bridging is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0106] like Figure 4 As shown, the anchor-frame-bridging-based first-and-last-frame video generation device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The random access memory 1004 also stores various programs and data required for the operation of the anchor-frame-bridging-based first-and-last-frame video generation device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the anchor-frame-bridging-based first-and-last-frame video generation device to wirelessly or wiredly communicate with other devices to exchange data. Although the figure shows anchor-frame-bridging-based first-and-last-frame video generation devices with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0107] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0108] The anchor-frame bridging-based first- and last-frame video generation device provided in this application, employing the anchor-frame bridging-based first- and last-frame video generation method described in the above embodiments, can solve the technical problem of intermediate frame content distortion caused by semantic attenuation in the first- and last-frame video generation method. Compared with the prior art, the beneficial effects of the anchor-frame bridging-based first- and last-frame video generation device provided in this application are the same as those of the anchor-frame bridging-based first- and last-frame video generation method described in the above embodiments, and other technical features in this anchor-frame bridging-based first- and last-frame video generation device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0109] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0110] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0111] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the first and last frame video generation method based on anchor frame bridging in the above embodiments.

[0112] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination thereof.

[0113] The aforementioned computer-readable storage medium may be included in a first- and last-frame video generation device based on anchor-frame bridging; or it may exist independently and not assembled into a first- and last-frame video generation device based on anchor-frame bridging.

[0114] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a video generation device based on anchor frame bridging, cause the video generation device to: perform a frame order reversal operation based on the input first and last frames to generate a reverse video sequence; perform quality evaluation on each frame in the reverse video sequence and determine the optimal anchor frame from the reverse video sequence based on the quality evaluation results; and input the first frame, the last frame, and the optimal anchor frame as joint conditions, combined with text prompts, into a video generation model to generate a target video.

[0115] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0116] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0117] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0118] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described anchor-frame bridging-based first and last frame video generation method. This solves the technical problem of intermediate frame content distortion caused by semantic attenuation in the first and last frame video generation method. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the anchor-frame bridging-based first and last frame video generation method provided in the above embodiments, and will not be repeated here.

[0119] This application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method for generating first and last frame videos based on anchor point frame bridging.

[0120] The computer program product provided in this application can solve the technical problem of distortion of intermediate frame content caused by semantic attenuation in the first and last frame video generation method. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of this application are the same as the beneficial effects of the first and last frame video generation method based on anchor frame bridging provided in the above embodiments, and will not be repeated here.

[0121] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

[0122] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0123] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0124] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for generating first and last frame videos based on anchor frame bridging, characterized in that, The method for generating first and last frame videos based on anchor frame bridging includes: Perform a frame order reversal operation based on the first and last input frames to generate a reverse video sequence; The quality of each frame in the reverse video sequence is evaluated, and the optimal anchor frame is determined from the reverse video sequence based on the quality evaluation results. The first frame, the last frame, and the optimal anchor frame are used as joint conditions, combined with text prompts, and input into the video generation model to generate the target video.

2. The method for generating first and last frame video based on anchor frame bridging as described in claim 1, characterized in that, The step of performing a frame order reversal operation based on the first and last input frames to generate a reverse video sequence includes: The timing positions of the first frame and the last frame are swapped to form a reverse input frame pair; Generate a reverse text prompt based on the reverse input frame pair; The reverse input frame pair and the reverse text prompt are input into the video generation model to generate the reverse video sequence.

3. The method for generating first and last frame video based on anchor frame bridging as described in claim 1, characterized in that, The step of performing quality assessment on each frame in the reverse video sequence and determining the optimal anchor frame from the reverse video sequence based on the quality assessment results includes: For each frame in the reverse video sequence, calculate the perceptual similarity value between the frame and its preceding and following adjacent frames to obtain the quality assessment value of each frame. Based on the quality assessment value, determine the frame with the lowest quality assessment value from the reverse video sequence; Based on the temporal position corresponding to the frame with the lowest quality assessment value, the target temporal position corresponding to the temporal position in the reverse video sequence is determined by mirror mapping. The frame at the target temporal position in the reverse video sequence is determined as the optimal anchor frame.

4. The method for generating first and last frame videos based on anchor frame bridging as described in claim 3, characterized in that, The step of calculating the perceptual similarity value between each frame and its preceding and following adjacent frames for each frame in the reverse video sequence to obtain the quality assessment value of each frame includes: By calculating the first perceptual similarity value between the frame and the previous adjacent frame, and the second perceptual similarity value between the frame and the next adjacent frame; Based on the first perceptual similarity value and the second perceptual similarity value, a quality assessment value for the frame is obtained, wherein the lower the quality assessment value, the worse the semantic continuity between the frame and its adjacent frames.

5. The method for generating first and last frame videos based on anchor frame bridging as described in claim 3, characterized in that, After the step of determining the frame at the target temporal position in the reverse video sequence as the optimal anchor frame, the method for generating the first and last frame videos based on anchor frame bridging further includes: When constructing the guide frame sequence, the target temporal position is determined as the target insertion position of the optimal anchor frame.

6. The method for generating first and last frame video based on anchor frame bridging as described in claim 1, characterized in that, The step of using the first frame, the last frame, and the optimal anchor frame as joint conditions, combined with text prompts, to input into the video generation model to generate the target video includes: A positive text prompt is generated based on the first frame and the last frame, and the positive text prompt is semantically encoded to obtain text features; Extract the visual features of the first frame and the last frame, and then concatenate the visual features corresponding to the first frame and the last frame to obtain joint visual features; Based on the temporal positions corresponding to the first frame, the last frame, and the optimal anchor frame, a binary mask is constructed. The binary mask is used to identify known frames and intermediate frames to be generated. The first frame, the last frame, the optimal anchor frame, and the zero-padding frame are concatenated in chronological order to form a guide frame sequence, and the guide frame sequence is subjected to latent spatial encoding to obtain latent features; The text features, the joint visual features, the latent features, and the binary mask are input into the video generation model to generate the target video.

7. The method for generating first and last frame video based on anchor frame bridging as described in claim 6, characterized in that, The step of inputting the text features, the joint visual features, the latent features, and the binary mask into the video generation model to generate the target video includes: The text features, the joint visual features, the latent features, and the binary mask are input into the diffusion Transformer architecture of the video generation model; In the diffusion Transformer architecture, the latent features are used as the denoising object, and based on the binary mask, the text features and the joint visual features are used as conditional information through a decoupled cross-attention mechanism to guide the multi-round iterative denoising process of the latent features. The multi-round iterative denoising process is executed to gradually eliminate noise in the latent features and generate clean latent features. The clean latent features are decoded to obtain the target video.

8. The method for generating first and last frame video based on anchor frame bridging as described in claim 6, characterized in that, The step of generating a positive text prompt based on the first frame and the last frame includes: Input the first frame and the last frame into the visual language model; The visual semantic content of the first frame and the last frame is analyzed by the visual language model, and the semantic evolution relationship from the first frame to the last frame is inferred based on the visual semantic content. Based on the semantic evolution relationship, positive text prompts are generated to represent the video content and the inter-frame action transition process.

9. A video generation device for first and last frames based on anchor frame bridging, characterized in that, The anchor-frame-bridging-based first and last frame video generation device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the anchor-frame-bridging-based first and last frame video generation method as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the video generation method for first and last frames based on anchor frame bridging as described in any one of claims 1 to 8.

Images