Video generation method and device, computer program product and electronic equipment

The process of event description text and video frame characteristics through timing position encoding and timing cross attention mechanisms is solved, and the problem of difficult event sequence and duration control in multi-event videos is achieved, and high-quality multi-event video generation is achieved.

CN120017929APending Publication Date: 2025-05-16NETEASE (HANGZHOU) NETWORK CO LTD
0 Cites 5 Cited by

Patent Information

Application Number
CN202510168668.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-02-14
Publication Date
2025-05-16

AI Technical Summary

Technical Problem

The prior art is difficult to accurately control the event sequence and duration of multi-event videos, resulting in the disordered sequence of events in the generated videos, the duration does not meet expectations, and the unnatural transition, which affects the generation quality of multi-event videos.

Method used

By receiving the event description text sequence, the reference frame feature vector of the video to be generated is determined, and the timing position encoding process is performed based on the start and end time information, the video frame and text features are fused, and the target video is generated using the timing cross attention mechanism.

Benefits of technology

Accurate time control of multi-event videos is achieved, ensuring the correct sequence of events and reasonable duration, and generating coherent and smooth videos, with natural transitions between events.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention relates to the technical field of video processing, and relates to a video generation method and device, a computer program product and electronic equipment. The video generation method comprises the steps that an event description text sequence is received, a reference frame feature vector of a video to be generated is determined according to the event description text sequence, and the event description text sequence comprises text description information of multiple events and corresponding starting and ending time information; performing time sequence position coding processing on the reference frame feature vector according to the starting and ending time information to obtain a video frame time sequence fusion feature, and performing time sequence position coding processing on the text description information to obtain a text time sequence fusion feature; based on a time sequence cross attention mechanism, fusing the video frame time sequence fusion feature and the text time sequence fusion feature to obtain a target fusion feature; and generating a dynamic image according to the target fusion feature corresponding to each reference frame feature vector to obtain a target video. The quality of the generated multi-event video can be improved.
Need to check novelty before this filing date? Find Prior Art