A video processing method, apparatus, device and medium

Abstract

This application discloses a video processing method, apparatus, device, and medium, relating to the field of artificial intelligence technology. It utilizes a target model to perform forward propagation, hidden state extraction, and hidden state discarding on an initial sequence to obtain a first hidden state and a target embedding vector. A draft model is then used to perform forward propagation on the first hidden state and the target embedding vector to obtain a second hidden state and a key-value pair cache. A spanning tree is then processed to obtain a token sequence and a tree mask. When the target model triggers a preset condition, the target model is used to calculate the token sequence and tree mask to obtain a third hidden state and a first token probability distribution. The draft model is then used to perform forward propagation on the third hidden state to obtain a second token probability distribution. Based on the comparison result of the first and second token probability distributions, a video processing result is generated. This addresses the problems of inaccurate fusion of video visual information and text semantic information, as well as the time-consuming nature of long sequence inference, thereby improving video processing efficiency.

Images