Video encoding and decoding method and system

Abstract

The invention discloses a video encoding and decoding method and system. Firstly, 2D features and processed 3D features are superposed according to a time sequence, and deep fusion of static and dynamic information is realized. Then, an attention mechanism is introduced to encode the fusion features at each moment t, a normalized weight is obtained through a softmax function, different weights are allocated to the fusion features, new fusion features are obtained, people-oriented features are learned, and thus final language description related to human behaviors is promoted. Finally, the new fusion features are inputted into a long short-term memory (LSTM) network, and decoding is carreid out as time goes on to obtain video description sentences. The video description obtained by the method is more logically fluent, and semantic coherence and clearness are realized.

Description

technical field [0001] The invention relates to the field of machine learning, in particular to a video encoding and decoding method and system. Background technique [0002] At present, although deep learning algorithms in artificial intelligence can perform video description functions, video information can be easily converted into language content. For example, before users watch a large amount of video information, it will save a lot of time and cost by forming an accurate text summary of the video information so that users can quickly understand the development of the event and its impact. In addition, extracting highlights from a two-hour movie and converting it into a text summary summarizing the movie will bring users a more perfect recommendation experience. However, this indiscriminate function of performing descriptions on video information cannot fully reflect the imagination, curiosity and intelligence of human beings to understand things, and these natures hav...

AI Technical Summary

Problems solved by technology

Traditional video understanding methods are difficult to fully consider the temporal relevance of human behavior in each frame of the video and the causality of events, and the extracted global temporal features contain a large number of redundant frame features, which will not only consume huge computing power, It also makes the model converge too slowly during the training phase, and cannot understand the development of things well from the human perspective of behavior as clues, so that the machine can understand the video more intelligently

Images