Short video classification method

Abstract

The invention discloses a short video classification method. The method comprises the following steps: network training; network test which comprises the following steps: performing equal-interval frame interception on to-be-classified short videos, and intercepting a certain number of frames from each video; enabling intercepted frames to be subjected image processing, taking out a picture with acertain size and inputting into the model for prediction; fusing the intercepted frame and the prediction scores of different streams before Softmax; and obtaing a final result. The model is obtainedin an manner that an input video is divided into K segments, one segment is randomly sampled from the corresponding segment, the category scores of different segments are fused by using a segment consensus function to generate a segment consensus, a video-level prediction is obtained, and the prediction of all modes are fused to generate a final prediction result.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and in particular relates to a short video classification method. Background technique [0002] In the prior art, video classification training is based on Two-Stream and its derivative series. The basic principle is to train two convolutional networks to model video frame images (spatial) and dense optical flow (temporal). The two networks The structure is the same, both are two-dimensional convolution (2D ConvNets), such as figure 1 shown. The networks of the two streams respectively judge the category of the video to obtain a class score, and then perform fusion of the scores to obtain the final classification result. There are the following problems in video classification training using the above methods: (1) The calculation of optical flow in advance requires additional GPU computing time and storage space, which has become the bottleneck of the two-stream algorithm; (2) the tradi...

AI Technical Summary

Problems solved by technology

There are the following problems in video classification training using the above methods: (1) The calculation of optical flow in advance requires additional GPU computing time and storage space, which has become the bottleneck of the two-stream algorithm; (2) the traditional optical flow calculation method is completely independent of two -stream framework, not end-to-end training, the motion information in advance is not optimal; (3) cannot solve the long-term sequence problem

Using the above method for video classification training has the following problems: the amount of parameters is huge, the training time is long, it is easy to overfit, and the performance on various public data sets generally requires a large amount of calculation and cannot be applied in real time.

Images