Spatio-temporal data stream video behavior recognition method based on deep learning

Abstract

The invention proposes a spatio-temporal data stream video behavior recognition method based on deep learning. Compared with a traditional deep learning method which utilizes a signal data stream, namely a single video stream, the spatio-temporal data stream video behavior recognition method based on deep learning utilizes spatio-temporal data streams, that is a spatial stream and a temporal stream, the temporal stream recognizes category of a target in a video from a static video frame, the spatial stream recognizes motion of the target from motion components in the video, and classification results of the temporal stream and the spatial stream are integrated to obtain a final behavior category. The spatio-temporal data stream video behavior recognition method based on deep learning recognizes the target and the motion separately, can reduce calculation burden of a neural network, and effectively increases accuracy.

Description

technical field [0001] The invention relates to various fields of computer vision, pattern recognition, and machine learning, especially deep learning in machine learning, and specifically relates to a deep learning-based spatio-temporal data stream video behavior recognition method. Background technique [0002] Most of the traditional video behavior recognition methods extract sparse spatiotemporal interest points, and then describe them as local spatiotemporal features, such as gradient histogram (HOG), optical flow histogram (HOF), and these features are represented by bag of features (BOF), Then use a classifier, such as SVM, to classify it. However, these features are all manually selected features. When the application scene changes, the originally selected features cannot play the best role, and selecting features again requires some work. Deep learning is an effective way to solve this problem. [0003] At present, some researchers are also trying to use deep lear...

AI Technical Summary

Problems solved by technology

[0003] At present, some researchers are also trying to use deep learning methods, such as C3D. To process video data, the existing network model must be modified. In addition, the complexity of video data leads to a huge amount of network calculation, which will generate high processing delays, making the system unsuitable for real-time video processing

However, if a single image data is processed, the accuracy and speed of the existing neural network can meet the requirements

[0004] The complexity of video data makes the combination of various features very large. If you want to represent a behavior, you must have both target and motion features. If you want to cover all behaviors, you need a large but redundant data set. This is in terms of workload and In terms of storage capacity, it is not feasible

Images