Double-flow network behavior recognition method based on multi-level spatial-temporal feature fusion enhancement

Abstract

The invention discloses a double-flow network behavior recognition method based on multi-level spatial-temporal feature fusion enhancement. According to the method, a network architecture based on a space-time double-flow network is adopted, and the network architecture is called as a multi-level space-time feature fusion enhancement network. The method aims at solving the problems that the effectof shallow features is ignored and the complementary features of a double-flow network cannot be fully utilized due to the fact that category probability distribution of two flows is only fused at the last layer in a traditional double-flow network. According to the method, a multi-level spatial-temporal feature fusion module is provided, and multi-depth-level mixed features are captured throughthe spatial-temporal feature fusion module at different depth levels of double flows so as to make full use of a double-flow network. In addition, in the network, all features weakens the effect of those features that contribute greatly to classification are equally treated. According to the method, a grouping attention enhancing module is provided in the network, and the saliency of effective areas and channels on features is automatically enhanced. Finally, the robustness of the behavior recognition model is further improved by collecting the classification results of the double-flow networkand feature fusion.

Description

technical field [0001] The invention belongs to the field of machine vision, and in particular relates to a dual-stream network behavior recognition method based on fusion and enhancement of multi-level spatio-temporal features. Background technique [0002] Action recognition has become an active field in the computer vision community and is widely used in various fields such as video surveillance, violence detection, human-computer interaction, etc. Video action recognition is to mine the key features that can express the target action represented by the video. Compared with static images, it contains rich motion information. However, the diversity of action scenes still makes the extraction of effective features challenging. Therefore, the present invention takes video as the research object, and proposes a unique feature fusion method and attention method to effectively extract discriminative features for behavior recognition in view of the problems faced by the network ...

AI Technical Summary

Problems solved by technology

However, the traditional dual-stream network still faces the following problems: (1) How to effectively use the information captured by the two streams separately? (2) Treating each region and channel of features equally in the n

Images