Behavior recognition method based on space-time attention enhancement feature fusion network

Abstract

The invention discloses a behavior recognition method based on a space-time attention enhancement feature fusion network, and belongs to the field of machine vision. According to the method, a networkarchitecture based on an appearance flow and motion flow double-flow network is adopted, and is called as a space-time attention enhancement feature fusion network. Aiming at a traditional double-flow network, simple feature or score fusion is adopted for different branches, an attention-enhanced multi-layer feature fusion flow is constructed to serve as a third branch to supplement a double-flowstructure. Meanwhile, aiming at the problem that the traditional deep network neglects modeling of the channel characteristics and cannot fully utilize the mutual relation between the channels, the channel attention modules of different levels are introduced to establish the mutual relation between the channels to enhance the expression capability of the channel characteristics. In addition, thetime sequence information plays an important role in segmentation fusion, and the representativeness of important time sequence features is enhanced by performing time sequence modeling on the frame sequence. Finally, the classification scores of different branches are subjected to weighted fusion.

Description

technical field [0001] The invention belongs to the field of machine vision, in particular to a behavior recognition method based on spatio-temporal attention enhancement feature fusion network. Background technique [0002] With the extensive research of machine vision in theory and practice, behavior recognition based on RGB video has gradually become a challenging branch. At present, behavior recognition for RGB video mainly uses a dual-stream network architecture, and the development trend is very good. In the two-stream architecture, the deep neural network obtains effective features by training respective deep convolutional networks on the RGB appearance stream and the optical flow motion stream respectively. However, the traditional two-stream network trained jointly with RGB appearance flow and optical flow motion flow still faces the following problems: (1) Simple fusion of features or scores obtained from different branches in the two-stream network cannot effecti...

AI Technical Summary

Problems solved by technology

However, the traditional two-stream network trained jointly with RGB appearance flow and optical flow motion flow still faces the following problems: (1) Simple fusion of features or scores obtained from different branches in the two-stream network cannot effectively utilize the characteristics of different branches, and at the same time in a single There will be feature overfitting in the branch

Images