Double-flow network behavior recognition method based on space-time significant behavior attention

Abstract

The invention discloses a double-flow network behavior recognition method based on space-time significant behavior attention, and belongs to the field of machine vision. The method adopts a network architecture based on a space-time double-flow network, and the network architecture is called as a space-time significant behavior attention network ST-SAMANet. For the problems of large memory consumption and excessive redundant information caused by directly inputting RGB frames and optical flow frames in a traditional double-flow network, a key frame mechanism is introduced to obtain the maximum difference between frames, and the memory consumption of a time network is remarkably reduced. Besides, in the network, a large number of feature redundancy and background disturbance exist on frames, and the performance of the network is greatly influenced. Mask R-CNN technology is introduced into a network, a human body and an object in each behavior category are highly concerned, and featureextraction is carried out on a salient region on each frame. Finally, the bidirectional LSTM and the C3D network are used to encode the space-time to obtain complete space-time information, and the robustness of the behavior recognition model is improved.

Description

technical field [0001] The invention belongs to the field of machine vision, in particular to a dual-stream network behavior recognition method based on spatio-temporal salient behavior attention. Background technique [0002] With the extensive research of machine vision in theory and practice, behavior recognition has gradually become an important branch of it. Due to the diversity of objective environments and the subjective complexity of human behavior, there are still many problems to be solved in human behavior recognition. At present, behavior recognition is mainly divided into two types of methods based on static pictures and based on videos. For a long time before video research became popular, most behavior recognition research was based on images, but the information provided by images was very limited, and it was not easy to capture effective information for recognition. In comparison, for behavior recognition on videos, researchers can extract sufficient 'acti...

AI Technical Summary

Problems solved by technology

However, the traditional two-stream network still faces the following problems: (1) How to make full use of the temporal semantic information of consecutive frames under the premise of effectively controlling the model complexity

(2) The network directly extracts the features on each frame, and it is difficult to guarantee the activity and availability of effective information in the network

Images