End-to-end behavior recognition method and system based on self-adaptive space-time attention mechanism

Abstract

The invention belongs to the field of behavior recognition, and provides an end-to-end behavior recognition method and system based on a self-adaptive space-time attention mechanism. In order to solvethe problem of poor behavior recognition precision, the behavior recognition method comprises the steps of receiving an image sequence of a video; processing the image sequence of the video by usinga behavior recognition model and outputting a behavior recognition result, wherein the behavior recognition model comprises a time attention module and a main convolutional neural network, and a spaceattention module is embedded in the main convolutional neural network; adaptively allocating a weight to each frame of image in the image sequence of the video according to the criticality of each frame of image by using a time attention module, and inputting an output result of the time attention module into a main convolutional neural network for behavior recognition; in the behavior recognition process of the main convolutional neural network, using the spatial attention module for focusing the behavior recognition of the main convolutional neural network on a motion related region, so asto quickly and accurately obtain a behavior recognition result.

Description

technical field [0001] The invention belongs to the field of behavior recognition, and in particular relates to an end-to-end behavior recognition method and system of an adaptive spatiotemporal attention mechanism. Background technique [0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art. [0003] In recent years, human behavior recognition has been widely used in video content analysis, video surveillance, human-computer interaction and other fields and has attracted the attention of academia and industry. However, human action recognition remains an intractable problem due to complex backgrounds, intra-class variation, low resolution, and high dimensionality. The key to accurately identifying various types of behaviors is to extract discriminative features and perform accurate modeling. The powerful image representation ability makes convolutional neural network widely ...

AI Technical Summary

Problems solved by technology

However, in the same frame, the action information contained in the action-independent background and the action-related motion region is not equal; in addition, due to the different degree of change in the action in different frames, consecutive frames have a high degree of redundancy, so Different frames in the video are of different importance for action recognition

Therefore, the above assumptions are inconsistent with the actual situation, which leads to poor recognition results of behaviors based on the above assumptions.

Images