End-to-end behavior recognition method and system based on self-adaptive space-time attention mechanism

A recognition method and attention technology, applied in the field of behavior recognition, can solve the problems of poor accuracy of recognition results and different importance of action recognition, and achieve the effect of fast and accurate acquisition, avoiding excessive calculation, and improving the speed of behavior recognition.

Active Publication Date: 2020-07-10
SHANDONG UNIV
View PDF2 Cites 20 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, in the same frame, the action information contained in the action-independent background and the action-related motion region is not equal; in addition, due to the different degree of change in the action in different frames, consecutive frames have a high

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • End-to-end behavior recognition method and system based on self-adaptive space-time attention mechanism
  • End-to-end behavior recognition method and system based on self-adaptive space-time attention mechanism
  • End-to-end behavior recognition method and system based on self-adaptive space-time attention mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0043] figure 1 A flowchart of an end-to-end behavior recognition method based on an adaptive spatiotemporal attention mechanism in this embodiment is given.

[0044] Combine below figure 1 The specific implementation process of the end-to-end behavior recognition method based on the adaptive spatio-temporal attention mechanism of this embodiment is described below.

[0045] Such as figure 1 As shown, this embodiment is based on the end-to-end behavior recognition method of the adaptive spatiotemporal attention mechanism, which includes:

[0046] Step S101: Receive a video image sequence.

[0047] In a specific implementation, videos in different monitoring scenarios are acquired, and an image sequence of consecutive frames is obtained.

[0048] For example: for home surveillance video to detect the behavior of the elderly and determine whether a fall has occurred;

[0049] For shopping mall surveillance video to detect and identify consumer shopping behavior and so on. ...

Embodiment 2

[0154] An end-to-end behavior recognition system based on an adaptive spatiotemporal attention mechanism, characterized in that it includes:

[0155] (1) image sequence receiving module, which is used to receive the image sequence of video;

[0156] (2) behavior recognition module, it is used to utilize behavior recognition model to process the image sequence of video and output behavior recognition result;

[0157] Among them, the behavior recognition model includes a temporal attention module and a main convolutional neural network, and a spatial attention module is embedded in the main convolutional neural network; the process of processing an image sequence by the behavior recognition model is:

[0158] Use the temporal attention module to adaptively distinguish the criticality of each frame of image, and assign corresponding weights to each frame of image, and input the output of the temporal attention module to the main convolutional neural network to identify behaviors;...

Embodiment 3

[0168] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps in the end-to-end behavior recognition method based on an adaptive spatiotemporal attention mechanism as described in Embodiment 1 are implemented.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the field of behavior recognition, and provides an end-to-end behavior recognition method and system based on a self-adaptive space-time attention mechanism. In order to solvethe problem of poor behavior recognition precision, the behavior recognition method comprises the steps of receiving an image sequence of a video; processing the image sequence of the video by usinga behavior recognition model and outputting a behavior recognition result, wherein the behavior recognition model comprises a time attention module and a main convolutional neural network, and a spaceattention module is embedded in the main convolutional neural network; adaptively allocating a weight to each frame of image in the image sequence of the video according to the criticality of each frame of image by using a time attention module, and inputting an output result of the time attention module into a main convolutional neural network for behavior recognition; in the behavior recognition process of the main convolutional neural network, using the spatial attention module for focusing the behavior recognition of the main convolutional neural network on a motion related region, so asto quickly and accurately obtain a behavior recognition result.

Description

technical field [0001] The invention belongs to the field of behavior recognition, and in particular relates to an end-to-end behavior recognition method and system of an adaptive spatiotemporal attention mechanism. Background technique [0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art. [0003] In recent years, human behavior recognition has been widely used in video content analysis, video surveillance, human-computer interaction and other fields and has attracted the attention of academia and industry. However, human action recognition remains an intractable problem due to complex backgrounds, intra-class variation, low resolution, and high dimensionality. The key to accurately identifying various types of behaviors is to extract discriminative features and perform accurate modeling. The powerful image representation ability makes convolutional neural network widely ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/00G06N3/04G06N3/08
CPCG06N3/08G06V40/23G06V20/41G06N3/045
Inventor 马昕刘少参宋锐荣学文田国会田新诚李贻斌
Owner SHANDONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap