Human skeleton behavior recognition method based on end-to-end spatio-temporal graph learning neural network

A neural network and human skeleton technology, applied in the field of computer vision, can solve problems such as not considering semantic information

Active Publication Date: 2020-11-24
ZHEJIANG UNIV
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Traditional methods often regard the human skeleton as a static invariant graph structure, without considering the semantic information associated between nodes under specific actions

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Human skeleton behavior recognition method based on end-to-end spatio-temporal graph learning neural network
  • Human skeleton behavior recognition method based on end-to-end spatio-temporal graph learning neural network
  • Human skeleton behavior recognition method based on end-to-end spatio-temporal graph learning neural network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0105] The implementation method of this embodiment is as described above, and specific steps are not described in detail, and the effect is shown below only for case data. The present invention is implemented on two data sets with truth value annotations, which are:

[0106] NTU-RGB+D data set: This data set contains 37920 training skeleton sequences and 18960 test skeleton sequences;

[0107] Kinetics data set: This data set extracts 2D skeleton sequences in the Kinetics video data set, including 240,000 training skeleton sequences and 20,000 test skeleton sequences;

[0108] The main process of skeleton-based behavior recognition is as follows:

[0109] 1) Use the results of clustering for each frame to obtain the spatial node relationship of the skeleton sequence;

[0110] 2) Use the trajectory of each node to obtain the time node relationship of the skeleton sequence;

[0111] 3) Use a 10-layer graph convolutional network, where the graph input of each layer of graph convolutional ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a behavior recognition method for learning a human skeleton of a neural network based on an end-to-end space-time diagram. The method is used for behavior recognition of a human 3D skeleton. The method specifically comprises the following steps: obtaining a human body 3D skeleton key point position data set for training, and defining an algorithm target; performing clustering expression on each frame based on the spatial position to obtain a spatial node relation; calculating a time track of each joint point, and performing relation measurement according to the time track to obtain a time node relation; establishing a joint learning framework of the space-time diagram learning and the diagram convolutional neural network; and estimating the behavior category of thecontinuous human body 3D skeleton by using the learning framework. The method is suitable for human body action analysis in a real video, and has a good effect and robustness for various complex conditions.

Description

Technical field [0001] The invention belongs to the field of computer vision, and particularly relates to a behavior recognition and detection method of a human body 3D skeleton of an end-to-end spatiotemporal graph learning neural network. Background technique [0002] The problem of behavior recognition based on human skeleton is defined as the following problem: predict the behavior category in a sequence of a set of key point positions of human skeleton containing multiple frames. Human skeleton joint points are often used as auxiliary information for some high-level visual tasks, such as video abnormal behavior detection and video action recognition. The key factors of human skeleton behavior recognition include the modeling of time series structure, the correspondence between joint points and the validity of calculation. Traditional methods often regard the human skeleton as a static and invariant graph structure without considering the semantic information associated with...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06K9/00G06K9/62G06N3/04
Inventor 李玺李斌张仲非
Owner ZHEJIANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap