Multi-stream fusion-based skeleton graph human body behavior identification method and system

Abstract

The invention discloses a skeleton graph human body behavior identification method and system based on multi-stream fusion, and the method comprises the steps: extracting four different data streams from video skeleton data, carrying out the network model training of the four different data streams, and obtaining four different training models; according to the human body behavior recognition method, human body skeleton point data is used as input, four different data streams are processed for model training, so that the network is more sensitive to channel information expressing different motions, and the human body behavior recognition efficiency is improved. Motion features are enhanced, and a behavior recognition model is trained through stacking of a multi-layer space-time diagram convolutional network structure; and finally, model results of four-stream training are fused, so that model outputs are mutually reinforced, and therefore, behavior action categories can be predicted more accurately.

Description

technical field [0001] The invention belongs to the technical field of behavior recognition, and in particular relates to a method and system for recognizing human behavior in a skeleton map based on multi-stream fusion. Background technique [0002] Humans have various behaviors and actions in daily life, which contain rich information. With the advent of the era of big data, massive pictures and videos have become the main carrier of information dissemination, and how to understand human behavior has become an important issue in the field of computer vision. Behavior recognition technology can be applied to human-computer interaction, intelligent monitoring, anomaly detection and other fields, and has strong application value and research significance. [0003] Compared with RGB data, the skeleton point data sequence has a clearer appearance representation, and the information expression of the human body structure is more intuitive. At the same time, the movement of huma...

AI Technical Summary

Problems solved by technology

If you only focus on adjacent joints, the distance between the two hands is very far in the skeletal point diagram, and it is difficult for the algorithm to capture long-distance dependencies, which will affect the accuracy of some behavior recognition

[0006] The second point is that the graph convolution structure of each layer in this method is basically similar, and the multi-layer stacking layer is deep, and the characteristics of each layer and channel are treated equally, lacking the flexibility and adaptability to focus on features.

With a fixed network, it is difficult to notice the different concerns of different actions, and it is difficult to obtain optimal modeling for multiple types of behaviors

[0007] The third point is that this method only uses the joint point information of the human body in the modeling and experiments. Although the space-time connection and time connection are considered in the construction of the space-time skeleton diagram, the motion information expressed by the joint points is still limited, which may there will be missing information

Images