Action prediction method based on human skeleton sequence

Abstract

The invention discloses an action prediction method based on a human skeleton sequence, and the method makes full use of a human topological graph structure, employs a graph convolution network to extract rich and representative spatial-temporal features, and employs a knowledge distillation technology of a multi-teacher model, a low-observation-rate sequence to learn prior knowledge from a complete sequence and a high-observation-rate sequence, thereby achieving the prediction of the human skeleton sequence. Specifically, each sample instance adaptively selects a corresponding pre-training teacher model according to the observation rate, and knowledge transfer learning is carried out. And the multi-teacher model adaptively guides the generated student model, and iteratively trains and generates a plurality of student models by adopting a regenerative network technology. In the regenerative network training process, due to the fact that the model classification performance is weak under the low observation rate, the cross entropy loss function is weighted on the basis of the sample instance observation rate, and therefore the student model is optimized. By adopting a weighted fusion strategy, prediction results of a plurality of student models are fused as a final prediction classification result, and the method has the advantages of high prediction accuracy and the like.

Description

technical field [0001] The present invention relates to the technical field of action prediction of computer vision, in particular to an action prediction method based on human skeleton sequences. Background technique [0002] Human actions usually take place in 3D space, and 3D skeleton sequences provide more comprehensive information than RGB videos captured by 2D cameras. Today, with the rapid development of low-cost depth sensors such as Microsoft Kinect and Asus Xtion, accurate bone data can be directly generated in real time, and capturing human skeleton movements has become more accurate, simple and convenient. Therefore, in the field of computer vision, the research of action prediction based on human skeleton sequences is more and more attractive. The research on action prediction based on human skeleton sequence is cutting-edge and challenging, and it has very broad practical application prospects, such as human-computer interaction, automatic driving, intelligent...

AI Technical Summary

Problems solved by technology

However, such action prediction research methods only consider the full video pre-training as the teacher model, regardless of the observation rate of the current input sample of the student model, there are strong constraints on the characteristics or soft labels output by the teacher model of the full video pre-training Alignment, does not make full and reasonable use of the prior knowledge of observation rate, ignores the potential of a student model to learn from multiple teacher models, and fails to capture the different importance of different teacher models for each sample instance

Images