Human skeleton behavior recognition method based on multi-modal enhancement converter and hypergraph structure

The multi-modal enhanced transformer and hypergraph structure method addresses the limitations of static topological structures in skeletal data recognition, enhancing the accuracy of dual-person interaction behavior recognition by refining spatial and temporal features.

CN120316581APending Publication Date: 2025-07-15CHONGQING UNIV
0 Cites 1 Cited by

Patent Information

Application Number
CN202510446363.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-04-10
Publication Date
2025-07-15

AI Technical Summary

Technical Problem

The existing human behavior recognition technology is not robust enough when dealing with light changes and viewing angle occlusion, making it difficult to adaptively adjust the intensity of joint associations, and is unable to effectively capture the global association across limbs and time in complex movements. In addition, the multimodal data fusion method leads to misalignment of features, neglecting key features, and it is difficult to identify advanced motion semantics in two-person interactive scenarios.

Method used

Using a method based on multimodal augmentation converter and hypergraph structure, the progressive multimodal augmentation module, the double-person hypergraph self-attention module and the global context fusion module are used to optimize feature extraction and fusion, build a learnable hypergraph structure, break through the limitations of paired joint modeling, and capture the cross-limb collaborative motion mode.

Benefits of technology

It improves the accuracy of human behavior recognition, especially in two-person interactive scenarios, which can accurately capture high-order motion patterns, improving the performance and recognition effect of the model.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention provides a human skeleton behavior recognition method based on a multi-modal enhancement converter and a hypergraph structure. The method comprises the following steps: S1, acquiring original skeleton data representing human behavior recognition; preprocessing the skeleton data to obtain four skeleton modal data in total; s2, carrying out single-mode feature extraction on the four types of skeleton modal data, and then carrying out multi-mode feature fusion to form unified skeleton features; s3, performing feature refining and three-dimensional dynamic feature reinforcement on the skeleton features of the unified mode in time and space; and S4, the refined skeleton features are subjected to a global average pooling layer and a full connection layer to obtain a behavior identification prediction result, and the result is a discrete action classification label. According to the method, a lightweight multi-modal converter network and a double hypergraph topological structure are constructed to enhance the multi-modal skeleton data fusion performance and capture the cooperative motion relationship between joint groups, so that the recognition accuracy and recognition efficiency of double interaction behaviors are improved.
Need to check novelty before this filing date? Find Prior Art