A multi-modal time-frequency coordinated high-precision human action recognition method and device

By employing a multimodal time-frequency collaborative method and utilizing a dynamic fusion strategy involving feature cross-modules and self-attention modules, the problems of intermodal confusion and inadequacy of time-frequency analysis in existing technologies are solved, achieving high-precision and robust human action recognition.

CN122196636APending Publication Date: 2026-06-12XIDIAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
XIDIAN UNIV
Filing Date
2026-03-18
Publication Date
2026-06-12

AI Technical Summary

Technical Problem

Existing human motion recognition technologies suffer from semantic confusion between modalities, low learning efficiency, inability of time-frequency analysis strategies to adapt to changes in motion, and lack of key time point focusing mechanisms in complex scenarios, resulting in insufficient recognition accuracy and robustness.

Method used

A multimodal time-frequency collaborative approach is adopted, which extracts temporal features of acceleration, angular velocity and quaternions through an independent long short-term memory network, performs explicit interactive fusion using a feature cross module, and combines a self-attention module and a gating network for dynamic weighted fusion, so as to achieve adaptive collaboration of time-frequency information and aggregation of key information.

Benefits of technology

It improves the accuracy of action recognition and the robustness of the model, reduces the risk of false triggering, and enhances the model's discriminative power and computational efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122196636A_ABST
    Figure CN122196636A_ABST
Patent Text Reader

Abstract

The application provides a multi-modal time-frequency cooperative high-precision human action recognition method and device. The method comprises: acquiring motion data of continuous human action; the motion data comprises acceleration, angular velocity and quaternion data; sequentially performing data preprocessing and channel normalization processing on the motion data to obtain a time domain tensor; performing frequency domain feature extraction on the time domain tensor according to the corresponding channel to obtain a frequency domain feature vector; inputting the time domain tensor and the frequency domain feature vector into a pre-trained action recognition model for recognition processing to obtain a gesture action recognition result; the pre-trained action recognition model is provided with a feature cross module, a self-attention module, a linear layer, a gate network and a plurality of independent long short-term memory networks (LSTM). The application enhances the model robustness while improving the action recognition accuracy through the modular design of the multi-modal time-frequency cooperation, and effectively reduces the risk of false triggering in actual application.
Need to check novelty before this filing date? Find Prior Art