一种基于可预见多模态泛化知识表示的持续行为识别方法
By constructing a multimodal behavior recognition model and conducting incentive-based training, the problem of insufficient generalization of inertial sensing modalities was solved, and the continuous learning capability of the multimodal behavior recognition model was realized, thereby improving recognition accuracy and generalization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UNIV OF ELECTRONICS SCI & TECH OF CHINA
- Filing Date
- 2024-01-17
- Publication Date
- 2026-07-17
AI Technical Summary
With egocentric multimodal data input, the knowledge representation of inertial sensing modalities lacks generalization, leading to severe confusion between behaviors and catastrophic forgetting problems as the number of behaviors increases.
A continuous behavior recognition method based on predictable multimodal generalized knowledge representation is adopted. By collecting multimodal behavior activity data, a multimodal behavior recognition model is constructed and trained with incentives, including representation masking and adaptive gradient pruning. The encoder is frozen for incremental task training, and the representation mean and standard deviation vectors are stored to alleviate modality imbalance.
It significantly improves the generalization ability of multimodal behavior recognition models, reduces catastrophic forgetting of the network in continuous tasks, and improves recognition accuracy, outperforming existing methods on the UESTC-MMEA-CL dataset.
Smart Images
Figure CN117746509B_ABST