Emotion recognition method based on pulse neural network multi-modal information fusion

CN122432776APending Publication Date: 2026-07-21JILIN UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
JILIN UNIVERSITY
Filing Date
2026-04-28
Publication Date
2026-07-21

AI Technical Summary

Technical Problem

Existing technologies suffer from high power consumption in multimodal emotion recognition, and traditional spiking neural networks (SNNs) struggle to achieve adaptive cross-modal spatiotemporal alignment and redundancy suppression, failing to fully utilize the deep complementary relationships between modalities.

Method used

A cross-modal dynamic threshold adjustment and collaborative coordinate attention mechanism are introduced. Multimodal information is fused through a spiking neural network. Cross-modal control features are used to back-adjust the firing threshold of cross-attention neurons. A collaborative coordinate attention module is introduced for precise spatiotemporal position encoding.

Benefits of technology

It achieves low-power emotion recognition, improves the accuracy and robustness of multimodal emotion recognition, can run effectively on resource-constrained edge devices, adapts to individual differences and environmental noise, and improves the accuracy of cross-modal information fusion.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure FT_1
    Figure FT_1
  • Figure FT_2
    Figure FT_2
  • Figure FT_3
    Figure FT_3
Patent Text Reader

Abstract

The application discloses a kind of multi-modal information fusion based on pulse neural network emotion recognition method, belong to artificial intelligence, brain-like intelligence and biomedical signal processing technical field.The purpose of the present application is to introduce mutual mode dynamic threshold adjustment and collaborative coordinate attention mechanism, improve the accuracy, robustness and energy efficiency of multi-modal emotion recognition, realize the efficient alignment and adaptive deep fusion of heterogeneous physiological signals in pulse domain based on pulse neural network multi-modal information fusion emotion recognition method.The application respectively inputs each mode signal into the pulse encoder with the same structure and parameter-configurable to extract aligned features, inputs the mutual attention output of two directions after splicing into collaborative coordinate attention module to generate joint pulse gating map, and finally performs selective gating and fusion on enhanced features.The application effectively eliminates the inherent spatial noise and time-domain artifacts in physiological electrical signals, while ensuring the sparsity of pulse features, greatly improving the information density of the fused features.
Need to check novelty before this filing date? Find Prior Art