A multi-modal fusion-based sentiment analysis method and system

By fusing facial expressions, voice, EEG signals, and visual pulse signals through a multi-layer deep learning network, the problem of insufficient fusion strategies in multimodal sentiment analysis is solved, and high-precision and robust sentiment analysis is achieved that adapts to complex environments and individual differences.

CN119272224BActive Publication Date: 2025-10-17无锡慧仁创欣智能科技有限公司
5 Cites 0 Cited by

Patent Information

Application Number
CN202411303810.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-19
Publication Date
2025-10-17
Estimated Expiration
2044-09-19

AI Technical Summary

Technical Problem

Existing multimodal sentiment analysis methods have deficiencies in feature fusion and context awareness, resulting in poor robustness of sentiment analysis in complex environments, low personalized recognition accuracy, and the inability to fully tap the potential value of multimodal information.

Method used

A multi-layer deep learning network is used to fuse facial expressions, voice, EEG signals and visual pulse signals. Through multi-head attention mechanism, adaptive weighting, bidirectional long short-term memory network, multi-scale causal convolutional network and graph neural network, combined with dynamic weight adjustment and context perception mechanism, dynamic fusion of multimodal data is achieved.

Benefits of technology

It improves the accuracy and robustness of sentiment analysis, enhances the system's adaptability in different environments and individual situations, and achieves global understanding and personalized recognition of emotional states.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The application discloses a kind of based on multi-modal fusion sentiment analysis method and system, comprising the following steps: S1, obtain multi-modal data and the initial feature representation of each mode;S2, the initial feature representation of each mode is input into multi-head attention mechanism and is preliminarily fused, generates primary fusion feature;S3, generate timing feature in conjunction with bidirectional long short-term memory network and multi-scale causal convolution network;S4, utilize multi-layer graph neural network to construct hierarchical dependency relationship network between modes, form global context perception feature representation;S5, dynamically generate and adjust the fusion weight of each mode;S6, utilize multi-layer fully connected network and multi-task learning framework joint classification and regression output, generate the multidimensional expression of emotional state;S7, by migration learning and individualized modeling technique, group sentiment model is migrated to specific sentiment analysis of individual user.The application utilizes multi-modal fusion and adaptive weight adjustment, realizes high-precision, multidimensional sentiment analysis.
Need to check novelty before this filing date? Find Prior Art