A Multimodal Dialogue Emotion Recognition Approach Aligned with Graph-Guided Interaction and Prototype-Driven Approach

Abstract

This invention discloses a multimodal dialogue sentiment recognition method and system based on graph-guided interaction and prototype-driven alignment. It employs a pre-trained large-scale multimodal dialogue sentiment recognition model for dialogue sentiment identification. During the training of the large-scale model, a temporal semantic graph-guided mechanism is used to explicitly model the temporal relationships and sentiment evolution paths between utterances by constructing a lightweight dynamic semantic dependency graph, providing structured semantic guidance for the local features of each modality. Based on a sentiment prototype-driven cross-modal semantic calibration mechanism, a set of cross-modal prototype representations is learned for each sentiment category. During the inference phase, the learned sentiment prototypes are used for modality alignment and confidence calibration. This invention achieves explicit modeling of the temporal relationships and sentiment evolution paths between utterances and establishes stable semantic anchors to guide accurate modality alignment and confidence calibration.

Images