Multi-modal sentiment classification method based on attention guidance two-way capsule network

Abstract

The invention discloses a multi-modal sentiment classification method based on an attention guidance two-way capsule network. Due to the fact that the capsule network has the trainable viewpoint invariant transformation characteristic, the effectiveness of the capsule network is proved in the aspect of analyzing heterogeneity of multi-modal learning at present. In the preprocessing stage, a multi-modal dynamic interaction enhancement module is provided, cross-modal homogeneity is explicitly enhanced on the feature level, and the model can effectively execute the multi-modal decoupling process in a more compact local public space. On the basis, a bidirectional capsule network (ABCN) based on attention guidance is provided, and a global multi-mode public message is explored through a new bidirectional dynamic routing mechanism. Secondly, guiding a multi-modal dynamic routing process by utilizing a global multi-modal context, and researching a global optimal common clue of each modal; this greatly improves learning efficiency and provides superior ability to erect a bridge between all modes.

Description

technical field [0001] The invention belongs to the field of multi-modal emotion recognition in the fields of natural language processing, vision, and voice, and relates to a multi-modal emotion classification method based on attention-guided bidirectional capsule network, specifically a capsule network-based learning and attention The attention-guided two-way capsule network technology of the mechanism is a method for two-way exploration, decoupling and fusion of multi-modal information to judge the emotional state of the test subject. Background technique [0002] Multimodal learning has increased interest in artificial intelligence systems, and linguistic, acoustic, and visual modalities are widely used to analyze relevant research tasks. Intuitively, a single modality can analyze task-related information from a specific perspective, while the integration of multiple modalities helps us effectively reason about the complex and comprehensive meaning of multimodal informati...

AI Technical Summary

Problems solved by technology

However, the above techniques only focus on the spatial relationship between the representation layer and the output layer through bottom-up attention

This indeed ignores the intrinsic contextual relationship between multiple modalities, fails to provide global guidance for each modality, and leads to a locally suboptimal decoupling process

Images