The invention discloses an anti-fact multi-mode dialogue emotion
causal reasoning method based on a double-
branch hypergraph. The method comprises the following steps: respectively extracting
sentence level feature vectors of three
modes of text, voice and vision from input multi-mode dialogue data; carrying out modeling on a high-order relationship in the modals and between the modals by utilizing a
hypergraph structure, and constructing a dialogue
hypergraph containing multi-
modal nodes and emotion nodes; introducing a hypergraph
attention network on the hypergraph, learning contribution weight of each
modal node to a target emotion node, and selecting a candidate reason node set; the candidate reason nodes are intervened, an anti-fact
branch is constructed, a fact situation and final node feature representation under the anti-fact situation are calculated, and a causal effect vector is obtained; and designing a joint optimization objective function, and carrying out joint training on
emotion recognition loss and causal consistency loss to realize synchronous prediction of emotion categories and emotion reasons. According to the method, a high-order
semantic relationship can be effectively modeled in a multi-
modal dialogue scene, and a key reason for emotion formation is reasoned.