Method, system and product for dialogue emotion recognition under modal missing condition

Abstract

The invention discloses a dialogue emotion recognition method, system and product under a mode missing condition, and the method comprises the steps: firstly constructing a context situation graph containing a missing mode, and defining an edge according to a time sequence type; each statement is defined as a node; defining a'context influence weight 'index, including global and local context influence weights; then constructing a time sequence relation graph convolutional network, integrating time sequence representation through the relation graph convolutional network, capturing a relation between modals of the statement, and reconstructing a multi-modal feature; filtering edges between corresponding nodes of different speakers, deleting the edges of which the time sequence category is from present to present, and adjusting the attention weight at the same time, so as to enhance the features from the same speaker; and finally, using emotion perception contrast learning to predict an emotion category. The method shows strong robustness and effectiveness in the aspect of processing missing modalities, can well adapt to modal missing of different degrees, and provides better emotion recognition ability under a higher missing rate.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence, and relates to a method, system and product for emotion recognition in conversations, and particularly to a method, system and product for dialogue emotion recognition under the condition of modality loss. Background Art

[0002] Multi-modal emotion recognition refers to the technology of analyzing data of multiple modalities (such as text, audio, video, etc.) to identify and predict human emotions, and this technology has extensive and key applications in fields such as mental health.

[0003] Original emotion recognition only focuses on the emotion expressed by a single sentence without considering the context in which the sentence is located, while dialogue emotion recognition needs to perform emotion recognition in a broader context. Dialogue emotion recognition not only needs to analyze multiple modalities of the current sentence, but also needs to consider all the discourse states and the relationship between speakers in the entire conversation, so as to avoid the ambiguity caused by analyzing only a single sentence.

[0004] Currently, more and more intelligent Internet of Things (IoT) and online service applications need to perform emotion recognition on users based on communication interactions. However, when performing emotion recognition, various interference factors (such as sensor failures, bandwidth limitations, etc.) will cause the loss of some modalities. The problem of modality loss not only hinders obtaining complementary information from different modalities, but also masks key context information, thereby directly affecting the effect of dialogue emotion recognition.

[0005] To solve the problem of modality loss in dialogue emotion recognition, existing methods mainly include two aspects: joint learning of shared multi-modal representations through a multi-modal fusion mechanism, and recovery of missing features through the dependency relationship between modalities. However, the former often ignores the emotional interconnection between sentences in the dialogue context, and the latter does not consider the dynamic nature of the dialogue and is difficult to understand modality conflicts or sudden emotion changes.

[0006] Therefore, fully modeling the context information in the dialogue and effectively learning the emotion change paradigm are the key issues for dialogue emotion recognition under the condition of modality loss. Summary of the Invention

[0007] To solve the above technical problems, the present invention proposes a method, system and product for dialogue emotion recognition under the condition of modality loss, which can introduce more relevant context information for sentence nodes in a scenario of severe modality loss and effectively learn the emotion changes in the dialogue, thereby effectively improving the effect of dialogue emotion recognition under the condition of modality loss.

[0008] The technical solution adopted by the method of the present invention is: a method for identifying conversation emotions under modality loss conditions, comprising the following steps: Step 1: Construct a context graph containing missing modalities and define edges based on time series types; Each sentence is defined as a node; a "context influence weight" indicator is defined, including global context influence weight and local context influence weight. This indicator considers the speaker's activity frequency and modality availability to dynamically adjust the optimal adjacent window size for each sentence in the context graph; Step 2: Build a temporal relational graph convolutional network to integrate temporal representations and capture the inter-modal relationships of sentences through the relational graph convolutional network, thereby reconstructing multimodal features. Step 3: Filter the edges between nodes corresponding to different speakers, remove the edges with the time series category of "now to now", and adjust the attention weights to enhance the features from the same speaker; Step 4: Use emotion-aware contrastive learning to predict emotion categories.

[0009] As a preference, in step 1, first calculate each node The global context influence weight , then calculate the initial window size The local context influence weight ; By and Compare to dynamically resize the window ;if Less than , it means that the context information in the current window is insufficient, increase the window size, and obtain the new window size ; otherwise, the window size remains unchanged.

[0010] As a preference, in step 1, for a window size of Node , whose local context influences the weight ; The 3 in the denominator indicates the three modal data considered, including audio, text, and vision; Indicates the window size The total number of statements in ; and Respectively represent the nodes and Associated speaker information, if and from the same speaker, then the indicator function is equal to 1, otherwise 0; Used to count the number of available modalities for a statement; Compute nodes Global context influence weight , that is, the window size should include the entire conversation, that is , represents the number of all statements in the conversation.

[0011] Preferably, in step 1, each edge in the context situation graph is defined as three temporal sequence types: "past to future", "future to past", and "present to present", so as to establish a temporal interaction pattern in the situation graph.

[0012] Preferably, in step 2, first perform temporal multi-modal feature fusion in the relational graph convolutional network, and the formula is as follows: ; wherein, represents the multi-modal feature of the th layer node , is the edge relation set, is the relation under which the node set connected to the node is; is the learnable weight matrix associated with the relation , is the learnable weight matrix applied to ; the function represents the ReLU activation function.

[0013] Then, a multi-modal linear decoder is adopted to reconstruct the complete feature set from the multi-modal fusion features.

[0014] Preferably, in step 3, the adjustment of the attention weight is to use the Graph Transformer mechanism to optimize the context information. By using multi-head attention, the attention weight is adjusted based on the edges with the same speaker, so as to ensure a more comprehensive understanding of emotions; the attention weight is calculated as follows:

[0015] wherein, and respectively represent the features corresponding to the nodes and , and represent the linearly learnable weights, represents the vector and dimensions; is the scaling factor, which is used to control the variance of the dot product result and prevent gradient disappearance or explosion; Based on the above mechanism, the features of each statement node are enhanced to ensure that statements from the same speaker have a greater impact on each other's features. The formula is as follows: ; where, and are linearly learnable weights, represents the neighborhood of feature .

[0016] Preferably, in step 4, the entire conversation is represented as to obtain the emotion category corresponding to each statement . Wherein, represents the total number of statements, represents the th statement, represents the speaker information associated with statement , represents the predicted value of the emotion of the th statement by the model.

[0017] Preferably, the temporal relationship graph convolutional network described in step 2 is a trained network; the loss function used in the training process is: ; where, represents the cross-entropy loss, is the mean squared error loss, is the contrastive loss, is the emotion entropy loss; coefficients and respectively control the relative weights of the contrastive loss and the emotion change loss in the formula;

[0018] where, represents the total number of statements, is the reconstructed multi-modal feature, is the complete multi-modal feature; ; where, for the statement nodes in the context situation graph, an anchor node is selected, and the adjacent nodes with the same emotion as the anchor node are regarded as positive samples , and the adjacent nodes with different emotions are regarded as negative samples ; for each sample triple ( , , ), the contrastive loss function based on the triple loss is used for training optimization; Represents the total number of triples, represents the Euclidean distance, a variable represents the margin value, used to ensure that positive samples are closer to the anchor point than negative samples; ; ; where, and respectively represent the th dialogue entropy of the predicted and true emotion distributions, represents the th probability of an emotion category, represents the total number of emotion categories.

[0019] The technical solution adopted by the system of the present invention is: A dialogue emotion recognition system under the condition of missing modality, including: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the dialogue emotion recognition method under the condition of missing modality.

[0020] The technical solution adopted by the product of the present invention is: A dialogue emotion recognition product under the condition of missing modality, including computer program instructions, which, when run on a computer, cause the computer to execute the dialogue emotion recognition method under the condition of missing modality.

[0021] Compared with the prior art, the beneficial effects of the present invention include: (1) By adjusting and comparing the global and local "context influence weights" of nodes, the present invention dynamically adjusts the window size, thereby selecting an appropriate context window for constructing a situation graph, complementing missing modality data while avoiding redundancy, and ensuring effective integration of valuable context information; (2) By defining the edges in the context situation graph as different temporal types, the present invention introduces temporal features into the graph, thereby establishing a time interaction pattern between nodes and strengthening the context consistency of the situation graph; (3) The present invention uses a relational graph convolutional network for temporal multi-modal feature fusion and reconstruction. In the above-mentioned temporal multi-modal fusion, independent Transformer encoders are used for different modalities, thereby extracting the features of each modality respectively and avoiding the influence of missing modality; (4) The present invention proposes a speaker emotion perception constraint mechanism, and enhances the features from the same speaker through the Graph Transformer mechanism to explicitly encode the emotional consistency of the speaker, thereby preserving the inherent emotional structure in the conversation. The speaker emotion perception constraint mechanism can model the emotional dynamics more coherently, enabling the model to better distinguish subtle emotional changes; (5) Through emotion perception contrast learning, the present invention uses nodes with the same emotion as the anchor node as positive samples and nodes with different emotions as negative samples, making the features corresponding to the same emotion closer in the feature space; (6) The present invention defines an emotion entropy loss function to capture the emotional changes in the conversation, which is calculated for all statements in the current conversation and quantifies the absolute difference between the predicted entropy and the true emotion distribution, thereby enabling the model to make more accurate predictions about the emotional changes in the conversation; (7) The present invention combines four different loss functions, assigns different weights, and forms a unified objective loss function to achieve end-to-end joint optimization of the model. The end-to-end training optimization of the model guided by the combination of the four loss functions not only improves the accuracy of emotion classification, but also enhances the stability and discriminability of the modal representation. At the same time, the model is capable of modeling the emotional evolution trend in the conversation, thus still having good robustness and generalization ability in complex scenarios such as modal absence and speaker change. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The following uses examples and specific implementation manners to further illustrate the technical solutions of the present invention. In addition, some drawings are also used in the process of explaining the technical solutions. For those skilled in the art, without creative efforts, other drawings and the intent of the present invention can also be obtained based on these drawings.

[0023] Figure 1 : Flowchart of the method according to the embodiment of the present invention; Figure 2 : Principle framework diagram of the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0024] In order to better elaborate the technical solutions and processes of the present invention, the following further describes the present invention in detail with reference to the drawings and examples. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0025] Generally, the task of dialogue emotion recognition under modal absence focuses on analyzing the emotional state of each statement through all available dialogue information. Let the entire dialogue input be , the statement order be , the total number of statements be , and each statement be , the speaker information associated with the statement is . Based on these inputs, the emotional category of each statement is predicted in sequence as .

[0026] The present invention aims to construct a multi-modal emotion recognition model, which can still accurately and efficiently convert the input dialogue (including available modalities and speaker information) into a series of predicted emotional categories in the case of modality absence.

[0027] Please refer to Figure 1 and Figure 2 . A method for dialogue emotion recognition under the condition of modality absence provided by this embodiment includes the following steps: Step 1: Construct a context situation graph containing missing modalities, and define edges according to the time series type; Define each statement as a node; define a "context influence weight" index, including a global context influence weight and a local context influence weight. This index simultaneously considers the activity frequency of the speaker and modality availability to dynamically adjust the optimal adjacent window size of each statement in the context situation graph; In one implementation, in order to more effectively capture context information, this embodiment proposes a new graph construction strategy, representing each statement as a node. Define a "context influence weight" index , which simultaneously considers the activity frequency of the speaker and modality availability to dynamically adjust the optimal adjacent window size of each statement in the graph . For the node statement with a window size of , its local "context influence weight" calculation formula is as follows: ; ; Among them, the 3 in the denominator represents three types of modality data considered, including audio, text, and vision; represents the window size ; and respectively represent the speaker information associated with nodes and . If and are from the same speaker, the indicator function is equal to 1, otherwise it is 0; is used to calculate the number of available modalities of the statement; Furthermore, to calculate the global "context influence weight" of , the window size should include the entire dialogue, that is, let , represents the number of all statements in the conversation.

[0028] In one embodiment, first calculate the global context influence weight of each node , and then calculate the local context influence weight of the initial window size ; dynamically adjust the window size by comparing with ; if is less than , it means that the context information in the current window is insufficient, increase the window size to obtain a new window size ; otherwise, the window size remains unchanged. is less than , it means that the context information in the current window is insufficient, increase the window size to obtain a new window size ; otherwise, the window size remains unchanged.

[0029] In one embodiment, each edge in the context scenario graph is defined as three temporal types: "past to future", "future to past", and "present to present", thereby establishing a temporal interaction pattern in the scenario graph.

[0030] Step 2: Construct a temporal relationship graph convolutional network, integrate temporal representations through the relationship graph convolutional network, capture the inter-modal relationships of statements, and reconstruct multi-modal features; In one embodiment, a temporal multi-modal fusion method is proposed, which integrates temporal representations through a relationship graph convolutional network and captures the inter-modal relationships of statements to solve the problem of missing modalities.

[0031] The above temporal multi-modal fusion method uses independent Transformer encoders for different modalities, thereby extracting the features of each modality separately and avoiding the influence of missing modalities.

[0032] First, perform temporal multi-modal feature fusion in the relationship graph convolutional network, and the formula is as follows: ; where represents the multi-modal feature of the node at the th layer, is the edge relationship set, is the node set connected to the node under the relationship . Here, is the learnable weight matrix associated with the relationship , and is the learnable weight matrix applied to . The function represents the ReLU activation function. represents the ReLU activation function.

[0033] Then, a multi-modal linear decoder is adopted to reconstruct the complete feature set from the multi-modal fusion features.

[0034] Step 3: By filtering the edges between the nodes corresponding to different speakers and deleting the edges with the temporal category of "now to now", and at the same time adjusting the attention weights to enhance the features from the same speaker; In one implementation, considering that the emotional state of the speaker is coherent and consistent in the conversation, this embodiment proposes a speaker emotion perception constraint mechanism to explicitly encode the emotional consistency of the speaker, thereby retaining the inherent emotional structure in the conversation. The speaker emotion perception constraint mechanism can model the emotional dynamics more coherently, enabling the model to better distinguish subtle emotional changes.

[0035] This embodiment adopts a speaker emotion filtering method to filter the edges between the nodes corresponding to different speakers and delete the edges with the temporal category of "now to now", thereby ensuring that the utterances from the same speaker are regarded as more relevant. The sequence of utterance features of the entire conversation is generated by the relational graph convolutional network, and its formula is expressed as follows: ; where, represents the feature corresponding to the utterance ;

[0036] In one implementation, since emotional expression is affected by local and global conversation contexts, relying solely on direct graph connections may lead to incomplete representations. To mitigate this limitation, this embodiment adopts the GraphTransformer mechanism to optimize the context information. This mechanism uses multi-head attention to adjust the attention weights based on the edges consistent with the speaker, thereby ensuring a more comprehensive understanding of emotions. The calculation formula of the attention weights is as follows: ; where, and respectively represent the features corresponding to the nodes and ; and represent the linearly learnable weights, represents the vector and ; As a scaling factor, it can control the variance of the dot product result and prevent the gradient from vanishing or exploding.

[0037] Furthermore, based on the above mechanism, the features of each utterance node are enhanced to ensure that the utterances from the same speaker have a greater impact on each other's features, and its formula is as follows: ; Among them, and are linearly learnable weights, represents the feature of the neighborhood.

[0038] Step 4: Use emotion-aware contrastive learning to predict the emotion category.

[0039] In one implementation, for the entire dialogue input , it will finally be mapped to a series of corresponding emotion categories . Among them, represents the total number of statements as, represents the th statement, represents the speaker information associated with the statement , represents the predicted value of the emotion of the th statement by the model.

[0040] In one implementation, the temporal relational graph convolutional network is a trained network; the loss function used during training is: ; Among them, represents the cross-entropy loss, is the mean squared error loss, is the contrastive loss, is the emotion entropy loss; the coefficients and respectively control the relative weights of the contrastive loss and the emotion change loss in the formula (in the following experiments, the coefficient is set to 0.3, and the coefficient is set to 0.5);

[0041] Among them, represents the total number of statements, is the reconstructed multi-modal feature, is the complete multi-modal feature; ; Among them, for the statement nodes in the context situation graph, an anchor node is selected, and the adjacent nodes with the same emotion as the anchor node are regarded as positive samples , and the adjacent nodes with different emotions are regarded as negative samples ; for each sample triple ( , , ), and a contrastive loss function based on triplet loss is adopted for training optimization; represents the total number of triplets, represents the Euclidean distance, and the variable represents the margin value, which is used to ensure that the positive sample is closer to the anchor point than the negative sample (in the following experiment, the variable is set to 0.5); In this embodiment, the emotion entropy loss is formulated according to L1 normalization (i.e., mean absolute error), and the formula is as follows: ; ; where and respectively represent the th dialogue entropy of the predicted and true emotion distributions, represents the th probability of the emotion category, represents the total number of emotion categories.

[0042] The following further elaborates on the present invention through specific experiments.

[0043] In the experiment, the datasets used include the IEMOCAP dataset and the MELD dataset. Among them, the IEMOCAP dataset is a multi-modal emotion recognition benchmark, consisting of 151 dialogue videos containing two speakers. It provides two emotion label schemes: the four-class label IEMOCAPFour (happy, sad, neutral, angry) and the six-class label IEMOCAPSix (happy, sad, neutral, angry, excited, frustrated). The IEMOCAPSix label contains 7433 utterances, and the IEMOCAPFour label contains 5531 utterances. The MELD dataset includes multi-speaker conversations from the TV series "Friends", including three modalities: audio, text, and vision, containing 1432 conversations and 13708 utterances. Among them, each conversation involves 1 to 9 speakers, and a seven-class emotion label (angry, disgusted, sad, happy, neutral, surprised, and fearful) is provided for each utterance.

[0044] In the experiment, the method was evaluated based on different modal data missing rates, and the missing data was randomly assigned. The model was trained using the Adam optimizer with a learning rate of 0.00001 and a dropout rate of 0.5.

[0045] In the experiments, the weighted F1 (wF1) score and accuracy (Acc.) were used as evaluation metrics for the proposed method. The proposed method was compared with existing methods for different modal data missing rates ranging from 0 to 0.6.

[0046] On the IEMOCAPFour dataset, our proposed method outperforms all existing methods at all missingness rates. At a missingness rate of 0.1, our method achieves a wF1 score and accuracy that are 3.00% and 3.05% higher than the second-best method, GCNet, respectively. This advantage persists at higher missingness rates, with leads ranging from 2.83% to 3.86% in wF1 score and 2.70% to 3.61% in accuracy.

[0047] On the IEMOCAPSix dataset, our method outperforms all existing methods across all missingness ratios. When the missingness ratio is above 0.1, our method leads the best existing methods by 2.44% to 3.89% in wF1 scores and by 2.37% to 4.34% in accuracy.

[0048] On the MELD dataset, the proposed method demonstrated superior recognition performance under most missingness rates, especially at higher missingness rates. At a missingness rate of 0.6, its wF1 score and accuracy improved by 1.29% and 1.57%, respectively, compared to the best existing method.

[0049] The proposed method can capture more contextual information in scenarios where modality is severely lacking, effectively modeling sentence nodes in contextual situation graphs. It can also effectively learn emotional changes in conversations and optimize the contextual relevance of graph nodes.

[0050] The method proposed in the present invention shows strong robustness and effectiveness in handling missing modalities. It can adapt well to different degrees of modality missingness and provide better emotion recognition ability under higher missing rates.

[0051] It should be understood that the embodiments described above are only some of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention may be arbitrarily combined with each other to form a feasible technical solution. Such combination is not restricted by the order of steps and / or structural composition mode, but must be based on the ability of ordinary technicians in this field to implement it. When the combination of technical solutions is mutually inconsistent or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0052] It should be understood that the above description of the preferred embodiment is relatively detailed, and it should not be considered as a limitation on the protection scope of the present invention patent. Under the inspiration of the present invention, those of ordinary skill in the art can also make substitutions or modifications without departing from the protection scope defined by the claims of the present invention, and all of them fall within the protection scope of the present invention. The scope of protection claimed by the present invention shall be subject to the appended claims.

Claims

1. A method for dialogue emotion recognition under the condition of modal absence, characterized in that It includes the following steps: Step 1: Construct a context situation graph containing missing modalities and define edges according to temporal types; Define each statement as a node; Define a "context influence weight" metric, including global context influence weight and local context influence weight, which simultaneously considers the speaker's activity frequency and modality availability to dynamically adjust the optimal adjacency window size of each statement in the context situation graph; Step 2: Construct a temporal relational graph convolutional network, integrate temporal representations through the relational graph convolutional network, capture the inter-modal relationships of statements, and reconstruct multi-modal features; Step 3: By filtering the edges between nodes corresponding to different speakers and deleting the edges with the temporal category of "now to now", while adjusting the attention weights to enhance the features from the same speaker; Step 4: Use emotion-aware contrastive learning to predict emotion categories.

2. The method for dialogue emotion recognition under the condition of modal absence according to claim 1, characterized in that: In step 1, first calculate each node The global context influence weight , then calculate the initial window size The local context influence weight ; By and Compare to dynamically resize the window ;if Less than , it means that the context information in the current window is insufficient, increase the window size, and obtain the new window size ; otherwise, the window size remains unchanged.

3. The method for identifying dialogue emotions under the condition of modal absence according to claim 1, wherein: In step 1, for a node with a window size of , its local context influence weight ; among them, the 3 in the denominator represents three types of modal data considered, including audio, text, and vision; ; represents the total number of statements in the window size ; and respectively represent the speaker information associated with nodes and . If and are from the same speaker, the indicator function is equal to 1, otherwise it is 0; is used to calculate the available number of modalities of a statement. Computing node Global context influence weight , that is, the window size should contain the entire conversation, that is , represents the number of all statements in the conversation.

4. The method for recognizing dialogue emotion under the condition of modal absence according to any one of claims 1, wherein: In Step 1, each edge in the context situation graph is defined as three temporal types: "past to future", "future to past", and "now to now", so as to establish the temporal interaction pattern in the situation graph.

5. The method for dialogue emotion recognition under the condition of modal absence according to claim 1, characterized in that: In Step 2, first perform temporal multi-modal feature fusion in the relational graph convolutional network, and the formula is as follows: ； Among them, represents the multimodal features of the layer nodes, is the edge relation set, is the relation under which the nodes are connected to the node set; is the learnable weight matrix associated with the relation and is the learnable weight matrix applied to ; the function represents the ReLU activation function;​​​ Then adopt a multi-modal linear decoder to reconstruct the complete feature set from the multi-modal fusion features.

6. The method for dialogue emotion recognition under the condition of modal absence according to claim 1, characterized in that: In step 3, the adjustment of the attention weight is to use the Graph Transformer mechanism to optimize the context information. By leveraging multi-head attention, the attention weight is adjusted based on the edges with the same speaker, thereby ensuring a more comprehensive understanding of emotions; the attention weight is calculated as follows: Among them, and respectively represent the features corresponding to nodes and ; and represent linearly learnable weights; represents the dimension of vectors and ; is a scaling factor used to control the variance of the dot product result and prevent gradient vanishing or explosion; Based on the above mechanism, enhance the features of each statement node to ensure that statements from the same speaker have a greater impact on each other's features, and the formula is as follows: ； Among them, and are linearly learnable weights, represents the feature neighborhood.

7. The method for recognizing dialogue emotions under the condition of modal absence according to claim 1, characterized in that: In step 4, the entire conversation is represented as , and the emotion category corresponding to each statement is obtained ; among them, represents the total number of statements, represents the th statement, represents the speaker information associated with the statement , represents the predicted value of the emotion of the th statement by the model.

8. The method for recognizing dialogue emotions under the condition of modal absence according to any one of claims 1-7, characterized in that: The temporal relational graph convolutional network described in Step 2 is a trained network; The loss function used in the training process is: ； Among them, represents the cross-entropy loss, is the mean squared error loss, is the contrastive loss, is the emotion entropy loss; the coefficients and control the relative weights of the contrastive loss and the emotion change loss in the formula respectively; Among them, represents the total number of sentences, is the reconstructed multi-modal feature, is the complete multi-modal feature; ； Among them, for the statement nodes in the context scenario graph, an anchor node is selected , and the adjacent nodes with the same emotion as the anchor node are regarded as positive samples , and the adjacent nodes with different emotions are regarded as negative samples ; for each sample triple ( , , ), a contrastive loss function based on triple loss is adopted for training optimization; represents the total number of triples, represents the Euclidean distance, and the variable represents the margin value, which is used to ensure that positive samples are closer to the anchor than negative samples; ； ； wherein, and respectively represent the -th dialogue entropy of the predicted and true emotion distributions, represents the probability of the -th emotion category, represents the total number of emotion categories.

9. A dialogue emotion recognition system under the condition of modal absence, characterized in that, It includes: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the method for dialogue emotion recognition under the condition of modality absence as described in any one of claims 1 to 8.

10. A product for dialogue emotion recognition under the condition of modal absence, including computer program instructions, characterized in that: When the computer program instructions run on a computer, cause the computer to execute the method for dialogue emotion recognition under the condition of modality absence as described in any one of claims 1 to 8.

Images