Dialogue emotion recognition method, electronic equipment, storage medium and product

Abstract

The invention provides a dialogue emotion recognition method, electronic equipment, a storage medium and a product. The method comprises the steps of obtaining a support set, and determining a dialogue relationship and an emotion category of each statement in the support set; extracting a statement representation of each statement in the support set, wherein the statement representation comprises statement information and context information of a dialogue to which the statement information belongs; calculating target class prototype features of each statement by adopting a gated propagation network based on statement representation and dialogue relations of each statement and adjacent statements in the support set; based on the class prototype feature of the statement corresponding to each emotion category, performing aggregation to obtain the class prototype feature of each emotion category; and based on the class prototype characteristics of each emotion class and the statement representation of the query statement, predicting the emotion class of the query statement. According to the method, the mutual dependency relationship between dialogues is fully utilized, statement emotion transmission is captured, the problem that prototype representation of separation modeling is insufficient is solved, and the accuracy of small sample dialogue emotion recognition is effectively improved.

Description

Technical Field

[0001] This application relates to natural language processing technology, and more particularly to a dialogue emotion recognition method, electronic device, storage medium, and product. Background Technology

[0002] With the rapid development of artificial intelligence technology, dialogue emotion recognition has become a research hotspot in academia and industry, with broad potential applications in fields such as dialogue empathy systems and smart healthcare. Since conventional dialogue emotion recognition methods rely on large amounts of high-quality labeled datasets to train models, and the refinement of emotion categories further increases the difficulty of labeling and computational costs, few-shot learning techniques have emerged as a key solution for dialogue emotion recognition because they can classify new samples with only a small number of samples, effectively solving the problem of dependence on labeled data.

[0003] Current few-shot sentiment recognition methods typically divide a dialogue into a support set and a query set, then use a recursive approach to extract contextual information. Next, they extract statements containing this contextual information and use a prototype network to calculate class-prototype features for each sentiment category. However, this separate modeling approach of context extraction and class-prototype feature calculation ignores the influence of intra-dialogue statements on the prototype. It struggles to capture the sentiment information conveyed by intra-dialogue statements during the calculation of class-prototype features, resulting in insufficient representation of the calculated class-prototype features. Summary of the Invention

[0004] This application provides a dialogue emotion recognition method, electronic device, storage medium, and product, aiming to improve the accuracy of dialogue emotion recognition.

[0005] The technical solution of this application embodiment is implemented as follows: In a first aspect, embodiments of this application provide a dialogue emotion recognition method, the method comprising: Obtain a support set and determine the dialogue relationship and sentiment category of each statement in the support set; Extract the statement representation of each statement in the support set, wherein the statement representation includes statement information and the context information of the statement in the dialogue; Based on the statement representation and dialogue relationship of each statement and adjacent statements in the support set, a gated propagation network is used to calculate the target class prototype features of each statement; Based on the class prototype features of the statements corresponding to each emotion category, the class prototype features of each emotion category are aggregated. Based on the target class prototype features of each sentiment category and the statement representation of the query statement, predict the sentiment category of the query statement.

[0006] In the above scheme, extracting the statement representation of each statement in the support set includes: Extract the statement features of each statement in the support set; Based on the statement features and dialogue relationships of each statement, a directed acyclic graph of each dialogue in the support set is constructed. The directed acyclic graph is input into a graph neural network, which outputs a statement representation of each statement in the dialogue. The directed acyclic graph includes nodes, directed edges, and edge relationships. The nodes represent the statement features of each statement, the directed edges represent the contextual dependencies between adjacent statements, and the edge relationships represent whether adjacent statements are spoken by the same person.

[0007] In the above scheme, the step of inputting the directed acyclic graph into a graph neural network and outputting a statement representation of each statement in the dialogue includes: Based on the statement characteristics of each statement, determine the initial layer statement representation for each statement; Feature fusion is performed on the nodes, directed edges, and edge relationships of the directed acyclic graph to obtain the aggregated information of each statement in each layer of the graph neural network; Based on the dual-gated loop unit, the aggregation information of the current layer and the statement representation of the previous layer are processed to obtain the statement information and context information of the current layer for each statement. Repeat the steps of processing the aggregated information of the current layer and the statement representation of the previous layer to obtain the statement representation of each statement in each layer of the graph neural network.

[0008] In the above scheme, the step of calculating the target class prototype features of each statement based on the statement representation and dialogue relationship of each statement and adjacent statements in the support set, using a gated propagation network, includes: Based on the statement representation of each statement in the support set, the class prototype features of each statement are updated using a dot product attention mechanism; Based on the statement representation and dialogue relationship of each statement and adjacent statements in the support set, a gated propagation network is used to update the class prototype features of each statement. Repeat the steps of updating the class prototype features of each statement using the dot product attention mechanism and updating the class prototype features of each statement using the gated propagation network until the set number of iterations is reached to obtain the target class prototype features of each statement.

[0009] In the above scheme, the step of updating the class prototype features of each statement using a dot product attention mechanism based on the statement representation of each statement in the support set includes: The dot product of the statement representation of each statement in the support set and the current class prototype feature is calculated to obtain the attention weight that represents the strength of association. The statement representation is weighted based on the attention weights to obtain the updated class prototype features for each statement.

[0010] In the above scheme, the step of updating the class prototype features of each statement based on the statement representation and dialogue relationship of each statement and adjacent statements in the support set, using a gated propagation network, includes: Based on the directed edges and edge relationships corresponding to the dialogue relationships, a gated propagation network is used to propagate the current class prototype features of each statement in the support set to adjacent statements to obtain the context class prototype features of each statement. Based on the statement representation of each statement, the current class prototype features, and the context class prototype features, an attention gating function is used to determine the context weight of each statement. Based on the context weights, the current class prototype features and context class prototype features of each statement are weighted and fused to obtain the updated class prototype features of each statement.

[0011] In the above scheme, predicting the sentiment category of the query statement based on the class prototype features of each sentiment category and the statement representation of the query statement includes: Calculate the similarity between the statement representation of the query and the class prototype features of each sentiment category; Based on the similarity scores, the sentiment category corresponding to the prototype feature with the highest similarity score is determined as the sentiment category of the query statement.

[0012] In a second aspect, embodiments of this application provide an electronic device, the electronic device comprising: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the steps of the method described in the first aspect.

[0013] Thirdly, embodiments of this application provide a storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0014] Fourthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0015] The technical solution of this application embodiment involves obtaining a support set and determining the dialogue relationship and sentiment category of each statement in the support set; extracting the statement representation of each statement in the support set, whereby the statement representation includes statement information and context information of the dialogue to which it belongs; based on the statement representations and dialogue relationships of each statement and adjacent statements in the support set, using a gated propagation network to calculate the target class prototype features of each statement; aggregating the class prototype features of each sentiment category based on the class prototype features of the statements corresponding to each sentiment category; and predicting the sentiment category of the query statement based on the class prototype features of each sentiment category and the statement representation of the query statement. Thus, this application embodiment fully utilizes the interdependencies between dialogues to capture the transmission of sentiment in statements, solves the problem of insufficient prototype representation in separate modeling, and effectively improves the accuracy of sentiment recognition in small-sample dialogues. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the first process of the dialogue emotion recognition method according to an embodiment of this application; Figure 2 This is a schematic diagram of the second process of the dialogue emotion recognition method according to an embodiment of this application; Figure 3 This is a flowchart of the dialogue emotion recognition method according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of the joint computation and extraction network in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of the joint computation and extraction network in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of the dialogue emotion recognition device according to an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0017] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] This application provides a method for dialogue emotion recognition, such as... Figure 1 As shown, the method includes: Step 101: Obtain the support set and determine the dialogue relationship and sentiment category of each statement in the support set.

[0020] Step 102: Extract the statement representation of each statement in the support set. The statement representation includes statement information and the context information of the statement in the dialogue.

[0021] Step 103: Based on the statement representation and dialogue relationship of each statement and adjacent statements in the support set, a gated propagation network is used to calculate the target class prototype features of each statement.

[0022] Step 104: Based on the target class prototype features of the statements corresponding to each emotion category, aggregate to obtain the class prototype features of each emotion category.

[0023] Step 105: Based on the class prototype features of each sentiment category and the statement representation of the query statement, predict the sentiment category of the query statement.

[0024] Here, the dialogue sentiment recognition method in this application embodiment is a few-sample dialogue sentiment recognition method.

[0025] Among them, the few-shot dialogue sentiment recognition method uses few-shot learning technology to recognize dialogue sentiment. Compared with conventional dialogue sentiment recognition methods that rely on large-scale, high-quality labeled datasets, the few-shot dialogue sentiment recognition method builds a learning mechanism that makes efficient use of data and relies on only a small number of labeled samples to predict the sentiment category of new dialogue sentences.

[0026] It is understood that the support set described above is a labeled sample of the embodiments of this application; the support set includes multiple dialogues, each dialogue includes multiple statements, each statement is pre-labeled with an emotion category, and there is a contextual relationship between adjacent statements in the same dialogue, that is, the dialogue relationship of statements includes the temporal and logical association of statements in the dialogue. For example, a dialogue in the support set is: Speaker A: "I failed the interview again today.", Speaker B: "Don't be sad, your written test score last time was obviously very good!", Speaker A: "But the interviewer said I was too nervous."; In the above dialogue, the statement "I failed the interview again today." is pre-labeled with the emotion category of disappointment, the statement "Don't be sad, your written test score last time was obviously very good!" is pre-labeled with the emotion category of comfort, and the statement "But the interviewer said I was too nervous." is pre-labeled with the emotion category of frustration; the dialogue relationship of the statement "Don't be sad, your written test score last time was obviously very good!" includes the preceding statement "I failed the interview again today" belonging to Speaker A and the following statement "But the interviewer said I was too nervous." belonging to Speaker A.

[0027] It should be noted that the above-mentioned emotion categories of "loss", "comfort", and "frustration" are only examples of emotion category classification. This application does not specifically limit the method and granularity of emotion category classification.

[0028] For example, the method further includes: acquiring a dataset; dividing the dataset into target class sets corresponding to each sentiment category; and dividing each target class set into a support set and a query set. Here, the dataset D includes multiple dialogues, and the statements in each dialogue are pre-labeled with sentiment categories; after acquiring the dataset D, the dataset D is first divided into multiple target class sets {D 1 D 2 D k D Nc}, where Nc is the number of sentiment categories, and D is the set of target classes k. k ={C1, C2, ..., C i C |Dk|}, |Dk| represents the number of dialogues in the target class set Dk; then N is selected from each target class set. S Each dialogue constitutes a support set S, and N is selected. Q The query set Q consists of N dialogues, where N is the number of dialogues. S ∩N Q =Ø.

[0029] The method for dividing dataset D into a set of target classes corresponding to each sentiment category includes: if a dialogue in dataset D contains at least one statement that belongs to target class k∈[1, Nc], then the dialogue is labeled as target class k.

[0030] Here, the support set is used to construct class-prototype features for each sentiment category, and the query set is used to verify the accuracy of the model's prediction of the sentiment category of sentences in new dialogues based on the class-prototype features constructed from the support set.

[0031] Here, the class prototype feature is the core feature corresponding to the sentiment category. The query statement determines the corresponding sentiment category by comparing it with the class prototype feature of each sentiment category.

[0032] It should be noted that prototype features are typically calculated based on contextual information extracted from the dialogue. Specifically, related techniques usually employ a recursive approach to extract contextual information from the dialogue, capturing the temporal sequence between sentences. Then, sentence features incorporating the aforementioned contextual information are extracted. Finally, a prototype network is used to aggregate sentence features of similar emotions, calculating prototype features for each emotion category. Taking the dialogue above as an example, by extracting contextual information, the temporal relationship between the sentences "I failed the interview again today," "Don't be sad, your written test score was clearly very good last time!" and "But the interviewer said I was too nervous." is obtained. Then, based on the sentence "I failed the interview again today," the sentence feature "failed the interview" corresponding to disappointment and its contextual annotation are extracted. Based on the sentence "Don't be sad, your written test score was clearly very good last time!", the sentence feature "Don't be sad" corresponding to comfort and its contextual annotation are extracted. Based on the sentence "But the interviewer said I was too nervous," the sentence feature "too nervous" corresponding to frustration and its contextual annotation are extracted. Finally, based on all sentence features of the same emotion category, the prototype features for the corresponding emotion category are aggregated and calculated.

[0033] However, the techniques used in related technologies that separate contextual information extraction from prototype feature calculation neglect the impact of phrasal information within the dialogue on prototype features. Taking the dialogue above as an example, only extracting phrasal features for each emotion category separately ignores the emotional impact of speaker B's "Don't be sad, you did very well on your last written test!" on speaker A's subsequent "But the interviewer said I was too nervous." In other words, the phrase "But the interviewer said I was too nervous" continues the negative emotion of "frustration" in response to speaker B's comfort and explanation of the failure. Clearly, the dialogue sentiment recognition methods in related technologies struggle to capture the emotional information conveyed by phrasal information within the dialogue group during prototype feature calculation, resulting in insufficient representation of the calculated prototype features.

[0034] It should be noted that, in order to solve the above problems, this application provides a dialogue sentiment recognition method based on joint extraction and calculation. Through a gated propagation network, the emotional influence of adjacent statements is introduced when calculating the class prototype features of a statement. The gated propagation network is a network structure with selective information transmission capabilities, which can dynamically adjust the information propagation intensity based on the relationship between nodes, achieving accurate transmission of effective information and suppression of influx of information. It is understood that the class prototype features calculated in this application reflect the emotional transmission logic within the dialogue. For example, when calculating the class prototype feature of the statement "But the interviewer said I was too nervous," which corresponds to frustration, the adjacent statement "Don't be sad, your last written test score was clearly very good!" is introduced to ensure the emotional association from comfort to frustration, rather than solely based on the statement feature "too nervous." This ensures that the final frustration class prototype feature reflects the association logic between the statement feature "too nervous" and the adjacent statement feature "Don't be sad."

[0035] Thus, the embodiments of this application fully utilize the interdependencies between dialogues, capture the emotional transmission of sentences, and calculate class prototype features that can reflect changes in the dialogue context, solving the problem of insufficient prototype representation in separate modeling and effectively improving the accuracy of small sample dialogue emotion recognition.

[0036] For example, extracting the statement representation of each statement in the support set includes: extracting the statement features of each statement in the support set; constructing a directed acyclic graph of each dialogue in the support set based on the statement features and dialogue relationships of each statement; inputting the directed acyclic graph into a graph neural network and outputting the statement representation of each statement in the dialogue.

[0037] Here, sentence features are the basic information carriers of each sentence in the dialogue. They include information such as the textual semantics and part-of-speech distribution of the sentences. For example, the sentence feature "Don't be sad" can be extracted from the sentence "Don't be sad, your last written test score was clearly very good!" In some embodiments, the sentence features can be extracted based on a feature extraction model, such as the RoBERTa (Robustly Optimized BERT Pretraining Approach) model. This application does not specifically limit the feature extraction model.

[0038] Here, a Directed Acyclic Graph (DAG) is a graph structure in which nodes are connected by directed edges and there are no cyclic paths. In this embodiment, a DAG is used to construct the temporal dependencies and relationships of statements in a dialogue.

[0039] The directed acyclic graph includes nodes, directed edges, and edge relations. Nodes represent the phrasing features of each statement, directed edges represent the contextual dependencies between adjacent statements, and edge relations represent whether adjacent statements are spoken by the same person.

[0040] It is understood that the nodes in the directed acyclic graph constructed in this application directly correspond to individual statements in the dialogue. Each node represents the statement features extracted from the corresponding statement, which is the basis for constituting statement information. Directed edges are used to connect nodes and have directionality. The direction of the directed edges is determined based on the temporal and logical order of the statements in the dialogue, representing the contextual dependency of the preceding statement on the following statement, and directly reflecting the linear relationship between adjacent statements. For example, the node corresponding to the statement "I failed the interview again today." is pointed to the node corresponding to the statement "Don't be sad, your written test score was obviously very good last time!" via a directed edge. The edge relationship is the association label of the directed edge, representing whether adjacent statements are from the same speaker, avoiding emotional ambiguity caused by the speaker's identity when calculating class prototype features. For example, if the statements corresponding to the nodes connected by the directed edge are from the same speaker, the edge relationship of the directed edge is set to 1; if the statements corresponding to the nodes connected by the directed edge are from different speakers, the edge relationship of the directed edge is set to 0.

[0041] In some embodiments, the directed edges of a directed acyclic graph are assigned edge weights.

[0042] Here, Graph Neural Network (GNN) is a neural network framework adapted to graph data structures, used to model nodes, edges, and relationships to achieve feature learning and information fusion. In this embodiment, a graph neural network is used to process the directed acyclic graph of each dialogue, extracting the statement information and context information of each statement in the dialogue, and then constructing a statement representation for each statement.

[0043] In some embodiments, the graph neural network is specifically a graph convolutional network (GCN). As a branch of graph neural networks, the graph convolutional network differs from conventional recursive models, which can only capture associations linearly in time sequence. It can use convolution operations to weightedly fuse the information of nodes and their neighboring nodes. In this embodiment, the graph convolutional network combines the directed edges and edge relationships in the input directed acyclic graph to assign differentiated weights to the contextual statements in the dialogue, fuses the statement information itself and the associated contextual information, and finally outputs a statement representation that conforms to the dialogue logic, which is adapted to the statement representation extraction scenario of this application.

[0044] For example, a directed acyclic graph (DAG) is input into a graph neural network (NN) to output a statement representation for each statement in the dialogue. This includes: determining an initial layer statement representation for each statement based on its statement features; performing feature fusion on the nodes, directed edges, and edge relationships of the DAG to obtain aggregated information for each statement at each layer of the NN; processing the aggregated information of the current layer and the statement representation of the previous layer using a dual-gated recurrent unit to obtain the statement information and context information of each statement at the current layer; and repeating the steps of processing the aggregated information of the current layer and the statement representation of the previous layer to obtain the statement representation for each statement at each layer of the NN.

[0045] Here, the embodiments of this application employ a multi-layer graph neural network, including node information units and context information units constructed based on gated recurrent units (GRUs). The node information units are used to obtain statement information, and the context information units are used to obtain statement context information. The gated recurrent unit is a neural network structure for processing sequence information. By controlling the retention and updating of information through a gating mechanism, it can accurately capture sequence associations.

[0046] Understandably, the multi-layer design based on graph neural networks progressively expands the coverage of associated information by fusing the node, directed edge, and edge relationship features of a directed acyclic graph layer by layer, providing directly usable multi-dimensional input for class prototype feature calculation. The two gated recurrent units take the aggregated information of the current layer and the statement representation of the previous layer as input, respectively. The gated recurrent unit of the node information unit takes the statement representation of the previous layer as initial input and the aggregated information of the current layer as subsequent input. When processing information, it prioritizes retaining the statement features of the statement and finally outputs the statement information. The gated recurrent unit of the context information unit takes the aggregated information of the current layer as initial input and the statement representation of the previous layer as subsequent input. When processing information, it prioritizes capturing the contextual associations of adjacent statements and finally outputs the contextual information of the statement.

[0047] In one application example of this application, after constructing a directed acyclic graph based on dialogues in a support set, the nodes u in the directed acyclic graph are... i Edge weight e ij And edge relationship r ij Inputting an L-layer graph neural network, extracting a sentence representation v containing contextual information. i Using node u in the dialogue i For example, first obtain the aggregation information of the node UI at layer l. Then, the statement information of layer l is obtained through two gated loop units. and context information and the statement information and context information Concatenation yields a statement representation containing contextual information. Among them, the aggregation information of layer l. Based on the following formula:

[0048] Statement information at layer l Based on the following formula:

[0049] Context information of layer l Based on the following formula:

[0050] L-level statement representation Based on the following formula:

[0051] For example, based on the statement representations and dialogue relationships of each statement and adjacent statements in the support set, a gated propagation network is used to calculate the target class prototype features of each statement, including: updating the class prototype features of each statement using a dot product attention mechanism based on the statement representations of each statement in the support set; updating the class prototype features of each statement using a gated propagation network based on the statement representations and dialogue relationships of each statement and adjacent statements in the support set; repeating the steps of updating the class prototype features of each statement using the dot product attention mechanism and updating the class prototype features of each statement using the gated propagation network until a set number of iterations is reached to obtain the target class prototype features of each statement.

[0052] Here, the target class prototype feature is the final class prototype feature of each statement, which is used to aggregate the class prototype features of the sentiment category corresponding to the statement. In order to ensure that the output target class prototype features fully take into account the sentiment transmission of adjacent statements, this embodiment of the application adopts multiple iterations to calculate the class prototype features of the statement. Each iteration is based on a joint extraction and calculation network, and uses a dot product attention mechanism combined with its own statement representation to update the class prototype features, as well as a prototype self-updating network, and uses a gated propagation mechanism combined with the statement representation of adjacent statements to update the class prototype features.

[0053] Accordingly, the steps of updating the class prototype features of each statement using the dot product attention mechanism and updating the class prototype features of each statement using the gated propagation network are repeated until the set number of iterations is reached to obtain the target class prototype features of each statement. This includes: after executing the step of updating the class prototype features of each statement using the gated propagation network, determining whether the current iteration number has reached the set number of iterations; if yes, determining the target class prototype features based on the current class prototype features of each statement; if no, incrementing the current iteration number by one, and sequentially executing the steps of updating the class prototype features of each statement using the dot product attention mechanism and updating the class prototype features of each statement using the gated propagation network.

[0054] Accordingly, the method also includes generating an initial class prototype feature for each statement based on the statement representation of each statement in the support set.

[0055] Understandably, after obtaining the statement representation of each statement in the support set, an initial class prototype feature is generated based on the statement representation of each statement. This initial class prototype feature serves as the initial input for updating the class prototype feature of each statement using a gated propagation network. In subsequent iterations, the output of updating the class prototype feature of each statement using the gated propagation network is used as the input for updating the class prototype feature of each statement using a dot product attention mechanism, and the output of updating the class prototype feature of each statement using the dot product attention mechanism is used as the input for updating the class prototype feature of each statement using the gated propagation network, forming a cyclic input chain of "gated propagation update → dot product attention update → gated propagation update" until the set number of iterations is reached.

[0056] Here, the methods for generating the initial class prototype features of each statement include, but are not limited to, vector projection methods or mean aggregation methods, and this application embodiment does not specifically limit them.

[0057] For example, based on the statement representation of each statement in the support set, the class prototype feature of each statement is updated using a dot product attention mechanism, including: performing a dot product calculation on the statement representation of each statement in the support set and the current class prototype feature to obtain an attention weight representing the association strength; and performing weighted processing on the statement representation based on the attention weight to obtain the updated class prototype feature of each statement.

[0058] It should be noted that the dot product attention mechanism is an attention mechanism that measures the strength of the association between a "query" and a "key". After obtaining the association strength calculation result through dot product calculation, the result is normalized to obtain the corresponding attention weight (also known as the propagation coefficient). Finally, the "value" is weighted based on the attention weight. It is easy to understand that in the embodiments of this application, the "query" in the dot product attention mechanism is the class prototype feature updated using a gated propagation network, the "key" is the statement representation, and the "value" is the statement representation. By iteratively updating the prototype feature using the dot product attention mechanism, information on the dynamic propagation and subtle changes of emotions in the dialogue can be accurately captured. Compared with calculating class prototype features only for the statement representation, the obtained class prototype features retain the original semantics of the statement while significantly enhancing the accuracy of emotional representation.

[0059] In one application example of this application, the statement representation containing context information at layer l is obtained. Then, the dot product attention mechanism is used to calculate the class prototype features related to the current statement. Attention weights between And utilize attention weights Weighted calculation of the current node u i Class prototype features after the t-th iteration update .

[0060] Among them, class prototype features Based on the following formula:

[0061] For example, based on the statement representations and dialogue relationships of each statement and adjacent statements in the support set, a gated propagation network is used to update the class prototype features of each statement. This includes: based on the directed edges and edge relationships corresponding to the dialogue relationships, the gated propagation network is used to propagate the current class prototype features of each statement in the support set to adjacent statements to obtain the context class prototype features of each statement; based on the statement representation, the current class prototype features, and the context class prototype features of each statement, an attention gating function is used to determine the context weight of each statement; based on the context weight, the current class prototype features and the context class prototype features of each statement are weighted and fused to obtain the updated class prototype features of each statement.

[0062] It should be noted that before using a gated propagation network for propagation processing, it is necessary to determine the corresponding set of adjacent statements based on the dialogue relationship of each statement, and then, based on the directed acyclic graph of the dialogue to which the statement belongs, query the edge weights and edge relationships of the directed edges between each adjacent statement in the set of adjacent statements.

[0063] It should be noted that a gated propagation network is introduced into the prototype self-updating network in this application embodiment. The class prototype features of the statements corresponding to adjacent nodes are used to update the class prototype features of the statements corresponding to the current node to enhance the feature representation capability. The gated propagation network calculates the class prototype features of adjacent statements and passes them to the context class prototype features of the current statement based on the context dependency relationship represented by the directed acyclic graph. Then, the weight ratio of the context class prototype features and the current class prototype features of the current statement in the updated class prototype features is determined by a predefined attention gating function, and then the updated class prototype features of the current node are calculated.

[0064] In one application example of this application, node u i The set of adjacent statements is Nu i The context class prototype feature updated after the t-th iteration of the node ui computed through the gated propagation network. Based on the following formula:

[0065] in, This is the gated propagation function for the gated propagation network.

[0066] In this example, an attention gating function is used. Calculated node u i Class prototype features after the t-th iteration update Based on the following formula:

[0067] It should be noted that the class prototype features of each statement updated using the dot product attention mechanism and gated propagation network are specifically the class prototype features of each statement in each layer of the graph neural network. After reaching the set number of iterations, the current class prototype features of each statement in each layer need to be concatenated to obtain the target class prototype features of each statement.

[0068] For example, based on the target class prototype features of the statements corresponding to each emotion category, the class prototype features of each emotion category are aggregated, including: aggregating the target class prototype features of all statements corresponding to the same emotion category in each dialogue, and the class prototype features of the dialogue corresponding to each emotion category; and aggregating the class prototype features of each emotion category based on the target class prototype features of the dialogue corresponding to each emotion category.

[0069] It is understood that the embodiments of this application first calculate the class prototype features of the emotion category within the dialogue, then calculate the class prototype features of the emotion category between dialogues, and finally obtain the class prototype features of each emotion category.

[0070] It should be noted that the statement representations output by the graph neural network are the statement representations at each layer of each statement. When calculating the class prototype features for each sentiment category, it is necessary to concatenate the statement representations at each layer of each statement to obtain the target statement representation for each statement. In some embodiments, the statement representations at each layer of each statement... The input is a multilayer perceptron (MLP), which is then used to compute the target statement representation v for each statement. i The target statement represents v. i Based on the following formula:

[0071] Here, the statement feature u i The corresponding sentiment category of the statement is yu i =k, then the statement feature u i The class prototype feature of the emotion category k corresponding to the dialogue The following formula was used to calculate:

[0072] in, The target statement representation is obtained by aggregating the statements corresponding to sentiment category k in the dialogue. It is obtained by aggregating the prototype features of the target class of the statement corresponding to the sentiment category k in the dialogue; These are the training parameters.

[0073] Here, the class prototype features of each dialogue are calculated. Subsequently, it will support weighted processing of all class prototype features, with the class prototype feature P of sentiment category k being weighted. k The following formula was used to calculate:

[0074] Where Nc*Ns is the number of dialogues in the support set S. To support the class prototype features of dialogues corresponding to sentiment category k in set S.

[0075] For example, predicting the sentiment category of a query statement based on the class prototype features of each sentiment category and the statement representation of the query statement includes: calculating the similarity between the statement representation of the query statement and the class prototype features of each sentiment category; and determining the sentiment category corresponding to the class prototype feature with the highest similarity as the sentiment category of the query statement based on each similarity.

[0076] Here, the statement representation of the query statement can be obtained based on the method described above for calculating the statement representation of each statement in the support set.

[0077] In some embodiments, Euclidean distance is used to calculate the similarity between the statement representation of the query and the class prototype feature of each sentiment category; correspondingly, based on each similarity, the sentiment category of the query is determined using an arg min function. Specifically, query statement u q sentiment category prediction results Based on the following formula:

[0078] in, This is the result of the Euclidean distance calculation.

[0079] Understandably, the sentiment category of a query can be predicted based on the class prototype features of each sentiment category calculated from the support set.

[0080] In some embodiments, the query statement is a statement in the query set; accordingly, after predicting the sentiment category of the query statement based on the class prototype features of each sentiment category and the statement representation of the query statement, the method further includes: training a dialogue sentiment recognition model based on the sentiment category labeling and sentiment category prediction results of each query statement in the query set, to obtain a trained dialogue sentiment recognition model.

[0081] Accordingly, the method also includes: inputting the dialogue to be identified into a dialogue sentiment recognition model to obtain the sentiment category of the dialogue to be identified.

[0082] It is understood that this application embodiment constructs a support set and a query set simultaneously, with each statement in the query set also labeled with a sentiment category. After calculating the class prototype features for each sentiment category based on the support set, a dialogue sentiment recognition model can be constructed based on the obtained class prototype features. The sentiment category label of each statement in the query set is used as the ground truth label, and the sentiment category prediction result of each statement in the query set is used as the model output. After training the model, a trained dialogue sentiment recognition model is obtained. The dialogue sentiment recognition model uses the standard cross-entropy function as the loss function, as shown in the following formula:

[0083] in, For query set statement v q The probability of correctly predicting the sentiment category.

[0084] In one application example of this application, a dialogue emotion recognition method is provided, such as... Figure 2 As shown, the method includes: Step 201: Construct a target class set and divide the target class set into a support set and a query set.

[0085] Step 202: Construct a directed acyclic graph neural network and extract contextual information for each dialogue in the support set based on the directed acyclic graph neural network.

[0086] Step 203: Construct a joint computation and extraction network, and update the class prototype features of each node using the dot product attention mechanism based on the joint computation and extraction network.

[0087] Step 204: Construct a prototype self-updating network and update the class prototype features of adjacent nodes based on the gating propagation mechanism in the prototype self-updating network.

[0088] Step 205: Perform sentiment recognition and classification on the dialogues in the query set based on the iteratively updated class prototype features.

[0089] Here, as Figure 2 The flowchart of the dialogue emotion recognition method shown is as follows: Figure 3 As shown, the structure of the joint computation and extraction network is as follows: Figure 4 As shown, the structure of the prototype self-updating network is as follows: Figure 5 As shown.

[0090] In order to implement the method of the embodiments of this application, the embodiments of this application also provide a dialogue emotion recognition device, which corresponds to the dialogue emotion recognition method described above. The steps in the embodiments of the dialogue emotion recognition method are also fully applicable to the embodiments of this device.

[0091] like Figure 6 As shown, the dialogue sentiment recognition device of this application embodiment includes: an acquisition module 601, an extraction module 602, a calculation module 603, an aggregation module 604, and a prediction module 605. The acquisition module 601 is used to acquire a support set and determine the dialogue relationship and sentiment category of each statement in the support set; the extraction module 602 is used to extract the statement representation of each statement in the support set, the statement representation including statement information and context information of the statement in the dialogue; the calculation module 603 is used to calculate the target class prototype feature of each statement using a gated propagation network based on the statement representation and dialogue relationship of each statement and adjacent statements in the support set; the aggregation module 604 is used to aggregate the class prototype features of each sentiment category based on the class prototype features of the statements corresponding to each sentiment category; and the prediction module 605 is used to predict the sentiment category of the query statement based on the class prototype features of each sentiment category and the statement representation of the query statement.

[0092] In some embodiments, the extraction module 602 is specifically used to: extract statement features of each statement in the support set; construct a directed acyclic graph (DAG) for each dialogue in the support set based on the statement features and dialogue relationships of each statement; input the DAG into a graph neural network and output the statement representation of each statement in the dialogue; wherein the DAG includes nodes, directed edges, and edge relationships, nodes represent the statement features of each statement, directed edges represent the contextual dependencies between adjacent statements, and edge relationships represent whether adjacent statements are from the same speaker.

[0093] In some embodiments, the extraction module 602 is specifically used to: determine the initial layer statement representation of each statement based on the statement features of each statement; perform feature fusion on the nodes, directed edges, and edge relationships of the directed acyclic graph to obtain the aggregate information of each statement in each layer of the graph neural network; process the aggregate information of the current layer and the statement representation of the previous layer based on the dual-gated recurrent unit to obtain the statement information and context information of each statement in the current layer; repeat the steps of processing the aggregate information of the current layer and the statement representation of the previous layer to obtain the statement representation of each statement in each layer of the graph neural network.

[0094] In some embodiments, the computation module 603 is specifically configured to: update the class prototype features of each statement using a dot product attention mechanism based on the statement representation of each statement in the support set; update the class prototype features of each statement using a gated propagation network based on the statement representations and dialogue relationships of each statement and adjacent statements in the support set; repeat the steps of updating the class prototype features of each statement using the dot product attention mechanism and updating the class prototype features of each statement using the gated propagation network until a set number of iterations is reached to obtain the target class prototype features of each statement.

[0095] In some embodiments, the calculation module 603 is specifically used to: perform a dot product calculation on the statement representation of each statement in the support set and the current class prototype feature to obtain an attention weight representing the association strength; and perform weighted processing on the statement representation based on the attention weight to obtain the updated class prototype feature of each statement.

[0096] In some embodiments, the calculation module 603 is specifically used to: based on the directed edges and edge relationships corresponding to the dialogue relationship, use a gated propagation network to propagate the current class prototype features of each statement in the support set to adjacent statements to obtain the context class prototype features of each statement; based on the statement representation, the current class prototype features, and the context class prototype features of each statement, use an attention gating function to determine the context weight of each statement; based on the context weight, perform weighted fusion of the current class prototype features and the context class prototype features of each statement to obtain the updated class prototype features of each statement.

[0097] In some embodiments, the prediction module 605 is specifically used to: calculate the similarity between the statement representation of the query statement and the class prototype feature of each sentiment category; and based on each similarity, determine the sentiment category corresponding to the class prototype feature with the highest similarity as the sentiment category of the query statement.

[0098] It should be noted that the dialogue emotion recognition device provided in the above embodiments is only illustrated by the division of the above-described program modules when performing dialogue emotion recognition. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the dialogue emotion recognition device and the dialogue emotion recognition method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0099] Based on the hardware implementation of the above program modules, and in order to implement the dialogue emotion recognition method of this application embodiment, this application embodiment also provides an electronic device, such as... Figure 7 As shown, electronic device 700 includes at least one processor 701, memory 702, user interface 703, and at least one network interface 704. The various components in electronic device 700 are coupled together via bus system 705. It can be understood that bus system 705 is used to implement communication between these components. In addition to a data bus, bus system 705 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 7 The general labeled all buses as Bus System 705.

[0100] The user interface 703 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0101] The memory 702 in this embodiment is used to store various types of data to support the operation of the electronic device 700. Examples of such data include any computer program used to operate on the electronic device 700.

[0102] The dialogue emotion recognition method disclosed in this application can be applied to or implemented by processor 701. Processor 701 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the dialogue emotion recognition method can be completed by the integrated logic circuitry of the hardware in processor 701 or by instructions in software form. The processor 701 can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 701 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can be located in a storage medium, specifically memory 702. Processor 701 reads information from memory 702 and, in conjunction with its hardware, completes the steps of the dialogue emotion recognition method provided in the embodiments of this application.

[0103] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned dialogue emotion recognition method.

[0104] It is understood that memory 702 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), EEPROM, ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Sync Link Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM). The memory 702 described in this application embodiment is intended to include, but is not limited to, these and any other suitable types of memory.

[0105] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 702 storing a computer program. This computer program can be executed by the processor 701 of the electronic device 700 to complete the steps described in the dialogue emotion recognition method of this application embodiment. The computer-readable storage medium can be a ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.

[0106] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a processor 701 of an electronic device 700 to perform the steps described in the method of this application embodiment.

[0107] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0108] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0109] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A dialogue emotion recognition method, characterized in that, The method includes: Obtain a support set and determine the dialogue relationship and sentiment category of each statement in the support set; Extract the statement representation of each statement in the support set, wherein the statement representation includes statement information and the context information of the statement in the dialogue; Based on the statement representation and dialogue relationship of each statement and adjacent statements in the support set, a gated propagation network is used to calculate the target class prototype features of each statement; Based on the target class prototype features of the statements corresponding to each sentiment category, the class prototype features of each sentiment category are aggregated. Based on the class prototype features of each sentiment category and the statement representation of the query statement, the sentiment category of the query statement is predicted.

2. The method according to claim 1, characterized in that, The extraction of the statement representation of each statement in the support set includes: Extract the statement features of each statement in the support set; Based on the statement features and dialogue relationships of each statement, a directed acyclic graph of each dialogue in the support set is constructed. The directed acyclic graph is input into a graph neural network, which outputs a statement representation of each statement in the dialogue. The directed acyclic graph includes nodes, directed edges, and edge relationships. The nodes represent the statement features of each statement, the directed edges represent the contextual dependencies between adjacent statements, and the edge relationships represent whether adjacent statements are spoken by the same person.

3. The method according to claim 2, characterized in that, The step of inputting the directed acyclic graph into a graph neural network and outputting a statement representation of each statement in the dialogue includes: Based on the statement characteristics of each statement, determine the initial layer statement representation for each statement; Feature fusion is performed on the nodes, directed edges, and edge relationships of the directed acyclic graph to obtain the aggregated information of each statement in each layer of the graph neural network; Based on the dual-gated loop unit, the aggregation information of the current layer and the statement representation of the previous layer are processed to obtain the statement information and context information of the current layer for each statement. Repeat the steps of processing the aggregated information of the current layer and the statement representation of the previous layer to obtain the statement representation of each statement in each layer of the graph neural network.

4. The method according to claim 2, characterized in that, Based on the statement representations and dialogue relationships of each statement and adjacent statements in the support set, a gated propagation network is used to calculate the target class prototype features of each statement, including: Based on the statement representation of each statement in the support set, the class prototype features of each statement are updated using a dot product attention mechanism; Based on the statement representation and dialogue relationship of each statement and adjacent statements in the support set, a gated propagation network is used to update the class prototype features of each statement. Repeat the steps of updating the class prototype features of each statement using the dot product attention mechanism and updating the class prototype features of each statement using the gated propagation network until the set number of iterations is reached to obtain the target class prototype features of each statement.

5. The method according to claim 4, characterized in that, The step of updating the class prototype features of each statement based on the statement representation of each statement in the support set using a dot product attention mechanism includes: The dot product of the statement representation of each statement in the support set and the current class prototype feature is calculated to obtain the attention weight that represents the strength of association. The statement representation is weighted based on the attention weights to obtain the updated class prototype features for each statement.

6. The method according to claim 4, characterized in that, The step of updating the class prototype features of each statement based on the statement representation and dialogue relationships of each statement and adjacent statements in the support set, using a gated propagation network, includes: Based on the directed edges and edge relationships corresponding to the dialogue relationships, a gated propagation network is used to propagate the current class prototype features of each statement in the support set to adjacent statements to obtain the context class prototype features of each statement. Based on the statement representation of each statement, the current class prototype features, and the context class prototype features, an attention gating function is used to determine the context weight of each statement. Based on the context weights, the current class prototype features and context class prototype features of each statement are weighted and fused to obtain the updated class prototype features of each statement.

7. The method according to claim 1, characterized in that, The prediction of the sentiment category of the query statement based on the class prototype features of each sentiment category and the statement representation of the query statement includes: Calculate the similarity between the statement representation of the query and the class prototype features of each sentiment category; Based on the similarity scores, the sentiment category corresponding to the prototype feature with the highest similarity score is determined as the sentiment category of the query statement.

8. An electronic device, characterized in that, The electronic device includes: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the steps of the method according to any one of claims 1 to 7.

9. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images