A speech classification method, apparatus and electronic device

Abstract

This application provides a discourse classification method, apparatus, and electronic device. The method includes: acquiring a dialogue sequence, the dialogue sequence including multiple historical discourses and a target discourse; using a first model to extract features from each of the multiple historical discourses and the target discourse to obtain a first feature vector for the multiple discourses; using a second model to extract features from a first graph network to obtain a second feature vector, the first graph network being constructed based on the first feature vectors of the multiple discourses; using a third model to extract features from the second feature vector to obtain a third feature vector; using a fourth model to extract features from a second graph network to obtain a fourth feature vector and a fifth feature vector, the second graph network being constructed based on the third feature vector; and predicting the fourth and fifth feature vectors respectively to obtain a classification result for the target discourse. This application can improve the accuracy of classification.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a discourse classification method, device, and electronic device. Background Technology

[0002] In multi-turn dialogues, dialogue action recognition and dialogue sentiment classification are two related tasks. Dialogue action recognition describes the function or purpose of statements and reflects the speaker's explicit intention, while sentiment reflects the implicit intention of the character. However, multi-task learning methods only consider the information contained in the dialogue history and the current utterance, and model dialogue action recognition and dialogue sentiment recognition tasks separately, resulting in lower recognition accuracy. Summary of the Invention

[0003] This application provides a discourse classification method, apparatus, and electronic device to solve the problem of low accuracy in dialogue classification and recognition.

[0004] Firstly, embodiments of this application provide a discourse classification method, including:

[0005] Obtain a dialogue sequence, which includes multiple historical utterances and a target utterance;

[0006] The first model is used to extract features from each of the multiple historical discourses and the target discourse to obtain the first feature vector of the multiple discourses.

[0007] The second model is used to extract features from the first graph network to obtain a second feature vector. The first graph network is constructed based on the first feature vector of the multiple utterances.

[0008] The third model is used to extract features from the second feature vector to obtain the third feature vector;

[0009] The fourth model is used to extract features from the second graph network to obtain the fourth feature vector and the fifth feature vector. The second graph network includes a first node for behavior classification and a second node for sentiment classification. The second graph network is constructed based on the third feature vector.

[0010] The fourth feature vector and the fifth feature vector are predicted respectively to obtain the classification result of the target discourse.

[0011] Secondly, embodiments of this application also provide a discourse classification device, comprising:

[0012] An acquisition module is used to acquire a dialogue sequence, which includes multiple historical utterances and a target utterance.

[0013] The first extraction module is used to extract features from each of the multiple historical discourses and the target discourse using a first model, so as to obtain a first feature vector of the multiple discourses.

[0014] The second extraction module is used to extract features from the first graph network using a second model to obtain a second feature vector. The first graph network is constructed based on the first feature vector of the multiple utterances.

[0015] The third extraction module is used to extract features from the second feature vector using the third model to obtain the third feature vector;

[0016] The fourth extraction module is used to extract features from the second graph network using the fourth model to obtain the fourth feature vector and the fifth feature vector. The second graph network includes a first node for behavior classification and a second node for sentiment classification. The second graph network is constructed based on the third feature vector.

[0017] The prediction module is used to predict the fourth feature vector and the fifth feature vector respectively to obtain the classification result of the target discourse.

[0018] Thirdly, embodiments of this application also provide an electronic device, including: a transceiver, a memory, a processor, and a program stored in the memory and executable on the processor; the processor is configured to read the program in the memory to implement the steps in the method described in the first aspect of embodiments of this application.

[0019] Fourthly, embodiments of this application also provide a readable storage medium storing a program that, when executed by a processor, implements the steps of the method described in the first aspect of embodiments of this application.

[0020] In this embodiment, a first model is used to extract features from each of the multiple historical utterances and the target utterance to obtain a first feature vector for the multiple utterances; a second model is used to extract features from a first graph network to obtain a second feature vector, the first graph network being constructed based on the first feature vectors of the multiple utterances; a third model is used to extract features from the second feature vector to obtain a third feature vector; that is, the contextual information of the dialogue sequence can achieve full interaction. A fourth model is used to extract features from a second graph network to obtain a fourth feature vector and a fifth feature vector, the second graph network including a first node for behavior classification and a second node for sentiment classification, the second graph network being constructed based on the third feature vector; that is, the information between the first node for behavior classification and the second node for sentiment classification can also interact to achieve feature fusion of behavior classification and sentiment classification, thereby predicting the fourth feature vector and the fifth feature vector respectively, improving the accuracy of the classification result of the target utterance. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of this application, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating a discourse classification method provided in an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of a multi-task collaborative model provided in an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of a graph attention network provided in an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the structure of a discourse classification device provided in an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] The terms "first," "second," etc., used in the embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. Additionally, the use of "and / or" in this application indicates at least one of the connected objects, such as A and / or B and / or C, representing seven possibilities: including A alone, B alone, C alone, and the presence of both A and B, both B and C, both A and C, and the presence of A, B, and C.

[0029] Please see Figure 1 , Figure 1 This is a flowchart illustrating a discourse classification method provided in an embodiment of this application, such as... Figure 1 As shown, it includes the following steps:

[0030] Step 101: Obtain the dialogue sequence, which includes multiple historical utterances and the target utterance.

[0031] The dialogue sequence includes dialogue between at least two characters. It can be understood that multiple utterances in the dialogue sequence can be arranged chronologically. The target utterance is the latest utterance in the dialogue sequence, and the multiple historical utterances are multiple utterances in the dialogue sequence that precede the target utterance.

[0032] Step 102: Use the first model to extract features from each of the multiple historical discourses and the target discourse to obtain the first feature vector of the multiple discourses.

[0033] The first model described above can extract features for each utterance, meaning that each utterance corresponds to a first feature vector, and the first feature vector of each utterance can be used as a representation of that utterance.

[0034] It is understood that each of the above utterances may include multiple words. When extracting features from each utterance, long-term dependency information can be learned through the first model mentioned above. For example, the first model mentioned above can use a bidirectional long short-term memory recurrent neural network (BiLSTM) model, a gated recurrent unit (GRU) model, etc.

[0035] Furthermore, before using the first model to extract features from each utterance, an embedding function can be used to transform the words in each utterance into word vectors, which are then input into the first model. Specifically, the dialogue sequence obtained in step 101 can be a dialogue sequence mapped to a corresponding vector form, which can be directly input into the first model in step 102; alternatively, the dialogue sequence can be in text form, in which case, before feature extraction in step 102, the dialogue sequence can be mapped to a corresponding vector form using an embedding function before being input into the first model.

[0036] Step 103: Use the second model to extract features from the first graph network to obtain a second feature vector. The first graph network is constructed based on the first feature vector of the multiple utterances.

[0037] The second model mentioned above can be a graph neural network model (e.g., a graph attention neural network model). By using the second model to iteratively update each node in the first graph network, the information interaction and fusion between the nodes are obtained to obtain the second feature vector.

[0038] Optionally, the nodes of the first graph network correspond one-to-one with the utterances in the dialogue sequence;

[0039] The first graph network is constructed in the following way:

[0040] The nodes of the first graph network are constructed based on the first feature vectors of the multiple utterances;

[0041] The edges of the first graph network are constructed based on the dialogue sequence. The edges of the first graph network include the edges between nodes corresponding to adjacent utterances in the dialogue sequence, and the edges between nodes corresponding to utterances belonging to the same role.

[0042] It can be understood that each node in the first graph network described above corresponds to a utterance; that is, a corresponding node can be initialized using the first feature vector of each utterance. The edges in the first graph network include two types: edges between nodes corresponding to adjacent utterances; and edges between nodes corresponding to utterances belonging to the same role. Adjacent utterances are those determined by chronological order, and utterances belonging to the same role are those fed back by the same role. Through feature extraction from the first graph network, the information of each node can be transmitted to nodes corresponding to adjacent utterances and nodes corresponding to utterances belonging to the same role, thereby capturing the dependencies between utterances in the dialogue sequence and capturing the utterance information between utterances of the same role in the dialogue sequence.

[0043] In this embodiment, during the construction of the first graph network, nodes of the first graph network are constructed based on the first feature vectors of the multiple utterances; edges of the first graph network are constructed based on the dialogue sequence. The edges of the first graph network include edges between nodes corresponding to adjacent utterances in the dialogue sequence, and edges between nodes corresponding to utterances belonging to the same role. That is, node information in the first graph network can be transmitted and updated between nodes corresponding to adjacent utterances and nodes corresponding to utterances belonging to the same role, which can better realize the interaction of utterance information between the same role and different roles and improve the accuracy of feature extraction.

[0044] Step 104: Use the third model to extract features from the second feature vector to obtain the third feature vector.

[0045] The third model mentioned above can be a model of the same type as the first model mentioned above. It can be understood that the second feature vector can be a vector obtained by concatenating the node vectors in multiple first graph networks. Each node vector can correspond to a utterance. Taking the BiLSTM model as the third model as an example, by extracting the second feature vector according to the order and reverse order of the utterances corresponding to each node vector, the information of each node vector can be fully integrated.

[0046] Step 105: Use the fourth model to extract features from the second graph network to obtain the fourth feature vector and the fifth feature vector. The second graph network includes a first node for behavior classification and a second node for sentiment classification. The second graph network is constructed based on the third feature vector.

[0047] The number of the first nodes is the same as the number of the second nodes. In the second graph network, the third feature vector includes multiple vectors that correspond one-to-one with the utterances. Each vector is used to initialize a first node and a second node. That is, the initialization information of each first node is the same as the initialization information of the corresponding second node.

[0048] It is understandable that in a dialogue sequence, the behavior category and sentiment category of each utterance are dependent on each other. For example, when the current speaker expresses agreement with the previous speaker, their sentiment categories should be close or the same; conversely, when the current speaker expresses disagreement with the previous speaker, their sentiment categories should be different or opposite. In some embodiments, the fourth model described above can be a graph neural network model. Taking a graph attention neural network model as an example, through iterative updates of the first and second nodes in the second graph network, sufficient information interaction can be achieved between the first node used for behavior classification and the second node used for sentiment classification.

[0049] Step 106: Predict the fourth feature vector and the fifth feature vector respectively to obtain the classification result of the target discourse.

[0050] Optionally, the fourth feature vector corresponds to the first node, the fifth feature vector corresponds to the second node, and the classification result of the target discourse includes behavior classification result and sentiment classification result;

[0051] The prediction of the fourth feature vector and the fifth feature vector respectively includes:

[0052] The fifth model is used to predict the fourth feature vector to obtain the behavior classification result of the target discourse;

[0053] The sixth model is used to predict the fifth feature vector to obtain the sentiment classification result of the target discourse.

[0054] It is understandable that the fourth and fifth feature vectors mentioned above can each correspond to two classification tasks. Before step 106, the information for the two classification tasks is fully fused. By encoding the semantic information corresponding to the two classification tasks into different representations, information gain and improved prediction accuracy of the classification tasks can be achieved. The fifth and sixth models mentioned above can be of the same type, but their model parameters differ for different recognition tasks. For example, they can both use a softmax (normalized exponential function) model to predict the probability of each category, obtaining the category with the highest probability as the target classification result.

[0055] In this implementation, the fourth feature vector and the fifth feature vector can be identified separately based on different classification and recognition tasks, ensuring the interaction between the two tasks in the information acquisition process and further improving the accuracy of classification and recognition.

[0056] In this embodiment, a first model is used to extract features from each of the multiple historical utterances and the target utterance to obtain a first feature vector for the multiple utterances; a second model is used to extract features from a first graph network to obtain a second feature vector, the first graph network being constructed based on the first feature vectors of the multiple utterances; a third model is used to extract features from the second feature vector to obtain a third feature vector; that is, the contextual information of the dialogue sequence can achieve full interaction. A fourth model is used to extract features from a second graph network to obtain a fourth feature vector and a fifth feature vector, the second graph network including a first node for behavior classification and a second node for sentiment classification, the second graph network being constructed based on the third feature vector; that is, the information between the first node for behavior classification and the second node for sentiment classification can also interact to achieve feature fusion of behavior classification and sentiment classification, thereby predicting the fourth feature vector and the fifth feature vector respectively, improving the accuracy of the classification result of the target utterance.

[0057] Optionally, the edges of the second graph network include edges between the first nodes, edges between the second nodes, and edges between the first node and the second node;

[0058] The method may also include the following steps:

[0059] During the feature extraction process of the second graph network in the fourth model, the first node and the second node are updated to each other based on the edges of the second graph network.

[0060] In this embodiment, the edges of the second graph network include the edges between the first nodes, the edges between the second nodes, and the edges between the first node and the second node. During the feature extraction process of the second graph network by the fourth model, the first node and the second node are updated to each other based on the edges of the second graph network, which can realize the interaction of behavior classification and sentiment classification information and improve the accuracy of the classification results.

[0061] Optionally, the second feature vector includes a plurality of sixth feature vectors corresponding to the nodes of the first graph network;

[0062] The step of using a third model to extract features from the second feature vector may specifically include:

[0063] The third model is used to extract features from the sixth feature vectors corresponding to the nodes of the first graph network.

[0064] In some embodiments, the above-mentioned multiple sixth feature vectors can be concatenated to obtain the above-mentioned second feature vector. Taking the above-mentioned third model as a BiLSTM model as an example, in the process of using the third model to extract features from the multiple sixth feature vectors corresponding to the nodes of the first graph network, the information of the above-mentioned multiple sixth feature vectors can be fully integrated, thereby capturing the regular information and dependencies between the utterances in the dialogue sequence through the third feature vector.

[0065] Optionally, the discourse prediction model is trained as follows:

[0066] The discourse prediction model is iteratively trained using multiple pre-labeled dialogue sequence samples;

[0067] The loss value of the output of the discourse prediction model is calculated using a loss function;

[0068] The loss value is used to update the model parameters of the discourse prediction model until the loss function converges.

[0069] The discourse prediction model includes the first model, the second model, the third model, the fourth model, the fifth model, and the sixth model.

[0070] It is understandable that the above-mentioned discourse prediction model can be used to predict the above-mentioned dialogue sequence and obtain the classification result of the above-mentioned target discourse. During the training process of the above-mentioned discourse prediction model, the model parameters of the first model, the second model, the third model, the fourth model, the fifth model, and the sixth model can be updated synchronously. That is, during the training process of the above-mentioned classification and recognition task, the accuracy of the final result can be improved by updating the parameters of each model.

[0071] Optionally, embodiments of this application also provide a discourse classification method, including:

[0072] Obtain a discourse prediction model;

[0073] The discourse prediction model is used to predict the dialogue sequence to obtain the classification result of the target discourse. The dialogue sequence includes multiple historical discourses and the target discourse.

[0074] It is understood that the above-mentioned discourse prediction model includes the first model, the second model, the third model, the fourth model, the fifth model, and the sixth model. The first model is used to extract features from each of the multiple historical discourses and the target discourse to obtain a first feature vector for the multiple discourses. The second model is used to extract features from a first graph network to obtain a second feature vector, and the first graph network is constructed based on the first feature vectors of the multiple discourses. The third model is used to extract features from the second feature vector to obtain a third feature vector. The fourth model is used to extract features from a second graph network to obtain a fourth and a fifth feature vector, and the second graph network is constructed based on the third feature vector. The fifth model is used to predict the fourth feature vector to obtain the behavior classification result of the target discourse. The sixth model is used to predict the fifth feature vector to obtain the sentiment classification result of the target discourse.

[0075] In this embodiment, the discourse prediction model includes the first model, the second model, the third model, the fourth model, the fifth model, and the sixth model. The discourse prediction model used for discourse classification can improve the accuracy of the prediction results by updating the model parameters of the discourse prediction model using the loss value.

[0076] The various optional implementation methods described in the embodiments of this application can be combined with each other or implemented individually without conflict. The embodiments of this application do not limit this.

[0077] For ease of understanding, a specific example is as follows:

[0078] Figure 2 This is a schematic diagram of a multi-task collaborative model provided in an embodiment of this application, such as... Figure 2 As shown, the multi-task collaborative model can specifically include a speaker interaction perception encoder module, a context collaboration graph interaction module, and a decoder module, which are used to realize dialogue behavior recognition and classification as well as emotion recognition and classification.

[0079] Among them, the speaker interaction perception module is used to fuse the speech information of different speakers, the context collaborative interaction module is used to fuse the context information in the dialogue sequence, and the decoder module is used to decode and predict the extracted feature vectors to achieve behavior recognition classification and emotion recognition classification.

[0080] Specifically, the speaker interaction perception module includes a dialogue statement encoder and a dialogue role encoder. In the speaker interaction perception module, the two tasks of dialogue behavior recognition and emotion classification share the same input information.

[0081] A dialogue statement encoder is used to encode each statement in a dialogue sequence. The input to the dialogue statement encoder is a sequence C = (u1, u2, ..., u...) containing the history of the dialogue and the current dialogue. N ), where N is the length of the dialogue sequence. (Discourse) (t∈N), where n is the number of words in the t-th round of utterance in the dialogue sequence. The goal of the multi-task collaborative model is to improve u [1，2…，n] (Dialogue sequence) modeling to simultaneously predict u n Dialogue behavior tags and dialogue emotion tags for (current round of speech).

[0082] In the dialogue statement encoder, when the sequence C = (u1, u2, ..., u...) is... N Before inputting the utterance into the model, each word should be transformed into its corresponding word embedding. This can be achieved using an embedding function. Will The tokens (characters) in the code are mapped to corresponding vectors, and then the BiLSTM encoder processes them sequentially (from...). arrive ) and reverse order (from arrive Read and generate a series of context-dependent latent vectors The specific process is as follows:

[0083]

[0084]

[0085]

[0086] in, Let represent the latent vector generated by the positive forward generation of the i-th character in the t-th utterance. Indicates a forward LSTM, Indicates an embedded function. This represents the i-th character in the utterance. This represents the latent vector generated by the forward propagation of the (i-1)th character of the t-th utterance. Let represent the latent vector generated by reversing the i-th character in the t-th utterance. Indicates reverse LSTM, This represents the latent vector generated by reversing the i-1th character of the t-th utterance. This represents the hidden vector generated by the BiLSTM encoder.

[0087] Among them, the last hidden vector can be Treat as a whole round statement u t The representation of e t Specifically manifested as Therefore, a dialogue sequence C containing N rounds of statements can be represented by the encoder as E = (e1, e2, ..., e...). N ).

[0088] The dialogue role encoder is used to merge contextual and interaction information, which can better reflect the internal logic of the roles (speakers) in a dialogue. The dialogue role encoder can use a Graph Attention Network (GAT) to fully utilize the dependencies between the same and different roles in the dialogue, enabling the model to better understand changes in emotional and behavioral intentions within and between roles. The Graph Attention Network provided in this application, based on traditional dialogue sequence methods, explicitly defines the dependencies between roles. For example... Figure 2 As shown, the vertices of the graph represent the statements in each round of dialogue. Specifically, for all i∈[1,2,…,N] that satisfy the conditions, the feature vector e is first encoded using serialization. i After initialization, the initial vectors of all nodes can be represented as E = (e1, e2, ..., e...). N Regarding the edge configuration in the graph network: to introduce dependencies between different roles, this application connects adjacent round statements in chronological order; to capture relationships between nodes of the same role, this application connects nodes belonging to the same role. Specifically, for the adjacency matrix A∈R... N×N If node i and node j belong to adjacent rounds or come from the same role, then A ij =1, otherwise A ij =0. For example... Figure 2 As shown, utterance u1 corresponds to node h1 in the graph network, utterance u2 corresponds to node h2, utterance u3 corresponds to node h3, utterance u4 corresponds to node h4, and utterance u5 corresponds to node h5. It can be understood that the temporal order of each utterance in the dialogue sequence is u1, u2, u3, u4, u5. Therefore, node h1 is connected to node h2, node h2 is connected to node h3, node h3 is connected to node h4, and node h4 is connected to node h5. Furthermore, u1, u3, and u5 belong to the same speaker, and u2 and u4 belong to the same speaker; therefore, node h1 is connected to node h3, node h3 is connected to node h5, and node h2 is connected to node h4. Through the above steps, the current node information can be passed to adjacent nodes with the same role and dialogue context. The node vector update is shown below:

[0089]

[0090] Among them, e i Let ' represent the updated node vector of node i, || represent vector concatenation, K represent the number of layers in the graph attention network, and σ represent the non-linear activation function. This indicates that the attention weight coefficients from node i to node j are calculated by the k-th attention mechanism. Let e ​​represent the learning parameters corresponding to the k-th attention mechanism. j S represents the node vector of node j. i A set of adjacent nodes or nodes with the same role.

[0091] like Figure 3 As shown, Figure 3 This is a schematic diagram of a single-layer graph attention network. Multiple keys are processed by an attention function f to calculate the degree of relevance of the sequence information to the query vector q, i.e., the attention weights (S1, S2, ..., S...). T Then, the attention weights are normalized using the softmax function to obtain the attention weight distribution (α1, α2, ..., α...). T This approach uses attention weights to weighted sums of values ​​to select more useful information. It can be understood that after m iterations of updates, the node vectors described above can obtain a deeper implicit vector representation. For behavior recognition and emotion classification tasks, bidirectional LSTM encoders can be used to specifically target E. m The encoding can be represented as D o =LSTM(E m ), S o =LSTM(E m ),in

[0092] For the context-cooperative graph interaction module, a graph attention network can also be used to model the context information and role interaction information, such as... Figure 2 As shown, cross-task connections facilitate capturing dependencies between tasks. The graph network has 2N vertices: N vertices for action recognition and N vertices for sentiment classification. We use the D obtained above... o and S o To initialize the vertex representation in the graph network, i.e. (d represents the dimension of the vector). There are two types of edges in the graph: connections between sentences and connections between tasks. Specifically, nodes belonging to the same task and the same dialogue are first connected to obtain contextual information, and then nodes from different tasks within the same dialogue are connected to obtain interaction information. The iterative update process of the interaction is as follows:

[0093]

[0094]

[0095] in, Let K represent the semantic representation of the behavior of node i in layer (l+1), K represent the number of layers in the graph attention network, σ represent the non-linear activation function, and D represent the semantic representation of the behavior of node i in layer (l+1). i This represents the set of nodes used for behavior recognition. This indicates that the attention weight coefficients from node i to node j are calculated by the k-th attention mechanism. This represents the learning parameters corresponding to the k-th attention mechanism. A represents the semantic representation of the behavior of node i at level l. i Let i represent the set of nodes connected to node i for emotion recognition. The sentiment semantic representation of node i in the l-th layer. D′ represents the sentiment semantic representation of node i at layer l+1. i Let A' represent the set of nodes used for emotion recognition. i This represents the set of nodes connected to node i for behavior recognition.

[0096] Through the above iterative update process, representations of the two types of tasks are obtained. and These represent the behavioral semantic representation of node m and the emotional semantic representation of node n, respectively.

[0097] In the decoder module, two separate encoders are used to obtain the classification results for behavior recognition and emotion classification. This is based on the representations of the two types of task nodes mentioned above. and The behavior recognition representation of the statement in the t-th round is obtained respectively. And the sentiment recognition representation of the t-th round of statements. The decoding process of the statement in round t is as follows:

[0098]

[0099]

[0100] in, W represents the probability of the behavior category of the statement in round t. d Indicating the decoder used for behavior recognition The weight, b represents the semantic representation of the behavior of the statement in round t. d This represents the bias of the decoder used for behavior recognition. W represents the probability of the sentiment category of the statement in round t. s This indicates the decoder used for emotion recognition. The weight, b represents the sentiment semantic representation of the statement in round t. s This indicates the bias of the decoder used for emotion recognition.

[0101] The behavioral and emotional categories of the t-th round of discourse are determined by the final probabilities obtained above.

[0102] In addition, the multi-task collaborative model is trained using multiple sample sequences, which contain behavior recognition labels and sentiment classification labels. The loss function L0 is shown below:

[0103] L0 = L1 + L2;

[0104]

[0105]

[0106] Where N represents the number of utterances in the sample sequence, N D N represents the number of labels used for behavior recognition. S The number of tags indicating the sentiment category. This represents the true label for the j-th action identified in the i-th utterance. Let represent the label identified for the j-th action in the i-th utterance. This represents the true label for the j-th sentiment category in the i-th utterance. This represents the label for the j-th emotion in the i-th utterance.

[0107] In this embodiment, by utilizing the different semantic information of historical rounds between tasks, the semantics of the current round are encoded into different representations. The resulting information gain can improve the accuracy of multi-task prediction. Furthermore, since behavior recognition tasks and emotion recognition tasks are highly correlated, the information gain between tasks can be fully utilized. In addition, through cross-discourse and cross-task connection methods, as well as network construction and iterative update methods, a modeling process that combines contextual information and interaction information can be achieved simultaneously, further improving the accuracy of multi-task prediction.

[0108] See Figure 4 , Figure 4 This is a schematic diagram of the structure of a data prediction device provided in an embodiment of this application. Figure 4 As shown, the discourse classification device 400 includes:

[0109] The acquisition module 401 is used to acquire a dialogue sequence, which includes multiple historical utterances and a target utterance.

[0110] The first extraction module 402 is used to extract features from each of the multiple historical discourses and the target discourse using a first model, so as to obtain a first feature vector of the multiple discourses.

[0111] The second extraction module 403 is used to extract features from the first graph network using a second model to obtain a second feature vector, wherein the first graph network is constructed based on the first feature vector of the multiple utterances;

[0112] The third extraction module 404 is used to extract features from the second feature vector using a third model to obtain a third feature vector;

[0113] The fourth extraction module 405 is used to extract features from the second graph network using the fourth model to obtain a fourth feature vector and a fifth feature vector. The second graph network includes a first node for behavior classification and a second node for sentiment classification. The second graph network is constructed based on the third feature vector.

[0114] The prediction module 406 is used to predict the fourth feature vector and the fifth feature vector respectively to obtain the classification result of the target discourse.

[0115] Optionally, the edges of the second graph network include edges between the first nodes, edges between the second nodes, and edges between the first node and the second node;

[0116] The device 400 further includes:

[0117] The update module is used to update the first node and the second node based on the edges of the second graph network during the feature extraction process of the second graph network by the fourth model.

[0118] Optionally, the nodes of the first graph network correspond one-to-one with the utterances in the dialogue sequence;

[0119] The first graph network is constructed in the following way:

[0120] The nodes of the first graph network are constructed based on the first feature vectors of the multiple utterances;

[0121] The edges of the first graph network are constructed based on the dialogue sequence. The edges of the first graph network include the edges between nodes corresponding to adjacent utterances in the dialogue sequence, and the edges between nodes corresponding to utterances belonging to the same role.

[0122] Optionally, the second feature vector includes a plurality of sixth feature vectors corresponding to the nodes of the first graph network;

[0123] The third extraction module 404 may specifically include:

[0124] The extraction unit is used to extract features from the sixth feature vectors corresponding to the nodes of the first graph network using the third model.

[0125] Optionally, the fourth feature vector corresponds to the first node, the fifth feature vector corresponds to the second node, and the classification result of the target discourse includes behavior classification result and sentiment classification result;

[0126] The prediction module 406 may specifically include:

[0127] The first prediction unit is used to predict the fourth feature vector using the fifth model to obtain the behavior classification result of the target discourse;

[0128] The second prediction unit is used to predict the fifth feature vector using the sixth model to obtain the sentiment classification result of the target discourse.

[0129] Optionally, the discourse prediction model is trained as follows:

[0130] The discourse prediction model is iteratively trained using multiple pre-labeled dialogue sequence samples;

[0131] The loss value of the output of the discourse prediction model is calculated using a loss function;

[0132] The loss value is used to update the model parameters of the discourse prediction model until the loss function converges.

[0133] The discourse prediction model includes the first model, the second model, the third model, the fourth model, the fifth model, and the sixth model.

[0134] The speech classification device 400 can achieve the functions described in the embodiments of this application. Figure 1 The various processes in the method embodiments, and the ways to achieve the same beneficial effects, will not be repeated here to avoid repetition.

[0135] This application also provides an electronic device. Because the principle by which the electronic device solves the problem is similar to that in the embodiments of this application... Figure 1 The discourse classification methods shown are similar; therefore, the implementation of this electronic device can be found in the implementation of the method, and repeated details will not be elaborated upon. For example... Figure 5 As shown, the electronic device in this application embodiment includes a memory 520, a transceiver 510, and a processor 500;

[0136] The memory 520 is used to store computer programs; the transceiver 510 is used to send and receive data under the control of the processor 500; the processor 500 is used to read the computer programs in the memory 520 and perform the following operations:

[0137] Obtain a dialogue sequence, which includes multiple historical utterances and a target utterance;

[0138] The first model is used to extract features from each of the multiple historical discourses and the target discourse to obtain the first feature vector of the multiple discourses.

[0139] The second model is used to extract features from the first graph network to obtain a second feature vector. The first graph network is constructed based on the first feature vector of the multiple utterances.

[0140] The third model is used to extract features from the second feature vector to obtain the third feature vector;

[0141] The fourth model is used to extract features from the second graph network to obtain the fourth feature vector and the fifth feature vector. The second graph network includes a first node for behavior classification and a second node for sentiment classification. The second graph network is constructed based on the third feature vector.

[0142] The fourth feature vector and the fifth feature vector are predicted respectively to obtain the classification result of the target discourse.

[0143] Among them, Figure 5 In this context, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 500) and memory (memory 520). The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 510 may be multiple elements, including transmitters and transceivers, providing a unit for communicating with various other devices over a transmission medium. The processor 500 is responsible for managing the bus architecture and general processing, and the memory 520 may store data used by the processor 500 during operation.

[0144] The processor 500 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD). The processor can also adopt a multi-core architecture.

[0145] Optionally, the edges of the second graph network include edges between the first nodes, edges between the second nodes, and edges between the first node and the second node;

[0146] Processor 500 is also used to read programs from memory 520 and perform the following steps:

[0147] During the feature extraction process of the second graph network in the fourth model, the first node and the second node are updated to each other based on the edges of the second graph network.

[0148] Optionally, the nodes of the first graph network correspond one-to-one with the utterances in the dialogue sequence;

[0149] The first graph network is constructed in the following way:

[0150] The nodes of the first graph network are constructed based on the first feature vectors of the multiple utterances;

[0151] The edges of the first graph network are constructed based on the dialogue sequence. The edges of the first graph network include the edges between nodes corresponding to adjacent utterances in the dialogue sequence, and the edges between nodes corresponding to utterances belonging to the same role.

[0152] The second feature vector includes multiple sixth feature vectors corresponding to nodes in the first graph network;

[0153] The step of using a third model to extract features from the second feature vector includes:

[0154] The third model is used to extract features from the sixth feature vectors corresponding to the nodes of the first graph network.

[0155] Optionally, the fourth feature vector corresponds to the first node, the fifth feature vector corresponds to the second node, and the classification result of the target discourse includes behavior classification result and sentiment classification result;

[0156] The prediction of the fourth feature vector and the fifth feature vector respectively includes:

[0157] The fifth model is used to predict the fourth feature vector to obtain the behavior classification result of the target discourse;

[0158] The sixth model is used to predict the fifth feature vector to obtain the sentiment classification result of the target discourse.

[0159] Optionally, the discourse prediction model is trained as follows:

[0160] The discourse prediction model is iteratively trained using multiple pre-labeled dialogue sequence samples;

[0161] The loss value of the output of the discourse prediction model is calculated using a loss function;

[0162] The loss value is used to update the model parameters of the discourse prediction model until the loss function converges.

[0163] The discourse prediction model includes the first model, the second model, the third model, the fourth model, the fifth model, and the sixth model.

[0164] The electronic device provided in this application embodiment can perform the above-described functions. Figure 1 The method embodiments shown are similar in principle and technical effect, and will not be described again here.

[0165] This application also provides a readable storage medium storing a program that, when executed by a processor, implements the following... Figure 1 The various processes in the Chinese method embodiment can achieve the same technical effect, and will not be described again here to avoid repetition.

[0166] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0167] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can be physically included separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0168] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute some steps of the transmission and reception methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0169] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A discourse classification method, characterized in that, include: Obtain a dialogue sequence, which includes multiple historical utterances and a target utterance; The first model is used to extract features from each of the multiple historical discourses and the target discourse to obtain the first feature vector of the multiple discourses. The second model is used to extract features from the first graph network to obtain a second feature vector. The first graph network is constructed based on the first feature vector of the multiple utterances. The third model is used to extract features from the second feature vector to obtain the third feature vector; The fourth model is used to extract features from the second graph network to obtain the fourth feature vector and the fifth feature vector. The second graph network includes a first node for behavior classification and a second node for sentiment classification. The second graph network is constructed based on the third feature vector. The fourth feature vector and the fifth feature vector are predicted respectively to obtain the classification result of the target discourse; The edges of the second graph network include the edges between the first nodes, the edges between the second nodes, and the edges between the first node and the second node; The method further includes: During the feature extraction process of the second graph network in the fourth model, the first node and the second node are updated to each other based on the edges of the second graph network.

2. The method as described in claim 1, characterized in that, The nodes of the first graph network correspond one-to-one with the utterances in the dialogue sequence; The first graph network is constructed in the following way: The nodes of the first graph network are constructed based on the first feature vectors of the multiple utterances; The edges of the first graph network are constructed based on the dialogue sequence. The edges of the first graph network include the edges between nodes corresponding to adjacent utterances in the dialogue sequence, and the edges between nodes corresponding to utterances belonging to the same role.

3. The method as described in claim 2, characterized in that, The second feature vector includes multiple sixth feature vectors corresponding to nodes in the first graph network; The step of using a third model to extract features from the second feature vector includes: The third model is used to extract features from the sixth feature vectors corresponding to the nodes of the first graph network.

4. The method as described in claim 1, characterized in that, The fourth feature vector corresponds to the first node, the fifth feature vector corresponds to the second node, and the classification result of the target discourse includes behavior classification result and sentiment classification result; The prediction of the fourth feature vector and the fifth feature vector respectively includes: The fifth model is used to predict the fourth feature vector to obtain the behavior classification result of the target discourse; The sixth model is used to predict the fifth feature vector to obtain the sentiment classification result of the target discourse.

5. The method as described in claim 4, characterized in that, The discourse prediction model is trained in the following way: The discourse prediction model is iteratively trained using multiple pre-labeled dialogue sequence samples; The loss value of the output of the discourse prediction model is calculated using a loss function; The loss value is used to update the model parameters of the discourse prediction model until the loss function converges. The discourse prediction model includes the first model, the second model, the third model, the fourth model, the fifth model, and the sixth model.

6. A discourse classification device, characterized in that, include: An acquisition module is used to acquire a dialogue sequence, which includes multiple historical utterances and a target utterance. The first extraction module is used to extract features from each of the multiple historical discourses and the target discourse using a first model, so as to obtain a first feature vector of the multiple discourses. The second extraction module is used to extract features from the first graph network using a second model to obtain a second feature vector. The first graph network is constructed based on the first feature vector of the multiple utterances. The third extraction module is used to extract features from the second feature vector using the third model to obtain the third feature vector; The fourth extraction module is used to extract features from the second graph network using the fourth model to obtain the fourth feature vector and the fifth feature vector. The second graph network includes a first node for behavior classification and a second node for sentiment classification. The second graph network is constructed based on the third feature vector. The prediction module is used to predict the fourth feature vector and the fifth feature vector respectively to obtain the classification result of the target discourse; The edges of the second graph network include the edges between the first nodes, the edges between the second nodes, and the edges between the first node and the second node; The device further includes: The update module is used to update the first node and the second node based on the edges of the second graph network during the feature extraction process of the second graph network by the fourth model.

7. An electronic device, comprising: A transceiver, a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that, The processor is configured to read a program from memory to implement the steps of the method as described in any one of claims 1 to 5.

8. A readable storage medium, characterized in that, A program is stored on the readable storage medium, which, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 5.

Images