Emotion recognition method and related apparatus

Abstract

Embodiments of the present application disclose a kind of sentiment recognition method and related device, wherein method includes: obtaining to be identified data;According to the first distance matrix of the to-be-identified data;According to the first distance matrix, first feature matrix and second feature matrix are obtained, wherein the first feature matrix is used to show the difference value feature of the to-be-identified data for each emotional label in the preset emotional label, the second feature matrix is used to show the contribution degree of the emotional information of the word vector to the sentiment recognition result of the to-be-identified data, and the emotional information of the word vector is the fusion information of the word vector for the preset emotional label;According to the first feature matrix and the second feature matrix, the sentiment recognition result of the to-be-identified data is output.The accuracy of sentiment recognition result can be improved by the rich feature information in the first feature matrix and the second feature matrix.

Description

Technical Field

[0001] This application relates to computer technology, applied in the field of natural language processing, and particularly to an emotion recognition method and related apparatus. Background Technology

[0002] In outbound calling scenarios, when intelligent agents communicate with customers, they need to adjust their communication scripts in a timely manner based on the emotional tendencies reflected in the customer's information, thereby improving communication efficiency and customer satisfaction with their services.

[0003] Currently, commonly used sentiment classification methods fuse the information of the data to be identified and the sentiment labels by constructing a cosine similarity matrix between them, and then obtain the sentiment recognition result of the text based on the feature information of this cosine similarity matrix. However, the cosine similarity matrix in this method contains relatively little information, which affects the accuracy of the final sentiment recognition result.

[0004] Therefore, how to improve the accuracy of emotion recognition results is a technical problem that those skilled in the art are studying. Summary of the Invention

[0005] This application discloses an emotion recognition method and related apparatus, which can improve the accuracy of emotion recognition results for the data to be recognized.

[0006] In a first aspect, embodiments of this application provide an emotion recognition method, the method comprising:

[0007] Acquire data to be identified, wherein the data to be identified is data containing user emotions;

[0008] A first distance matrix is ​​obtained based on the data to be identified, wherein the first distance matrix includes a difference vector, the difference vector being the distance difference between word vectors and sentiment tag vectors, the word vectors being the vectors corresponding to the words contained in the data to be identified, and the sentiment tag vectors being the vectors corresponding to preset sentiment tags;

[0009] A first feature matrix and a second feature matrix are obtained based on the first distance matrix. The first feature matrix is ​​used to indicate the difference features of the data to be identified for each of the preset sentiment tags. The second feature matrix is ​​used to indicate the degree of contribution of the sentiment information of the word vector to the sentiment recognition result of the data to be identified. The sentiment information of the word vector is the fusion information of the word vector for the preset sentiment tags.

[0010] The emotion recognition result of the data to be identified is output based on the first feature matrix and the second feature matrix.

[0011] The method described above obtains a first distance matrix through vector subtraction. The difference vectors in this first distance matrix reflect not only the angle between the word vectors and the sentiment tag vectors, but also the length between them. In other words, the first distance matrix contains rich fusion information between the data to be identified and the preset sentiment tags. The richer this fusion information, the richer the information contained in the feature matrix obtained by subsequent feature extraction from the first matrix, thus improving the accuracy of the sentiment recognition results for the data to be identified.

[0012] In addition, when extracting features from the first distance matrix, the above method takes into account both the difference features of the data to be identified for each sentiment label and the contribution of the sentiment information of each word in the data to the sentiment recognition result. Therefore, the first feature matrix and the second feature matrix obtained by the above method based on the first distance matrix contain more comprehensive feature information.

[0013] It should be noted that the number of preset sentiment tags has a significant impact on the richness of feature information contained in the first feature matrix, while the number of words in the data to be identified has a significant impact on the richness of feature information contained in the second feature matrix. Therefore, when the number of preset sentiment tags is small or the number of words in the data to be identified is small, the first feature matrix and the second feature matrix can complement each other to further improve the accuracy of the sentiment recognition results of the data to be identified.

[0014] Optionally, the data to be recognized can be either text or speech.

[0015] In conjunction with the first aspect, in one possible implementation, obtaining the first feature matrix and the second feature matrix based on the first distance matrix includes:

[0016] A second distance matrix is ​​obtained based on the first distance matrix and the first attention matrix, wherein the first attention matrix includes attention feature parameters of the word vector relative to the sentiment tag vector;

[0017] The third distance matrix is ​​processed to obtain the first feature matrix, wherein the third distance matrix is ​​determined based on the vectors in the column directions of the second distance matrix;

[0018] A fourth distance matrix is ​​obtained based on the first distance matrix and the second attention matrix, wherein the second attention matrix includes attention feature parameters of the sentiment tag vector relative to the word vector;

[0019] The second feature matrix is ​​obtained based on the fifth distance matrix and the word matrix containing the word vector, wherein the fifth distance matrix is ​​determined based on the vectors in the row direction of the fourth distance matrix.

[0020] The difference vectors along the columns of the first distance matrix represent the difference information of different words in the data to be identified for the same sentiment label. Considering that different words have different levels of importance for the same sentiment label, and the importance of the difference information of different words for the same sentiment label also varies, the above method sets up a first attention matrix. Therefore, the vectors along the columns of the second distance matrix can represent the difference information of different words for the same sentiment label after incorporating attention features. Feature extraction is performed on the vectors along the columns of the second distance matrix, and the resulting first feature matrix can represent the difference features of the data to be identified for each sentiment label based on the fused attention features, which helps to increase the accuracy of the sentiment recognition results of the data to be identified.

[0021] The difference vectors along the rows in the first distance matrix represent the difference information of different sentiment labels for the same word. Considering that different words have corresponding sentiment tendencies, meaning that different sentiment labels have different levels of importance for the same word, and the importance of the difference information of different sentiment labels for the same word also varies, the above method sets up a second attention matrix. Therefore, the vectors along the rows in the fourth distance matrix can represent the difference information of different sentiment labels for the same word, incorporating attention features. Furthermore, considering that the sentiment tendencies of different words have different levels of importance to the sentiment of the data to be identified, the second feature matrix obtained by combining the fourth distance matrix and the word matrix can represent the contribution of the sentiment tendencies of words to the sentiment of the data to be identified, based on the fusion of attention features, which helps to increase the accuracy of the sentiment recognition results of the data to be identified.

[0022] In conjunction with the first aspect, or any of the above possible implementations of the first aspect, in another possible implementation, the processing of the third distance matrix to obtain the first feature matrix includes:

[0023] The third distance matrix is ​​regularized to obtain a regularized matrix;

[0024] The regularization matrix is ​​subjected to feature extraction to obtain the first feature matrix.

[0025] In the above methods, regularization can avoid the gradient explosion problem caused by large amounts of data.

[0026] In a further possible implementation, in conjunction with the first aspect, or any of the above possible implementations of the first aspect, the step of obtaining the second feature matrix based on the fifth distance matrix and the word matrix containing the word vector includes:

[0027] The third attention matrix is ​​obtained based on the fifth distance matrix, wherein the third attention matrix includes attention feature parameters of the word vector relative to the data to be identified;

[0028] The second feature matrix is ​​obtained by performing a dot product on the third attention matrix and the word matrix.

[0029] In conjunction with the first aspect, or any of the above possible implementations of the first aspect, in yet another possible implementation, the emotion recognition result includes an emotion recognition matrix, and the step of outputting the emotion recognition result of the data to be recognized based on the first feature matrix and the second feature matrix includes:

[0030] The first feature matrix and the second feature matrix are concatenated to obtain the first fused feature matrix;

[0031] The first fused feature matrix is ​​subjected to feature extraction to obtain the second fused feature matrix;

[0032] The emotion recognition matrix is ​​output based on the second fusion feature matrix, wherein the emotion recognition matrix is ​​used to reflect the probability information that the emotions of the identified data to be identified are the emotions corresponding to the preset emotion tags.

[0033] The above method maps the first fusion feature matrix to an emotion recognition matrix, which uses the probability information of the emotion of the data to be recognized to represent the emotion recognition result. Therefore, the emotion recognition result is easy for people to understand.

[0034] Secondly, embodiments of this application provide yet another emotion recognition method, the method comprising:

[0035] Obtain training data, wherein the training data is data with corresponding training sentiment labels;

[0036] A first distance matrix is ​​obtained based on the training data, wherein the first distance matrix includes a difference vector, the difference vector being the distance difference between word vectors and sentiment tag vectors, the word vectors being the vectors corresponding to the words contained in the training data, and the sentiment tag vectors being the vectors corresponding to preset sentiment tags;

[0037] The first distance matrix is ​​processed to obtain a first feature matrix and a second feature matrix. The first feature matrix is ​​used to indicate the difference features of the training data for each of the preset sentiment labels. The second feature matrix is ​​used to indicate the degree of contribution of the sentiment information of the word vectors to the sentiment recognition result of the training data. The sentiment information of the word vectors is the fusion information of the word vectors for the preset sentiment labels.

[0038] Based on the first feature matrix and the second feature matrix, output the emotion recognition result of the training data;

[0039] Based on the emotion recognition results and the trained emotion labels, determine the parameters used in obtaining the emotion recognition results.

[0040] The above method continuously optimizes relevant parameters based on the training sentiment labels of the training data and the sentiment recognition results obtained by recognizing the training data, thereby improving the accuracy of sentiment recognition results.

[0041] In conjunction with the second aspect, in one possible implementation, obtaining the first feature matrix and the second feature matrix based on the first distance matrix includes:

[0042] A second distance matrix is ​​obtained based on the first distance matrix and the first attention matrix, wherein the first attention matrix includes attention feature parameters of the word vector relative to the sentiment tag vector;

[0043] The third distance matrix is ​​processed to obtain the first feature matrix, wherein the third distance matrix is ​​determined based on the vectors in the column directions of the second distance matrix;

[0044] A fourth distance matrix is ​​obtained based on the first distance matrix and the second attention matrix, wherein the second attention matrix includes attention feature parameters of the sentiment tag vector relative to the word vector;

[0045] The second feature matrix is ​​obtained based on the fifth distance matrix and the word matrix containing the word vector, wherein the fifth distance matrix is ​​determined based on the vectors in the row direction of the fourth distance matrix.

[0046] In conjunction with the second aspect, or any of the above possible implementations of the second aspect, in another possible implementation, the processing of the third distance matrix to obtain the first feature matrix includes:

[0047] The third distance matrix is ​​regularized to obtain a regularized matrix;

[0048] The regularization matrix is ​​subjected to feature extraction to obtain the first feature matrix.

[0049] In conjunction with the second aspect, or any of the above possible implementations of the second aspect, in yet another possible implementation, obtaining the second feature matrix based on the fifth distance matrix and the word matrix containing the word vector includes:

[0050] The third attention matrix is ​​obtained based on the fifth distance matrix, wherein the third attention matrix includes attention feature parameters of the word vectors relative to the training data;

[0051] The second feature matrix is ​​obtained by performing a dot product on the third attention matrix and the word matrix.

[0052] In conjunction with the second aspect, or any of the possible implementations of the second aspect described above, in yet another possible implementation, the emotion recognition result includes an emotion recognition matrix, and the step of outputting the emotion recognition result of the training data based on the first feature matrix and the second feature matrix includes:

[0053] The first feature matrix and the second feature matrix are concatenated to obtain the first fused feature matrix;

[0054] The first fused feature matrix is ​​subjected to feature extraction to obtain the second fused feature matrix;

[0055] The emotion recognition matrix is ​​output based on the second fusion feature matrix, wherein the emotion recognition matrix is ​​used to reflect the probability information of the emotions of the identified training data corresponding to the preset emotion labels.

[0056] In conjunction with the second aspect, or any of the above possible implementations of the second aspect, in yet another possible implementation, the emotion recognition result includes an emotion recognition matrix, used to reflect the probability information that the recognized emotions of the training data are the emotions represented by the preset emotion labels; the training emotion labels include a training emotion matrix, used to reflect the probability information that the true emotions of the training data are the emotions represented by the preset emotion labels.

[0057] The step of determining the parameters used in obtaining the emotion recognition result based on the emotion recognition result and the trained emotion label includes:

[0058] A first loss value, a second loss value, a third loss value, and a fourth loss value are determined based on the sentiment recognition matrix and the training sentiment matrix. The first loss value is the loss between the sentiment recognition matrix and the training sentiment matrix calculated using a first loss function. The second loss value is the loss between the first feature matrix and the training sentiment matrix calculated using a second loss function. The third loss value is the loss between the second feature matrix and the training sentiment matrix calculated using a third loss function. The fourth loss value is the loss between the sentiment label matrix containing the sentiment label vector and the training sentiment matrix calculated using a fourth loss function.

[0059] If the fifth loss value is less than a preset threshold, the parameters used in obtaining the emotion recognition result are determined based on the first loss value, the second loss value, the third loss value, and the fourth loss value, wherein the fifth loss value is related to the first loss value, the second loss value, the third loss value, and the fourth loss value.

[0060] The first loss value measures the separability of the sentiment recognition matrix, the second loss value measures the separability of the first feature matrix, the third loss value measures the separability of the second feature matrix, and the fourth loss value measures the separability of the sentiment label matrix. Considering that the separability of the first feature matrix, the second feature matrix, and the sentiment label matrix affects the separability of the sentiment recognition matrix, the method described above uses a fifth loss value to determine the parameters, which helps to further improve the separability of the sentiment recognition matrix, thus further improving the accuracy of the sentiment recognition results.

[0061] Thirdly, embodiments of this application provide an emotion recognition device, the device comprising:

[0062] An acquisition unit is used to acquire data to be identified, wherein the data to be identified is data containing user emotions;

[0063] The first processing unit is configured to obtain a first distance matrix based on the data to be identified, wherein the first distance matrix includes a difference vector, the difference vector being the distance difference between a word vector and a sentiment tag vector, the word vector being the vector corresponding to the word contained in the data to be identified, and the sentiment tag vector being the vector corresponding to a preset sentiment tag;

[0064] The second processing unit is configured to obtain a first feature matrix and a second feature matrix based on the first distance matrix, wherein the first feature matrix is ​​used to indicate the difference features of the data to be identified for each of the preset sentiment tags, and the second feature matrix is ​​used to indicate the degree of contribution of the sentiment information of the word vector to the sentiment recognition result of the data to be identified, and the sentiment information of the word vector is the fusion information of the word vector for the preset sentiment tags;

[0065] The output unit is used to output the emotion recognition result of the data to be identified based on the first feature matrix and the second feature matrix.

[0066] In conjunction with the third aspect, in one possible implementation, regarding the obtaining of the first feature matrix and the second feature matrix based on the first distance matrix, the second processing unit is specifically used for:

[0067] A second distance matrix is ​​obtained based on the first distance matrix and the first attention matrix, wherein the first attention matrix includes attention feature parameters of the word vector relative to the sentiment tag vector;

[0068] The third distance matrix is ​​processed to obtain the first feature matrix, wherein the third distance matrix is ​​determined based on the vectors in the column directions of the second distance matrix;

[0069] A fourth distance matrix is ​​obtained based on the first distance matrix and the second attention matrix, wherein the second attention matrix includes attention feature parameters of the sentiment tag vector relative to the word vector;

[0070] The second feature matrix is ​​obtained based on the fifth distance matrix and the word matrix containing the word vector, wherein the fifth distance matrix is ​​determined based on the vectors in the row direction of the fourth distance matrix.

[0071] In conjunction with the third aspect, or any of the above possible implementations of the third aspect, in another possible implementation, regarding the processing of the third distance matrix to obtain the first feature matrix, the second processing unit is specifically used for:

[0072] The third distance matrix is ​​regularized to obtain a regularized matrix;

[0073] The regularization matrix is ​​subjected to feature extraction to obtain the first feature matrix.

[0074] In conjunction with the third aspect, or any of the above possible implementations of the third aspect, in yet another possible implementation, regarding the obtaining of the second feature matrix based on the fifth distance matrix and the word matrix containing the word vectors, the second processing unit is specifically used for:

[0075] The third attention matrix is ​​obtained based on the fifth distance matrix, wherein the third attention matrix includes attention feature parameters of the word vector relative to the data to be identified;

[0076] The second feature matrix is ​​obtained by performing a dot product on the third attention matrix and the word matrix.

[0077] In conjunction with the third aspect, or any of the above possible implementations of the third aspect, in yet another possible implementation, the emotion recognition result includes an emotion recognition matrix. Regarding the output of the emotion recognition result of the data to be recognized based on the first feature matrix and the second feature matrix, the output unit is specifically used for:

[0078] The first feature matrix and the second feature matrix are concatenated to obtain the first fused feature matrix;

[0079] The first fused feature matrix is ​​subjected to feature extraction to obtain the second fused feature matrix;

[0080] The emotion recognition matrix is ​​output based on the second fusion feature matrix, wherein the emotion recognition matrix is ​​used to reflect the probability information that the emotions of the identified data to be identified are the emotions corresponding to the preset emotion tags.

[0081] Fourthly, embodiments of this application provide yet another emotion recognition device, the device comprising:

[0082] An acquisition unit is used to acquire training data, wherein the training data is data with corresponding training sentiment labels;

[0083] The first processing unit is configured to obtain a first distance matrix based on the training data, wherein the first distance matrix includes a difference vector, the difference vector being the distance difference between word vectors and sentiment tag vectors, the word vectors being vectors corresponding to words contained in the training data, and the sentiment tag vectors being vectors corresponding to preset sentiment tags;

[0084] The second processing unit is used to process the first distance matrix to obtain a first feature matrix and a second feature matrix. The first feature matrix is ​​used to indicate the difference features of the training data for each of the preset sentiment tags. The second feature matrix is ​​used to indicate the degree of contribution of the sentiment information of the word vector to the sentiment recognition result of the training data. The sentiment information of the word vector is the fusion information of the word vector for the preset sentiment tags.

[0085] The output unit is configured to output the emotion recognition result of the training data based on the first feature matrix and the second feature matrix.

[0086] The determining unit is used to determine the parameters used in obtaining the emotion recognition result based on the emotion recognition result and the trained emotion label.

[0087] In conjunction with the fourth aspect, in one possible implementation, regarding the obtaining of the first feature matrix and the second feature matrix based on the first distance matrix, the second processing unit is specifically used for:

[0088] A second distance matrix is ​​obtained based on the first distance matrix and the first attention matrix, wherein the first attention matrix includes attention feature parameters of the word vector relative to the sentiment tag vector;

[0089] The third distance matrix is ​​processed to obtain the first feature matrix, wherein the third distance matrix is ​​determined based on the vectors in the column directions of the second distance matrix;

[0090] A fourth distance matrix is ​​obtained based on the first distance matrix and the second attention matrix, wherein the second attention matrix includes attention feature parameters of the sentiment tag vector relative to the word vector;

[0091] The second feature matrix is ​​obtained based on the fifth distance matrix and the word matrix containing the word vector, wherein the fifth distance matrix is ​​determined based on the vectors in the row direction of the fourth distance matrix.

[0092] In conjunction with the fourth aspect, or any of the above possible implementations of the fourth aspect, in another possible implementation, regarding the processing of the third distance matrix to obtain the first feature matrix, the second processing unit is specifically used for:

[0093] The third distance matrix is ​​regularized to obtain a regularized matrix;

[0094] The regularization matrix is ​​subjected to feature extraction to obtain the first feature matrix.

[0095] In conjunction with the fourth aspect, or any of the above possible implementations of the fourth aspect, in yet another possible implementation, regarding the obtaining of the second feature matrix based on the fifth distance matrix and the word matrix containing the word vectors, the second processing unit is specifically used for:

[0096] The third attention matrix is ​​obtained based on the fifth distance matrix, wherein the third attention matrix includes attention feature parameters of the word vectors relative to the training data;

[0097] The second feature matrix is ​​obtained by performing a dot product on the third attention matrix and the word matrix.

[0098] In conjunction with the fourth aspect, or any of the above possible implementations of the fourth aspect, in yet another possible implementation, regarding the aspect where the emotion recognition result includes an emotion recognition matrix, and the output unit outputs the emotion recognition result of the training data based on the first feature matrix and the second feature matrix, the output unit is specifically used for:

[0099] The first feature matrix and the second feature matrix are concatenated to obtain the first fused feature matrix;

[0100] The first fused feature matrix is ​​subjected to feature extraction to obtain the second fused feature matrix;

[0101] The emotion recognition matrix is ​​output based on the second fusion feature matrix, wherein the emotion recognition matrix is ​​used to reflect the probability information of the emotions of the identified training data corresponding to the preset emotion labels.

[0102] In conjunction with the fourth aspect, or any of the above possible implementations of the fourth aspect, in yet another possible implementation, the emotion recognition result includes an emotion recognition matrix, used to reflect the probability information that the recognized emotions of the training data are the emotions represented by the preset emotion labels; the training emotion labels include a training emotion matrix, used to reflect the probability information that the true emotions of the training data are the emotions represented by the preset emotion labels.

[0103] In determining the parameters used in obtaining the emotion recognition result based on the emotion recognition result and the trained emotion label, the determining unit is specifically used for:

[0104] A first loss value, a second loss value, a third loss value, and a fourth loss value are determined based on the sentiment recognition matrix and the training sentiment matrix. The first loss value is the loss between the sentiment recognition matrix and the training sentiment matrix calculated using a first loss function. The second loss value is the loss between the first feature matrix and the training sentiment matrix calculated using a second loss function. The third loss value is the loss between the second feature matrix and the training sentiment matrix calculated using a third loss function. The fourth loss value is the loss between the sentiment label matrix containing the sentiment label vector and the training sentiment matrix calculated using a fourth loss function.

[0105] If the fifth loss value is less than a preset threshold, the parameters used in obtaining the emotion recognition result are determined based on the first loss value, the second loss value, the third loss value, and the fourth loss value, wherein the fifth loss value is related to the first loss value, the second loss value, the third loss value, and the fourth loss value.

[0106] Fifthly, embodiments of this application provide an emotion recognition device, which includes a processor, a memory, and a communication interface. The communication interface is used to perform receiving and / or sending operations under the control of the processor. The memory is used to store a computer program, and the processor is used to call the computer program to implement the method described in the first aspect or any possible implementation of the first aspect.

[0107] In a sixth aspect, embodiments of this application provide yet another emotion recognition device, which includes a processor, a memory, and a communication interface, wherein the communication interface is used to perform receiving and / or sending operations under the control of the processor, the memory is used to store a computer program, and the processor is used to call the computer program to implement the method described in the second aspect or any possible implementation of the second aspect.

[0108] In a seventh aspect, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a processor, implements the method described in the first aspect or any possible implementation thereof.

[0109] Eighthly, embodiments of this application provide yet another computer-readable storage medium storing a computer program that, when run on a processor, implements the method described in the second aspect or any possible implementation thereof.

[0110] The beneficial effects of the technical method provided in the second aspect of this application can be referred to the beneficial effects of the technical solution in the first aspect, and will not be repeated here.

[0111] The beneficial effects of the related devices provided in aspects three to eight of this application can be referred to the beneficial effects of the technical solution in aspect one, and will not be repeated here. Attached Figure Description

[0112] The accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0113] Figure 1 This is a schematic diagram of the architecture of an emotion recognition system provided in an embodiment of this application;

[0114] Figure 2 This is a flowchart illustrating an emotion recognition method provided in an embodiment of this application;

[0115] Figure 3 This is a schematic diagram of an emotion recognition algorithm provided in an embodiment of this application;

[0116] Figure 4 This is a flowchart illustrating another emotion recognition method provided in an embodiment of this application;

[0117] Figure 5 This is a schematic diagram of the structure of an emotion recognition device 50 provided in an embodiment of this application;

[0118] Figure 6 This is a schematic diagram of the structure of an emotion recognition device 60 provided in an embodiment of this application. Detailed Implementation

[0119] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0120] Please see Figure 1 , Figure 1 This is a schematic diagram of the architecture of an emotion recognition system provided in an embodiment of this application. The system includes an outbound calling device 101 and an emotion recognition device 102. The number of outbound calling devices 101 is not strictly limited in this application. Figure 1 The quantities shown are for illustrative purposes only.

[0121] Outbound calling device 101 is an electronic device with both data processing and data transmission / reception capabilities, used to receive and intelligently respond to customer voice messages. Outbound calling device 101 can send voice messages as data to be identified to emotion recognition device 102, facilitating subsequent identification of the emotion type of the data. Outbound calling device 101 can also receive the emotion recognition results corresponding to the data to be identified from emotion recognition device 102, enabling intelligent agents to select appropriate response scripts based on these results, thereby improving customer satisfaction with their service.

[0122] Optionally, the outbound calling device 101 can be a standalone device such as a handheld terminal, desktop terminal, or wearable device, or it can be a component (such as a chip or integrated circuit) included in a standalone device. When the outbound calling device 101 is a handheld terminal, it can be a mobile phone, tablet computer, computer (such as a laptop computer, PDA, etc.).

[0123] The emotion recognition device 102 is a device with computing power and storage space, which can store speech recognition algorithms and emotion recognition algorithms. The emotion recognition device 102 can receive data to be recognized sent by the outbound calling device 101, use the speech recognition algorithm to convert the data to be recognized into corresponding text, then use the emotion recognition algorithm to recognize the emotion of the text, and output the emotion recognition result corresponding to the data to be recognized to the outbound calling device 101.

[0124] Optionally, the outbound calling device 101 can also receive and intelligently reply to customers' text messages, and send the text messages as data to be identified to the emotion recognition device 102. That is to say, when the data to be identified is a text message, the emotion recognition device 102 can directly identify the emotion of the text message through the emotion recognition algorithm.

[0125] Optionally, the emotion recognition algorithm stored in the emotion recognition device 102 can be an algorithm obtained by the emotion recognition device 102 through training multiple training data with training emotion labels, or it can be a pre-trained algorithm obtained directly from other training devices.

[0126] Optionally, the emotion recognition device 102 can be in the cloud, such as a single service or a server cluster consisting of multiple servers in the cloud, or it can be a local device, such as a single service or a server cluster consisting of multiple servers locally.

[0127] The emotion recognition device 102 and the outbound calling device 101 can be directly connected via wired communication, such as tangible media like metal wires or optical fibers, or indirectly connected via wireless communication, such as intangible media like wireless local area networks or Bluetooth.

[0128] Optionally, the outbound calling device 101 may perform the aforementioned calculations performed by the emotion recognition device 102.

[0129] In summary, Figure 1 In the system architecture shown, the intelligent agent can obtain customer messages (voice messages or text messages) through the outbound calling device, as well as the emotion recognition result corresponding to the customer message sent by the emotion recognition device, so as to facilitate timely adjustment of response scripts based on the customer's emotions.

[0130] Please see Figure 2 , Figure 2 This is a flowchart illustrating an emotion recognition method provided in an embodiment of this application. The method can be based on... Figure 1 The system architecture shown is used to implement this method, which includes, but is not limited to, the following steps:

[0131] Step S201: The emotion recognition device acquires the data to be recognized.

[0132] Emotion recognition devices can be Figure 1 The emotion recognition device 102 shown can also be other devices. The data to be recognized is data containing user emotions, which can be voice or text messages received from customers by the outbound calling device. The outbound calling device can be... Figure 1 The outbound calling device 101 shown can also be other devices.

[0133] After receiving data to be identified from an outbound calling device, the emotion recognition device preprocesses the data. When the data is a voice message, the device first converts it into corresponding text data and then processes the text using a language processing model to obtain a word matrix. The word matrix includes word vectors for each word in the data, and each word vector contains the encoding information for that word. When the data is a text message, the device directly processes the text message using a language processing model to obtain the word matrix.

[0134] Considering that the dimension of the word matrix will change with the number of words in the data to be identified, the emotion recognition device sets a maximum number of words (denoted by t) for the data to be identified, and fixes the dimension of the word matrix. When the number of words in the text is less than t, the missing word vectors in the word matrix are supplemented with zero vectors.

[0135] In other words, the word matrix can be represented by A1(dхt).

[0136] A1 = [H] n ] = [H1 H2 … H t-1 H t ]

[0137] Among them, H n Let be the column vector (dх1) of the word matrix A1, representing the word vector of the nth word in the data to be identified, where n equals 1, 2, ..., t-1, t. d represents the dimension of the word vector output by the language processing model, and both t and d are positive integers greater than 1.

[0138] Optionally, the language processing model can be a bi-directional long short-term memory (Bi-LSTM) neural network model, a bi-directional encoder representation from transformers (BERT) model, or a lite bi-directional encoder representation from transformers (ALBERT) model.

[0139] Step S202: The emotion recognition device obtains the first distance matrix based on the data to be recognized.

[0140] Specifically, the emotion recognition device obtains the first distance matrix based on the word matrix corresponding to the data to be recognized and the emotion tag matrix corresponding to the preset emotion tags.

[0141] The preset emotion labels can represent specific emotion types, such as "happy", "sad", "calm" and "angry", or they can represent abstract emotion types, such as "positive", "negative" and "neutral".

[0142] For example, when an emotion label represents a specific emotion type, the emotion label vector can be obtained by predefining it through developers and training it with a label training model; when an emotion label represents an abstract emotion type, the emotion label vector can be obtained from the information attached to the emotion label. For instance, when the emotion label is "positive," its attached information is "has a positive meaning," then the vector representation corresponding to "has a positive meaning" can be used as the emotion label vector corresponding to the emotion label "positive."

[0143] It should be noted that the emotion tag vector can be generated by the emotion recognition device, or it can be output to the emotion recognition device and stored by other tag generation devices.

[0144] In one alternative embodiment, the emotion recognition device can obtain an emotion tag matrix based on the emotion tag vectors, wherein the emotion tag matrix contains the emotion tag vector corresponding to each emotion tag.

[0145] For example, the sentiment label matrix can be represented by A2(dхk).

[0146] A2 = [C m ] = [C1 C2 … C k-1 C k ]

[0147] Among them, C m Let (dх1) be the column vector of the sentiment label matrix A2, representing k sentiment label vectors, where m equals 1, 2, ..., k-1, k. d is the dimension of the sentiment label vector (the same as the dimension of the word vectors in the word matrix A1).

[0148] Furthermore, the emotion recognition device can obtain a first distance matrix based on the word matrix A1 and the emotion tag matrix A2.

[0149] For ease of understanding, examples are provided below. Please refer to [link / reference]. Figure 3 , Figure 3 This is a schematic diagram of an emotion recognition algorithm provided in an embodiment of this application. Figure 3 As shown, the emotion recognition device obtains the first distance matrix D1 (15x4) based on the word matrix A1 (3x5) corresponding to the data to be recognized, "I feel very angry now," and the emotion tag matrix A2 (3x4) corresponding to the preset emotion tags ("calm," "happy," "angry," and "sad").

[0150]

[0151] Where n equals 1, 2, 3, 4, 5; m equals 1, 2, 3, 4.

[0152] The first distance matrix D1 includes the difference vector between the nth word vector in the word matrix A1 and the mth sentiment label vector in the sentiment label matrix A2, i.e., word vector H. n With the emotional tag C m The distance difference between vectors. For example, the difference vector H1-C1(3х1) represents the distance difference between the word vector corresponding to the word "I" and the sentiment tag vector corresponding to the sentiment tag "calm".

[0153] The difference vector contains the angle and distance information between the word vector and the sentiment tag vector. In other words, the first distance matrix contains rich fusion information between the data to be identified and the preset sentiment tags. The richer this fusion information, the richer the information contained in the feature matrix obtained by subsequent feature extraction on the first matrix, and the more accurate the final sentiment recognition result of the data to be identified.

[0154] It should be noted that, Figure 3 In a corresponding embodiment, the maximum number of words in the emotion recognition device for the number of words in the data to be recognized is set to 5.

[0155] Step S203: The emotion recognition device obtains the first feature matrix and the second feature matrix based on the first distance matrix.

[0156] Specifically, the difference vectors along the column directions of the first distance matrix can represent the difference information of different words in the data to be identified for the same sentiment label. Considering that different words have different levels of importance for the same sentiment label, and that the importance of the difference information of different words for the same sentiment label also varies, the sentiment recognition device can preset a first attention matrix.

[0157] For ease of understanding, examples are provided below. Please refer to [link / reference]. Figure 3 The first attention matrix can be represented by F1(5x4):

[0158]

[0159] The attention parameter f in the first attention matrix F1 nm It can represent the word vector H in the word matrix A1 n The sentiment label vector C in the relative sentiment label matrix A2 m The importance of.

[0160] It should be noted that emotion recognition devices can calculate the attention parameter f using the first normalization exponent Softmax function. nm :

[0161]

[0162] Where tanh represents the hyperbolic tangent function, e is the natural constant in mathematics, and the superscript T represents the transpose of the matrix. (3х3), B 1m (3х1) (1x3) is the emotion label vector C preset by the emotion recognition device to obtain the first attention matrix. m The weight matrix should be noted that different C m The weight matrices are different.

[0163] Emotion recognition devices can perform a dot product of the first attention matrix and the first distance matrix to obtain a second distance matrix, which can be represented by the second distance matrix D2(15x4):

[0164]

[0165] It is easy to understand that the vector (3x1) in the column direction of the second distance matrix D2 can represent the difference information of different words for the same sentiment label that have been fused with attention features.

[0166] Furthermore, the emotion recognition device sums the vectors along the column directions of the second distance matrix to obtain the third distance matrix D3(3х4):

[0167] D3 = [V m ] = [V1 V2 V3 V4]

[0168]

[0169]

[0170]

[0171]

[0172] The column vector V in the third distance matrix D3 m (3х1) can represent the difference information of the data to be identified for each sentiment label, which incorporates attention features.

[0173] The emotion recognition device first bases its data on the column vectors V in the third distance matrix D3, using the L2 norm. m Perform regularization (denoted by ||||2), and then apply the regularized column vector ||V m The ||2 is concatenated, and then the concatenated column vectors are regularized based on the L2 norm to obtain the regularization matrix G(12х1).

[0174] G = [||L||2]

[0175]

[0176] The symbol ⊕ represents the concatenation of vectors.

[0177] It should be noted that, Figure 3 In the corresponding embodiment, column vector V m The dimension is 3x1, while in practical applications the column vector V m The dimension of V may be much greater than 3x1, therefore, to prevent the column vector V from m The excessively large dimension after concatenation can cause gradient explosion in subsequent computations. Emotion recognition devices, on the other hand, rely on the L2 norm to pair column vectors V. m The regularization matrix G is obtained by performing a second regularization process.

[0178] Next, the emotion recognition device extracts features from the regularization matrix G to obtain the first feature matrix P1(1х3).

[0179]

[0180] in, (12х3) and B V (1х3) is the weight matrix preset for the regularization matrix G by the emotion recognition device.

[0181] Therefore, the first feature matrix can be used to indicate the difference features of the data to be identified for each of the preset sentiment labels.

[0182] Since the difference vector in the row direction of the first distance matrix can represent the difference information of different sentiment labels for the same word, and considering that different words have corresponding sentiment tendencies, that is, different sentiment labels have different importance for the same word, and the difference information of different sentiment labels for the same word also has different importance, the sentiment recognition device can preset a second attention matrix.

[0183] For ease of understanding, examples are provided below. Please refer to [link / reference]. Figure 3 The second attention matrix can be represented by F2(5x4):

[0184]

[0185] The attention parameter f in the second attention matrix F2 n ′ m This can represent the sentiment label vectors in the sentiment label matrix A2 for C. m Word vectors H in word matrix A1 n The importance of.

[0186] It should be noted that emotion recognition devices can calculate the attention parameter f using the second Softmax function. n ′ m:

[0187]

[0188] in, (3х3), B 2m (3х1) (1x3) is the preset emotion tag vector C of the emotion recognition device. m The weight matrix should be noted that different C m The weight matrices are different.

[0189] Emotion recognition devices can perform a dot product between the second attention matrix and the first distance matrix to obtain a fourth distance matrix, which can be represented by matrix D4(15x4):

[0190]

[0191] It is easy to understand that the vector (3х1) in the row direction of the fourth distance matrix D4 can represent the difference information of different sentiment labels for the same word that incorporate attention features.

[0192] Furthermore, the emotion recognition device concatenates the vectors along the rows of the fourth distance matrix to obtain the fifth distance matrix D5(12x5):

[0193]

[0194]

[0195]

[0196]

[0197]

[0198]

[0199] Furthermore, considering that the emotional connotations of different words have varying degrees of importance to the emotional content of the data to be identified, the emotion recognition device can use the third Softmax function based on the fifth distance matrix D5 to calculate the attention feature parameter 'a' for each word vector in relation to the data to be identified. n The third attention matrix F3(1x5) is obtained:

[0200] F3 = [a n ]=[a1a2a3a4a5]

[0201]

[0202] in, (12х12), B3m (12х1) (1x12) is the weight matrix preset by the emotion recognition device for the fifth distance matrix.

[0203] Furthermore, the emotion recognition device can obtain the second feature matrix P2(1x3) based on the third attention matrix F3 and the word matrix A1:

[0204] P2=[a1*H1+a2*H2+a3*H3+a4*H4+a5*H5] T

[0205] Therefore, the second feature matrix can be used to indicate the degree to which the sentiment information of the word vectors contributes to the sentiment recognition result of the data to be recognized.

[0206] It should be noted that the embodiments of this application take into account both the difference features of the data to be identified for each sentiment label and the contribution of the sentiment information of each word in the data to be identified to the sentiment recognition result. Therefore, the feature information contained in the first feature matrix and the second feature matrix obtained by the embodiments of this application based on the first distance matrix is ​​more comprehensive, which is conducive to improving the accuracy of the subsequent sentiment recognition result obtained based on the feature information.

[0207] Step S204: The emotion recognition device outputs the emotion recognition result of the data to be recognized based on the first feature matrix and the second feature matrix.

[0208] The emotion recognition result is used to represent the emotion type of the data to be identified. Optionally, the emotion recognition result can be an emotion recognition matrix or a result that directly represents the emotion type of the data to be identified.

[0209] For ease of understanding, the following explanation uses the emotion recognition matrix as an example:

[0210] like Figure 3 As shown, the emotion recognition device concatenates the first feature matrix and the second feature matrix to obtain the first fused feature matrix Q1(1x6), and then linearly maps the first fused feature matrix to the second fused feature matrix Q2(1x3). That is, it further extracts features from the first fused feature matrix and reduces its dimensionality to the second fused feature matrix. The relevant algorithm is as follows:

[0211]

[0212] Right now:

[0213] Q2 = [q m ] = [q1 q2 q3 q4]

[0214] Wherein, W4(6х3) is the weight matrix preset by the emotion recognition device for the first fusion feature matrix Q1.

[0215] Furthermore, the emotion recognition device can process the second fusion feature matrix Q2 using the fourth softmax function to obtain the emotion recognition matrix Q3(1x4):

[0216] Q3 = [y m ] = [y1 y2 y3 y4]

[0217]

[0218] It should be noted that the emotion recognition matrix Q3 includes probability information for the emotion types of the data to be identified, namely "calm", "happy", "angry" and "sad".

[0219] For example, the emotion recognition matrix Q3 corresponding to the data to be identified, "I feel very angry now," is [0.0, 0.0, 0.9, 0.1]. This indicates that the probability of the emotion being "calm" is 0.0, the probability of being "happy" is 0.0, the probability of being "angry" is 0.9, and the probability of being "sad" is 0.1. Therefore, it can be seen from the emotion recognition matrix Q3 that the emotion of the data to be identified is "angry."

[0220] Finally, the emotion recognition device can send the emotion recognition results corresponding to the data to be recognized to the outbound calling device, so that the intelligent agent can select the response script in a timely manner according to the emotion type of the data to be recognized, thereby improving customer satisfaction.

[0221] It should be noted that the dimensions of the multiple matrices mentioned in steps S201-S204 above are only examples and are not strictly limited. For example, the numbers (i.e., dimensions) in matrix A1 (3x5) and matrix A2 (3x4) can be set to other values ​​according to the actual application scenario and requirements.

[0222] Furthermore, the parameters used by the emotion recognition device in steps S201-S204 above during the output of the emotion recognition matrix (e.g., weight parameters in multiple weight matrices) can be parameters determined by the emotion recognition device through training multiple training data with training emotion labels. For better understanding, please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a flowchart illustrating another emotion recognition method provided in the embodiments of this application.

[0223] S401: The emotion recognition device acquires training data.

[0224] The training data can be obtained from other devices (which may be...) Figure 2In the corresponding embodiment, the outbound call device sends the message to the emotion recognition device, or it can be input to the emotion recognition device by copy transmission.

[0225] Specifically, the training data corresponds to training sentiment labels, which are used to represent the true sentiment type of the training data. This training data can be either audio or text information. The following explanation uses text information as an example.

[0226] The emotion recognition device first preprocesses the training data using a language processing model to obtain a word matrix. The word matrix includes word vectors for each word in the training data, and each word vector contains the encoded information for that word. Optionally, the language processing model can be a Bi-LSTM model, a BERT model, or an ALBERT model.

[0227] Considering that the dimension of the word matrix will change with the number of words in the training data, the sentiment recognition device sets a maximum number of words for the training data and fixes the dimension of the word matrix. When the number of words in the text is less than the maximum number of words, the missing word vectors in the word matrix are supplemented with zero vectors.

[0228] S402: The emotion recognition device obtains the first distance matrix based on the training data.

[0229] The emotion recognition device can obtain the first distance matrix based on the word matrix corresponding to the training data and the emotion tag matrix corresponding to the preset emotion tags.

[0230] Specifically, the sentiment label matrix includes a sentiment label vector corresponding to each sentiment label; the first distance matrix includes a difference vector, which is the distance difference between the word vector and the sentiment label vector. The difference vector contains the angle and distance information between the word vector and the sentiment label vector; that is, the first distance matrix contains rich fusion information between the training data and the preset sentiment labels. The richer this fusion information, the richer the information contained in the feature matrix obtained by subsequent feature extraction from the first matrix, and the more accurate the final sentiment recognition result of the training data.

[0231] S403: The emotion recognition device obtains the first feature matrix and the second feature matrix based on the first distance matrix.

[0232] Since the column-wise difference vectors in the first distance matrix can represent the difference information of different words in the training data for the same sentiment label, and considering that different words have different levels of importance for the same sentiment label, and that the importance of the difference information of different words for the same sentiment label also varies, the sentiment recognition device can preset a first attention matrix. The attention parameters in the first attention matrix can represent the importance of the word vectors in the word matrix relative to the sentiment label vectors in the sentiment label matrix.

[0233] The emotion recognition device performs a dot product between the first attention matrix and the first distance matrix to obtain a second distance matrix. The column vectors in the second distance matrix represent the difference information between different words that have incorporated attention features for the same emotion label. Next, the emotion recognition device sums the column vectors of the second distance matrix to obtain a third distance matrix. The column vectors in the third distance matrix represent the difference information between the training data (incorporating attention features) and each emotion label.

[0234] Furthermore, the emotion recognition device can regularize the column vectors in the third distance matrix based on the L2 norm, then concatenate the regularized column vectors, and finally regularize the concatenated column vectors as a whole based on the L2 norm to obtain a regularization matrix. The emotion recognition device can then extract features from the regularization matrix to obtain a first feature matrix. The first feature matrix is ​​used to represent the difference features of the training data for each of the preset emotion labels.

[0235] Since the difference vectors along the rows in the first distance matrix can represent the difference information of different sentiment labels for the same word, and considering that different words have corresponding sentiment tendencies (i.e., different sentiment labels have different levels of importance for the same word, and the difference information of different sentiment labels for the same word also has different levels of importance), the sentiment recognition device can preset a second attention matrix. The attention parameters in the second attention matrix can represent the importance of the sentiment label vectors in the sentiment label matrix to the word vectors in the word matrix.

[0236] Next, the emotion recognition device performs a dot product between the second attention matrix and the first distance matrix to obtain a fourth distance matrix. The vectors along the rows of the fourth distance matrix represent the difference information between different emotion tags that incorporate attention features for the same word. The emotion recognition device then concatenates the vectors along the rows of the fourth distance matrix to obtain a fifth distance matrix.

[0237] Considering that the emotional connotations of different words have varying degrees of importance to the emotional state of the training data, the emotion recognition device calculates the attention feature parameters of each word vector relative to the training data based on the fifth distance matrix, resulting in the third attention matrix. Then, the emotion recognition device uses the third attention matrix and the word matrix to obtain the second feature matrix. The second feature matrix indicates the degree to which the emotional information of the word vectors contributes to the emotion recognition results of the training data.

[0238] S404: The emotion recognition device outputs the emotion recognition result of the training data based on the first feature matrix and the second feature matrix.

[0239] The emotion recognition result is used to represent the emotion type of the training data. Optionally, the emotion recognition result can be an emotion recognition matrix or a result that directly represents the emotion type of the training data.

[0240] For ease of understanding, the following explanation uses the emotion recognition matrix as an example. The emotion recognition device concatenates the first feature matrix and the second feature matrix to obtain the first fused feature matrix. Then, it linearly maps the first fused feature matrix to the second fused feature matrix; that is, it further extracts features from the first fused feature matrix and reduces its dimensionality to the second fused feature matrix. Further, the emotion recognition device processes the second fused feature matrix to obtain the emotion recognition matrix. The emotion recognition matrix reflects the probability information of the recognized emotions in the training data, which are represented by preset emotion labels.

[0241] It should be noted that in practical applications, in order to better optimize the multiple parameters involved in the above steps S401-S404 (such as the attention feature parameters in the first attention parameter matrix), the emotion recognition device often uses a large amount of training data for model training. For each of the multiple training data, the same operation as the above steps S401-S404 needs to be performed.

[0242] S405: The emotion recognition device determines the parameters used in obtaining the emotion recognition result based on the emotion recognition result and the trained emotion labels.

[0243] The emotion recognition matrix can distinguish the emotion type of the training data, that is, the emotion recognition matrix is ​​separable. The greater the separability, the more accurate the emotion recognition result.

[0244] To evaluate the separability of the emotion recognition matrix, the emotion recognition device converts the training emotion labels into a training emotion matrix. The training emotion matrix reflects the probability information of the actual emotions represented by the preset emotion labels in the training data. Considering that the first and second feature matrices can also be used as classifiable feature matrices and their separability significantly affects the separability of the emotion recognition matrix, the emotion recognition device linearly maps the first and second feature matrices to a first probability matrix and a second probability matrix respectively using a fourth softmax function. That is, the first and second probability matrices can also reflect the emotion categories of the training data to a certain extent. Considering that the separability of the emotion label matrix also affects the separability of the emotion recognition matrix, the emotion recognition device linearly maps the emotion label matrix to a third probability matrix using a fourth softmax function.

[0245] It should be noted that the dimensions of the training sentiment matrix, the first probability matrix, the second probability matrix, and the third probability matrix are consistent with the dimension (1×m) of the sentiment recognition matrix, where m is the number of preset sentiment tags.

[0246] In one optional embodiment, the position of the element with the largest value in the training sentiment matrix is ​​designated as y0; the position of the element with the largest value in the sentiment recognition matrix is ​​designated as y1; the position of the element with the largest value in the first probability matrix is ​​designated as y2; the position of the element with the largest value in the second probability matrix is ​​designated as y3; and the position of the element with the largest value in the third probability matrix is ​​designated as y4. For example, if the training sentiment matrix is ​​[0, 0, 1, 0], then y0 is 3.

[0247] Furthermore, the emotion recognition device uses the cross-entropy loss function to compare the difference between the emotion recognition matrix and the training emotion matrix, obtaining the first loss value L1:

[0248]

[0249] Specifically, z represents the number of data points to be trained, and y... 0i Let y0 be the i-th training data point. 1i Let y1 be the i-th training data; i equals 1, 2, ..., z-1, z.

[0250] Similarly, the emotion recognition device uses the cross-entropy loss function to compare the difference between the first feature matrix and the trained emotion matrix to obtain the second loss value L2:

[0251]

[0252] Specifically, y 0i Let y0 be the i-th training data point. 2iLet y1 be the ith training data.

[0253] The emotion recognition device uses the cross-entropy loss function to compare the difference between the second feature matrix and the trained emotion matrix to obtain a third loss value L3:

[0254]

[0255] Specifically, y 0i Let y0 be the i-th training data point. 3i Let y3 be the ith training data.

[0256] This emotion recognition device uses the cross-entropy loss function to compare the difference between the emotion label feature matrix and the trained emotion matrix, and obtains a fourth loss value L4:

[0257]

[0258] Specifically, y 0i Let y0 be the i-th training data point. 4i Let y4 be the ith training data.

[0259] It should be noted that the smaller the loss value, the stronger the separability of the matrix.

[0260] Furthermore, the emotion recognition device can calculate a fifth loss value L5 based on the first loss value, the second loss value, the third loss value, and the fourth loss value:

[0261] L5 = α*L1 + β*L2 + γ*L3 + ε*L4

[0262] Specifically, α, β, γ, and ε are the preset weights (also known as hyperparameters) corresponding to each loss value. It should be noted that to ensure the sentiment label matrix is ​​in the same feature space as the first or second feature matrix, the weight ε is the same as the weight β or weight γ. For example, weight α can be equal to 0.5, weight β can be equal to 0.2, and weights γ and ε can be equal to 0.3.

[0263] The emotion recognition device continuously optimizes the parameters used in the process of obtaining the emotion recognition matrix, such as the attention parameters in the first attention matrix, until the result of the fifth loss value is less than the preset threshold, that is, until the similarity between the emotion recognition result and the training emotion label reaches the preset effect.

[0264] Optionally, the preset threshold can be the default threshold set by the emotion recognition device, or it can be a threshold that is changed according to the requirements for the accuracy of the emotion recognition results.

[0265] Finally, the emotion recognition device saves multiple optimized parameters to facilitate subsequent emotion recognition of the data to be identified.

[0266] In addition, in this embodiment of the application, the parameters in steps S401-S405 above can be pre-trained parameters obtained directly from other training devices, and the training steps performed by the training device are the same as those in steps S401-S405 above.

[0267] Through the embodiments of this application, the emotion recognition device can receive data to be recognized sent by an outbound calling device, and accurately identify the emotion type of the data to be recognized through the rich feature information contained in the first feature matrix and the second feature matrix, which facilitates subsequent applications.

[0268] The methods of the embodiments of this application have been described in detail above. In order to facilitate better implementation of the above solutions of the embodiments of this application, the apparatus of the embodiments of this application is provided below.

[0269] It is understood that the apparatus provided in the embodiments of this application, such as an emotion recognition device, includes hardware structures, software modules, or combinations of hardware structures and software structures to perform the functions described in the above method embodiments in order to achieve the functions.

[0270] Those skilled in the art will readily recognize that, based on the units and steps described in conjunction with the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can implement the foregoing method embodiments using different device implementations in different usage scenarios, and such different device implementations should not be considered beyond the scope of the embodiments of this application.

[0271] This application embodiment can divide the device into functional modules. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one functional module. The integrated module can be implemented in hardware or software. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation. For example, taking the case of dividing the device into functional modules through integration as an example, this application illustrates several possible processing devices.

[0272] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an emotion recognition device 50 provided in an embodiment of this application. The emotion recognition device 50 can be... Figure 1The emotion recognition device 102 shown, or the components in the emotion recognition device 102; the emotion recognition apparatus 50 may include an acquisition unit 501, a first processing unit 502, a second processing unit 503, and an output unit 504, wherein the detailed description of each unit is as follows:

[0273] The acquisition unit 501 is used to acquire data to be identified, wherein the data to be identified is data containing user emotions;

[0274] The first processing unit 502 is configured to obtain a first distance matrix based on the data to be identified, wherein the first distance matrix includes a difference vector, the difference vector being the distance difference between a word vector and a sentiment tag vector, the word vector being the vector corresponding to the word contained in the data to be identified, and the sentiment tag vector being the vector corresponding to a preset sentiment tag;

[0275] The second processing unit 503 is used to obtain a first feature matrix and a second feature matrix based on the first distance matrix. The first feature matrix is ​​used to indicate the difference features of the data to be identified for each of the preset sentiment tags. The second feature matrix is ​​used to indicate the degree of contribution of the sentiment information of the word vector to the sentiment recognition result of the data to be identified. The sentiment information of the word vector is the fusion information of the word vector for the preset sentiment tags.

[0276] The output unit 504 is used to output the emotion recognition result of the data to be identified based on the first feature matrix and the second feature matrix.

[0277] In one possible implementation, regarding the obtaining of the first feature matrix and the second feature matrix based on the first distance matrix, the second processing unit 503 is specifically configured to:

[0278] A second distance matrix is ​​obtained based on the first distance matrix and the first attention matrix, wherein the first attention matrix includes attention feature parameters of the word vector relative to the sentiment tag vector;

[0279] The third distance matrix is ​​processed to obtain the first feature matrix, wherein the third distance matrix is ​​determined based on the vectors in the column directions of the second distance matrix;

[0280] A fourth distance matrix is ​​obtained based on the first distance matrix and the second attention matrix, wherein the second attention matrix includes attention feature parameters of the sentiment tag vector relative to the word vector;

[0281] The second feature matrix is ​​obtained based on the fifth distance matrix and the word matrix containing the word vector, wherein the fifth distance matrix is ​​determined based on the vectors in the row direction of the fourth distance matrix.

[0282] In another possible implementation, regarding the processing of the third distance matrix to obtain the first feature matrix, the second processing unit 503 is specifically used for:

[0283] The third distance matrix is ​​regularized to obtain a regularized matrix;

[0284] The regularization matrix is ​​subjected to feature extraction to obtain the first feature matrix.

[0285] In another possible implementation, regarding the process of obtaining the second feature matrix based on the fifth distance matrix and the word matrix containing the word vectors, the second processing unit 503 is specifically configured to:

[0286] The third attention matrix is ​​obtained based on the fifth distance matrix, wherein the third attention matrix includes attention feature parameters of the word vector relative to the data to be identified;

[0287] The second feature matrix is ​​obtained by performing a dot product on the third attention matrix and the word matrix.

[0288] In another possible implementation, the emotion recognition result includes an emotion recognition matrix, and in the process of outputting the emotion recognition result of the data to be recognized based on the first feature matrix and the second feature matrix, the output unit 504 is specifically used for:

[0289] The first feature matrix and the second feature matrix are concatenated to obtain the first fused feature matrix;

[0290] The first fused feature matrix is ​​subjected to feature extraction to obtain the second fused feature matrix;

[0291] The emotion recognition matrix is ​​output based on the second fusion feature matrix, wherein the emotion recognition matrix is ​​used to reflect the probability information that the emotions of the identified data to be identified are the emotions corresponding to the preset emotion tags.

[0292] It should be noted that, in the embodiments of this application, the specific implementation and technical effects of each unit can also be referred to accordingly. Figure 2 The corresponding description of the illustrated embodiment.

[0293] Please see Figure 6 , Figure 6This application provides an emotion recognition device 60, which includes a processor 601, a memory 602, and a communication interface 603. The processor 601, the memory 602, and the communication interface 603 are interconnected via a bus.

[0294] Processor 601 can be one or more central processing units (CPUs). If processor 601 is a CPU, the CPU can be a single-core CPU or a multi-core CPU.

[0295] The memory 602 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), which is used for related computer programs and data.

[0296] The communication interface 603 is used to receive and send data. Optionally, the communication interface 603 receives data to be identified sent by the outbound calling device and sends the data to be identified to the processor 601; optionally, the communication interface 603 receives the emotion recognition result corresponding to the data to be identified sent by the processor 601 and sends the emotion recognition result to the outbound calling device.

[0297] Processor 601 is used to read the computer program code stored in memory 602 and perform the following operations:

[0298] Acquire data to be identified, wherein the data to be identified is data containing user emotions;

[0299] A first distance matrix is ​​obtained based on the data to be identified, wherein the first distance matrix includes a difference vector, the difference vector being the distance difference between word vectors and sentiment tag vectors, the word vectors being the vectors corresponding to the words contained in the data to be identified, and the sentiment tag vectors being the vectors corresponding to preset sentiment tags;

[0300] A first feature matrix and a second feature matrix are obtained based on the first distance matrix. The first feature matrix is ​​used to indicate the difference features of the data to be identified for each of the preset sentiment tags. The second feature matrix is ​​used to indicate the degree of contribution of the sentiment information of the word vector to the sentiment recognition result of the data to be identified. The sentiment information of the word vector is the fusion information of the word vector for the preset sentiment tags.

[0301] The emotion recognition result of the data to be identified is output based on the first feature matrix and the second feature matrix.

[0302] In one possible implementation, in obtaining the first feature matrix and the second feature matrix based on the first distance matrix, the processor 601 is specifically configured to:

[0303] A second distance matrix is ​​obtained based on the first distance matrix and the first attention matrix, wherein the first attention matrix includes attention feature parameters of the word vector relative to the sentiment tag vector;

[0304] The third distance matrix is ​​processed to obtain the first feature matrix, wherein the third distance matrix is ​​determined based on the vectors in the column directions of the second distance matrix;

[0305] A fourth distance matrix is ​​obtained based on the first distance matrix and the second attention matrix, wherein the second attention matrix includes attention feature parameters of the sentiment tag vector relative to the word vector;

[0306] The second feature matrix is ​​obtained based on the fifth distance matrix and the word matrix containing the word vector, wherein the fifth distance matrix is ​​determined based on the vectors in the row direction of the fourth distance matrix.

[0307] In another possible implementation, regarding the processing of the third distance matrix to obtain the first feature matrix, the processor 601 is specifically configured to:

[0308] The third distance matrix is ​​regularized to obtain a regularized matrix;

[0309] The regularization matrix is ​​subjected to feature extraction to obtain the first feature matrix.

[0310] In yet another possible implementation, in obtaining the second feature matrix based on the fifth distance matrix and the word matrix containing the word vectors, the processor 601 is specifically configured to:

[0311] The third attention matrix is ​​obtained based on the fifth distance matrix, wherein the third attention matrix includes attention feature parameters of the word vector relative to the data to be identified;

[0312] The second feature matrix is ​​obtained by performing a dot product on the third attention matrix and the word matrix.

[0313] In yet another possible implementation, the emotion recognition result includes an emotion recognition matrix, and in terms of outputting the emotion recognition result of the data to be recognized based on the first feature matrix and the second feature matrix, the processor 601 is specifically configured to:

[0314] The first feature matrix and the second feature matrix are concatenated to obtain the first fused feature matrix;

[0315] The first fused feature matrix is ​​subjected to feature extraction to obtain the second fused feature matrix;

[0316] The emotion recognition matrix is ​​output based on the second fusion feature matrix, wherein the emotion recognition matrix is ​​used to reflect the probability information that the emotions of the identified data to be identified are the emotions corresponding to the preset emotion tags.

[0317] It should be noted that the implementation of each operation can also be referenced accordingly. Figure 2 The corresponding description of the illustrated embodiment.

[0318] This application also provides a computer-readable storage medium storing a computer program that, when run on a network device,... Figure 2 or Figure 4 The method flow shown is thus implemented.

[0319] In this application, the term "multiple" in the embodiments refers to two or more objects; "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural; the character " / " generally indicates that the preceding and following associated objects are in an "or" relationship. Furthermore, unless otherwise stated, the term "first" in the embodiments of this application is only used for name identification and is not used to limit the order, sequence, priority, or importance of multiple objects, such as a first distance matrix, a first feature matrix, and a first fusion feature matrix. This rule also applies to "second," "third," and "fourth," etc.

[0320] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An emotion recognition method, characterized by, The method comprises: acquiring to-be-identified data, wherein the to-be-identified data is data containing user emotion; obtaining a first distance matrix according to the to-be-identified data, wherein the first distance matrix comprises a difference vector, the difference vector being a distance difference between a word vector and an emotion label vector, the word vector being a vector corresponding to a word contained in the to-be-identified data, and the emotion label vector being a vector corresponding to a preset emotion label; obtaining a first feature matrix and a second feature matrix according to the first distance matrix, wherein the first feature matrix is used to indicate a difference feature of the to-be-identified data with respect to each of the preset emotion labels, and the second feature matrix is used to indicate a contribution degree of emotion information of the word vector to an emotion recognition result of the to-be-identified data, the emotion information of the word vector being fusion information of the word vector with respect to the preset emotion labels; outputting the emotion recognition result of the to-be-identified data according to the first feature matrix and the second feature matrix; the obtaining of the first feature matrix and the second feature matrix according to the first distance matrix comprises: obtaining a second distance matrix according to the first distance matrix and a first attention matrix, wherein the first attention matrix comprises an attention feature parameter of the word vector with respect to the emotion label vector; processing a third distance matrix to obtain the first feature matrix, wherein the third distance matrix is determined according to a vector in a column direction of the second distance matrix; obtaining a fourth distance matrix according to the first distance matrix and a second attention matrix, wherein the second attention matrix comprises an attention feature parameter of the emotion label vector with respect to the word vector; obtaining the second feature matrix according to a fifth distance matrix and a word matrix in which the word vector is located, wherein the fifth distance matrix is determined according to a vector in a row direction of the fourth distance matrix.

2. The method of claim 1, wherein, the processing of the third distance matrix to obtain the first feature matrix comprises: regularizing the third distance matrix to obtain a regularized matrix; extracting features of the regularized matrix to obtain the first feature matrix.

3. The method of claim 1, wherein, the obtaining of the second feature matrix according to the fifth distance matrix and the word matrix in which the word vector is located comprises: obtaining a third attention matrix according to the fifth distance matrix, wherein the third attention matrix comprises an attention feature parameter of the word vector with respect to the to-be-identified data; point-multiplying the third attention matrix and the word matrix to obtain the second feature matrix.

4. The method according to any one of claims 1 to 3, characterized in that, the emotion recognition result comprises an emotion recognition matrix, and the outputting of the emotion recognition result of the to-be-identified data according to the first feature matrix and the second feature matrix comprises: splicing the first feature matrix and the second feature matrix to obtain a first fusion feature matrix; extracting features of the first fusion feature matrix to obtain a second fusion feature matrix; Output the sentiment recognition matrix according to the second fusion feature matrix, wherein the sentiment recognition matrix is used to reflect probability information that the recognized sentiment of the to-be-recognized data is the sentiment corresponding to the preset sentiment label.

5. An emotion recognition method characterized by, The method comprises: Obtaining to-be-trained data, wherein the to-be-trained data is data corresponding to a training sentiment label; Obtaining a first distance matrix according to the to-be-trained data, wherein the first distance matrix comprises a difference vector, the difference vector is a distance difference between a word vector and a sentiment label vector, the word vector is a vector corresponding to a word contained in the to-be-trained data, and the sentiment label vector is a vector corresponding to a preset sentiment label; Processing according to the first distance matrix to obtain a first feature matrix and a second feature matrix, wherein the first feature matrix is used to indicate a difference feature of the to-be-trained data for each sentiment label in the preset sentiment labels, and the second feature matrix is used to indicate a contribution degree of sentiment information of the word vector to a sentiment recognition result of the to-be-trained data, the sentiment information of the word vector being fusion information of the word vector for the preset sentiment labels; Outputting the sentiment recognition result of the to-be-trained data according to the first feature matrix and the second feature matrix; Determining parameters used in the process of obtaining the sentiment recognition result according to the sentiment recognition result and the training sentiment label; The first feature matrix and the second feature matrix obtained according to the first distance matrix comprise: Obtaining a second distance matrix according to the first distance matrix and a first attention matrix, wherein the first attention matrix comprises attention feature parameters of the word vector relative to the sentiment label vector; Processing a third distance matrix to obtain the first feature matrix, wherein the third distance matrix is determined according to vectors in a column direction of the second distance matrix; Obtaining a fourth distance matrix according to the first distance matrix and a second attention matrix, wherein the second attention matrix comprises attention feature parameters of the sentiment label vector relative to the word vector; Obtaining the second feature matrix according to a fifth distance matrix and a word matrix in which the word vector is located, wherein the fifth distance matrix is determined according to vectors in a row direction of the fourth distance matrix.

6. The method of claim 5, wherein: The sentiment recognition result comprises a sentiment recognition matrix, which is used to reflect probability information that the recognized sentiment of the to-be-trained data is the sentiment represented by the preset sentiment label; The training sentiment label comprises a training sentiment matrix, which is used to reflect probability information that the true sentiment of the to-be-trained data is the sentiment represented by the preset sentiment label. The parameters used in the process of obtaining the emotion recognition result are determined according to the emotion recognition result and the training emotion label, including determining a first loss value, a second loss value, a third loss value and a fourth loss value according to the emotion recognition matrix and the training emotion matrix, the first loss value being a loss value between the emotion recognition matrix and the training emotion matrix calculated by a first loss function, the second loss value being a loss value between the first feature matrix and the training emotion matrix calculated by a second loss function, the third loss value being a loss value between the second feature matrix and the training emotion matrix calculated by a third loss function, and the fourth loss value being a loss value between the emotion label matrix where the emotion label vector is located and the training emotion matrix calculated by a fourth loss function. In a case where the fifth loss value is less than a preset threshold, the parameters used in the process of obtaining the emotion recognition result are determined according to the first loss value, the second loss value, the third loss value and the fourth loss value, wherein the fifth loss value is related to the first loss value, the second loss value, the third loss value and the fourth loss value.

7. An emotion recognition apparatus, characterized by, The device comprises: an acquisition unit configured to acquire to-be-recognized data, wherein the to-be-recognized data is data containing user emotion; a first processing unit configured to obtain a first distance matrix according to the to-be-recognized data, wherein the first distance matrix comprises a difference vector, the difference vector being a distance difference between a word vector and an emotion label vector, the word vector being a vector corresponding to a word contained in the to-be-recognized data, and the emotion label vector being a vector corresponding to a preset emotion label; a second processing unit configured to obtain a first feature matrix and a second feature matrix according to the first distance matrix, wherein the first feature matrix is used to indicate a difference feature of the to-be-recognized data with respect to each of the preset emotion labels, and the second feature matrix is used to indicate a contribution degree of emotion information of the word vector to an emotion recognition result of the to-be-recognized data, the emotion information of the word vector being fusion information of the word vector with respect to the preset emotion labels; an output unit configured to output the emotion recognition result of the to-be-recognized data according to the first feature matrix and the second feature matrix; In the aspect of obtaining the first feature matrix and the second feature matrix according to the first distance matrix, the second processing unit is specifically configured to: obtain a second distance matrix according to the first distance matrix and a first attention matrix, wherein the first attention matrix comprises attention feature parameters of the word vector with respect to the emotion label vector; process a third distance matrix to obtain the first feature matrix, wherein the third distance matrix is determined according to vectors in a column direction of the second distance matrix; obtain a fourth distance matrix according to the first distance matrix and a second attention matrix, wherein the second attention matrix comprises attention feature parameters of the emotion label vector with respect to the word vector; and process a fifth distance matrix to obtain the second feature matrix, wherein the fifth distance matrix is determined according to vectors in a row direction of the fourth distance matrix. The second feature matrix is obtained according to a fifth distance matrix and a word matrix in which the word vectors are located, wherein the fifth distance matrix is determined according to vectors in a row direction of the fourth distance matrix.

8. An affect recognition device, characterized by, The device comprises a processor, a memory and a communication interface, wherein the communication interface is configured to perform receiving and / or sending operations under control of the processor, the memory is configured to store a computer program, and the processor is configured to invoke the computer program to implement the method in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and when the computer program runs on the processor, the method in any one of claims 1-6 is implemented.

Images