Conversation sample generation method, model training method, emotion recognition method and device

Abstract

The present invention provides a conversation sample generation method, a model training method, an emotion recognition method and a device, and relates to the field of artificial intelligence. The specific technical scheme of the conversation sample generation method includes: obtaining a time series conversation sequence, wherein the time series conversation sequence includes multiple rounds of conversation contents arranged in chronological order; determining a mask matrix generated for the time series conversation sequence, wherein the mask matrix includes multiple mask coefficients, and the mask coefficients have a one-to-one correspondence with the conversation content; and determining a conversation sample according to the mask matrix, the time series conversation sequence and the one-to-one correspondence.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more specifically, to a conversation sample generation method, a model training method, and an emotion recognition method and device. Background Art

[0002] Sentiment analysis in conversations can be included in multiple rounds of conversations among participants, and the emotions carried by each round of conversation can be determined, such as happiness, anger, sadness, etc. This technology can empower intelligent robots to provide more humane services.

[0003] In the process of realizing the concept of the present invention, the inventors found that there are at least the following problems in the related art: when performing emotion recognition for each round of conversation, it is difficult and the recognition accuracy is low. Summary of the invention

[0004] In view of this, the present invention provides a conversation sample generation method, a model training method, and an emotion recognition method and device.

[0005] One aspect of the present invention provides a method for generating a conversation sample, comprising: obtaining a time-series conversation sequence, wherein the time-series conversation sequence includes conversation contents of multiple rounds arranged in chronological order; determining a mask matrix generated for the time-series conversation sequence, wherein the mask matrix includes multiple mask coefficients, and the mask coefficients have a one-to-one correspondence with the conversation contents; and determining a conversation sample based on the mask matrix, the time-series conversation sequence and the one-to-one correspondence.

[0006] Another aspect of the present invention provides a method for training a conversation emotion recognition model, comprising: inputting a time-series conversation sequence into the conversation emotion recognition model to obtain a first conversation emotion prediction result of the time-series conversation sequence, wherein the time-series conversation sequence has a conversation emotion label; inputting a conversation sample into the conversation emotion recognition model to obtain a second conversation emotion prediction result of the conversation sample, wherein the conversation sample is obtained by processing the time-series conversation sequence using the conversation sample generation method described in the present invention; and training the conversation emotion recognition model according to the first conversation emotion prediction result, the second conversation emotion prediction result and the conversation emotion label to obtain a trained conversation emotion recognition model.

[0007] Another aspect of the present invention provides a conversation emotion recognition method, comprising: obtaining conversation content to be recognized and part of the conversation content located before the conversation content to be recognized; and inputting the part of the conversation content and the conversation content to be recognized into a conversation emotion recognition model to obtain a conversation emotion recognition result of the conversation content to be recognized, wherein the conversation emotion recognition model is trained using the conversation emotion recognition model training method described in the present invention.

[0008] Another aspect of the present invention provides a conversation sample generation device, including: a first acquisition module, used to acquire a time-series conversation sequence, wherein the time-series conversation sequence includes multiple rounds of conversation content arranged in chronological order; a first determination module, used to determine a mask matrix generated for the time-series conversation sequence, wherein the mask matrix includes multiple mask coefficients, and the mask coefficients have a one-to-one correspondence with the conversation content; and a second determination module, used to determine the conversation sample based on the mask matrix, the time-series conversation sequence and the one-to-one correspondence.

[0009] Another aspect of the present invention provides a training device for a conversation emotion recognition model, comprising: a first acquisition module, used to input a time-series conversation sequence into the conversation emotion recognition model to obtain a first conversation emotion prediction result of the time-series conversation sequence, wherein the time-series conversation sequence has a conversation emotion label; a second acquisition module, used to input a conversation sample into the conversation emotion recognition model to obtain a second conversation emotion prediction result of the conversation sample, wherein the conversation sample is obtained by processing the time-series conversation sequence using the conversation sample generation device described in the present invention; and a training module, used to train the conversation emotion recognition model according to the first conversation emotion prediction result, the second conversation emotion prediction result and the conversation emotion label to obtain a trained conversation emotion recognition model.

[0010] Another aspect of the present invention provides a conversation emotion recognition device, comprising: a second acquisition module, used to acquire conversation content to be recognized and part of the conversation content located before the conversation content to be recognized; and a third acquisition module, used to input the part of the conversation content and the conversation content to be recognized into a conversation emotion recognition model to obtain a conversation emotion recognition result of the conversation content to be recognized, wherein the conversation emotion recognition model is trained using the conversation emotion recognition model training device described in the present invention.

[0011] Another aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement at least one of the conversation sample generation method, conversation emotion recognition model training method, and conversation emotion recognition method described in the present invention.

[0012] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement at least one of the conversation sample generation method, conversation emotion recognition model training method, and conversation emotion recognition method described in the present invention.

[0013] Another aspect of the present invention provides a computer program product, which includes computer executable instructions, which, when executed, are used to implement at least one of the conversation sample generation method, conversation emotion recognition model training method, and conversation emotion recognition method as described above.

[0014] According to an embodiment of the present invention, a mask matrix is ​​used to perform different degrees of information masking on the conversation content in a time-series conversation sequence to obtain a technical means for obtaining enhanced conversation samples. When a model is trained based on the conversation samples, the model's effective use of the conversation content in the time-series conversation sequence can be enhanced, and the recognition accuracy of the model can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0016] Figure 1 An exemplary system architecture to which at least one of a conversation sample generation method, a conversation emotion recognition model training method, and a conversation emotion recognition method can be applied according to an embodiment of the present invention is schematically shown;

[0017] Figure 2 Schematically shows a flow chart of a method for generating a session sample according to an embodiment of the present invention;

[0018] Figure 3 A flowchart of a method for training a conversation emotion recognition model according to an embodiment of the present invention is schematically shown;

[0019] Figure 4 A schematic diagram schematically shows a training process of a pre-trained language model based on Transformer according to an embodiment of the present invention;

[0020] Figure 5A flowchart of a conversation emotion recognition method according to an embodiment of the present invention is schematically shown;

[0021] Figure 6 Schematically shows a block diagram of a conversation sample generating device according to an embodiment of the present invention;

[0022] Figure 7 A block diagram schematically shows a training device for a conversation emotion recognition model according to an embodiment of the present invention;

[0023] Figure 8 A block diagram schematically shows a conversation emotion recognition device according to an embodiment of the present invention; and

[0024] Fig. 9 A block diagram of an electronic device suitable for implementing at least one of a conversation sample generation method, a conversation emotion recognition model training method, and a conversation emotion recognition method according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0025] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0026] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0027] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0028] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0029] It should be noted that in the technical solution of the present invention, the collection, use, storage, sharing and transfer of data (for example, including but not limited to user personal information) involved shall comply with the provisions of relevant laws and regulations, and the user shall be informed and the user's consent or authorization shall be obtained. When used, the user's personal information is de-identified and / or anonymized and / or encrypted.

[0030] Sentiment analysis in conversations is different from ordinary sentiment classification tasks. It is necessary to distinguish each round of speech. Since some rounds of speech may be short sentences with low information density, such as "yes" and "good", such information-scarce sentences make it difficult to distinguish emotions. In addition, the same sentence may carry completely different emotions in different contexts. Due to these characteristics of conversations, namely the progressiveness and continuity of content, conversation emotion recognition is more difficult than traditional classification problems.

[0031] In the process of realizing the concept of the present invention, the inventors found that the context in the conversation is the key to understanding the progress of the conversation, and the conversation history plays a vital role in modeling the current discourse. Modeling in combination with the context in the conversation can not only provide certain conversation background information and a more complete information flow for the current discourse, but also fill in the missing information in the conversation to a certain extent. However, most methods directly splice the context and input it into the model for encoding. There is no guided learning process, and effective conversation context modeling methods have not been fully explored, resulting in the model attaching the same importance to all contexts, resulting in poor conversation emotion recognition effect.

[0032] The embodiment of the present invention provides a sample generation method, a model training method, an emotion recognition method and a device. The method includes obtaining a time-series conversation sequence, wherein the time-series conversation sequence includes a plurality of rounds of conversation contents arranged in chronological order; determining a mask matrix generated for the time-series conversation sequence, wherein the mask matrix includes a plurality of mask coefficients, and the mask coefficients have a one-to-one correspondence with the conversation contents; and determining a conversation sample according to the mask matrix, the time-series conversation sequence and the one-to-one correspondence.

[0033] Figure 1 The exemplary system architecture 100 is schematically shown to which at least one of the conversation sample generation method, conversation emotion recognition model training method and conversation emotion recognition method can be applied according to an embodiment of the present invention. It should be noted that: Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present invention may be applied, to help those skilled in the art understand the technical content of the present invention, but do not mean that the embodiments of the present invention may not be used in other devices, systems, environments or scenarios.

[0034] like Figure 1As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0035] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).

[0036] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0037] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0038] It should be noted that at least one of the conversation sample generation method, conversation emotion recognition model training method, and conversation emotion recognition method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, at least one of the conversation sample generation device, conversation emotion recognition model training device, and conversation emotion recognition device provided in the embodiment of the present invention can generally be set in the server 105. At least one of the conversation sample generation method, conversation emotion recognition model training method, and conversation emotion recognition method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105. Accordingly, at least one of the conversation sample generation device, conversation emotion recognition model training device, and conversation emotion recognition device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105. Alternatively, at least one of the conversation sample generation method, conversation emotion recognition model training method, and conversation emotion recognition method provided in the embodiments of the present invention may also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or may also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, at least one of the conversation sample generation device, conversation emotion recognition model training device, and conversation emotion recognition device provided in the embodiments of the present invention may also be set in the first terminal device 101, the second terminal device 102, or the third terminal device 103, or may be set in other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0039] For example, the time-series conversation sequence, the conversation emotion recognition model, the conversation content to be recognized, and the part of the conversation content located before the conversation content to be recognized may be originally stored in any one of the first terminal device 101, the second terminal device 102, or the third terminal device 103 (for example, the first terminal device 101, but not limited thereto), or stored in an external storage device and may be imported into the first terminal device 101. Then, the first terminal device 101 may locally execute at least one of the conversation sample generation method, conversation emotion recognition model training method, and conversation emotion recognition method provided in the embodiment of the present invention, or send the time-series conversation sequence, the conversation emotion recognition model, the conversation content to be recognized, and the part of the conversation content located before the conversation content to be recognized to other terminal devices, servers, or server clusters, and the other terminal devices, servers, or server clusters that receive the image to be processed may execute at least one of the conversation sample generation method, conversation emotion recognition model training method, and conversation emotion recognition method provided in the embodiment of the present invention.

[0040] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0041] Figure 2 The flowchart of the method for generating a session sample according to an embodiment of the present invention is schematically shown.

[0042] like Figure 2 As shown, the method includes operations S201 to S203.

[0043] In operation S201, a time-series conversation sequence is obtained, wherein the time-series conversation sequence includes conversation contents of multiple rounds arranged in chronological order.

[0044] According to an embodiment of the present invention, a time-series conversation sequence may include multiple continuous or discontinuous conversation contents intercepted from various conversation scenarios, as long as the conversation contents are stored in the order of the conversation generation time. A round of conversation contents may include one or more conversation contents, which is not limited here.

[0045] It should be noted that when the time-series session sequences collected contain user personal information, the data will be desensitized and de-identified through technical means.

[0046] In operation S202, a mask matrix generated for a time-series conversation sequence is determined, wherein the mask matrix includes a plurality of mask coefficients, and the mask coefficients have a one-to-one correspondence with the conversation content.

[0047] According to an embodiment of the present invention, a mask matrix may be generated according to the number of rounds of conversation content included in a time-series conversation sequence. For example, a 1×N-order mask matrix including N mask coefficients may be generated for conversation content of N rounds in a time-series conversation sequence. The obtained mask matrix may perform information masking on a round-by-round basis for conversation content of multiple rounds in a time-series conversation sequence.

[0048] According to an embodiment of the present invention, the mask coefficient may be a value greater than or equal to 0 or less than or equal to 1. The one-to-one correspondence may indicate that each conversation content corresponds to a mask coefficient.

[0049] In operation S203, a session sample is determined according to the mask matrix, the time-series session sequence, and the one-to-one correspondence.

[0050] According to an embodiment of the present invention, the obtained information-masked conversation content can be determined as a conversation sample. The conversation sample can be used as a training sample for various models that need to use conversation information as training data. In this embodiment, for example, the conversation sample can be used in combination with the emotion label to train the conversation emotion recognition model.

[0051] According to an embodiment of the present invention, the original conversation contents of multiple rounds can be first collected to obtain a time-series conversation sequence. Then, the habit of people handling conversations in daily conversations can be simulated, and a mask coefficient can be generated for the conversation contents of each round to obtain a mask matrix. After that, the original conversation contents of multiple rounds in the time-series conversation sequence can be subjected to information enhancement processing to different degrees in units of rounds according to the mask matrix in combination with a one-to-one correspondence, to obtain information-shielded enhanced conversation contents of multiple rounds, and the enhanced conversation contents of multiple rounds can be determined as conversation samples.

[0052] Through the above embodiments of the present invention, a method for constructing enhanced conversation samples is proposed, which can perform different degrees of information masking on the conversation content in the time-series conversation sequence based on the mask matrix to obtain enhanced conversation samples. When model training is performed based on the conversation samples, the model can enhance the effective use of the conversation content in the time-series conversation sequence and improve the model training accuracy.

[0053] In conjunction with specific embodiments, Figure 2 The method shown is further explained.

[0054] According to an embodiment of the present invention, before performing the aforementioned operations S201 to S203, it is necessary to first obtain a time-series conversation sequence. The method for obtaining a time-series conversation sequence may include: determining a target conversation content and a historical conversation content. The historical conversation content is located before the target conversation content. The historical conversation content includes conversation content of a preset number of rounds. Determine a time-series conversation sequence based on the historical conversation content and the target conversation content.

[0055] According to the embodiment of the present invention, the target conversation content may be any one or more conversation contents in the generated conversation. The historical conversation content may include conversation contents that are close to the target conversation content or conversation contents that are long from the target conversation content, which is not limited here.

[0056] According to an embodiment of the present invention, all conversation contents collected in the same conversation scene can be spliced ​​in sequence according to the order of conversation advancement to obtain a time-series conversation sequence. Alternatively, a preset number can be set first. Then, the target conversation content and the historical conversation contents of the preset number of rounds are spliced ​​according to the order of conversation advancement to obtain a time-series conversation sequence.

[0057] For example, we can concatenate the target conversation content with the historical conversation content of w rounds to obtain a time-series conversation sequence of w+1 rounds, where w is the number of rounds of the historical conversation content. The concatenated time-series conversation sequence is input into the pre-trained language model for encoding, and the final semantic representation can be shown in formula (1).

[0058] Formula (1)

[0059] In formula (1), The encoding result of the time-series conversation sequence of w+1 rounds can be represented. Can represent the target session content. It can represent w rounds of historical conversation content, where w can be a positive integer.

[0060] According to an embodiment of the present invention, the introduction of historical conversation content can provide information assistance for the semantic understanding of the target conversation content, but not all information is equally important for encoding the target conversation content. In this embodiment, when modeling historical conversations, multiple ways of constructing conversation samples can be designed by simulating the habits and states of people's communication in daily conversations. These ways can be reflected by controlling the mask coefficients of the mask matrix in the above operation S202.

[0061] In the first approach, when people have daily conversations, they usually have a deeper impression of the content of the conversation that just happened, but they do not remember the content of the conversation that happened a long time ago. To simulate this memory habit, a soft mask mechanism can be used to model the content of historical conversations.

[0062] According to an embodiment of the present invention, corresponding to the first method, the above operation S202 may include: for each round of conversation content, determining the round number to which the conversation content belongs in the time-series conversation sequence. Determining a mask coefficient corresponding to the conversation content according to the round number and the total round number of the time-series conversation sequence. Determining a mask matrix according to multiple mask coefficients corresponding to the conversation contents of multiple rounds.

[0063] According to an embodiment of the present invention, the round number can represent the position of the conversation content of each round in the time-series conversation sequence. For example, the round number of the conversation content located first in the time-series conversation sequence can be 1. In this case, a first ratio of the round number to the total round number of the time-series conversation sequence can be calculated, and a mask coefficient corresponding to the conversation content of the round number can be determined according to the first ratio.

[0064] According to an embodiment of the present invention, the round number can also characterize that the historical conversation content of each round in the time-series conversation sequence is located in the first few rounds relative to the target conversation content. For example, the round number of the conversation content located at the second to last position in the time-series conversation sequence can be 1. In this case, the second ratio of the round number to the total round number of the time-series conversation sequence can be calculated first. Then, the difference between 1 and the second ratio is calculated, and the mask coefficient corresponding to the conversation content of the round number is determined according to the difference.

[0065] According to an embodiment of the present invention, the mask coefficients corresponding to the conversation content of each round in the time-series conversation sequence can be determined according to the aforementioned method, and then the mask matrix generated for the time-series conversation sequence can be determined.

[0066] According to an embodiment of the present invention, the operation S203 may include: performing dot product calculations on the conversation contents of multiple rounds and multiple mask coefficients according to a one-to-one correspondence to obtain dot product calculation results. Determining conversation samples according to the click calculation results.

[0067] According to an embodiment of the present invention, each mask coefficient in the mask matrix can be used as a weight of a conversation content corresponding to the mask coefficient. The mask coefficient corresponding to each conversation content in the time-series conversation sequence is first determined from the multiple mask coefficients in the mask matrix according to a one-to-one correspondence. Then, the mask coefficients with a one-to-one correspondence are multiplied with the conversation content to perform a dot product calculation on the conversation content of multiple rounds and the multiple mask coefficients, so that information masking can be performed on the conversation content in the time-series conversation sequence in units of rounds.

[0068] For example, corresponding to the first method, we can first use different degrees of mask matrices according to the distance between the historical conversation content and the target conversation content. The closer the historical conversation content is to the target conversation content, the less information it masks. Conversely, the farther the distance is, the more information it masks. Then, based on the mask matrix, we perform information masking on the w+1 rounds of sequential conversation sequence shown in formula (1), and we can get the result shown in formula (2-1).

[0069] Formula (2-1)

[0070] In formula (2-1), The session samples may be generated based on the mask matrix determined in the first way. It can be a mask coefficient for controlling the conversation content of the nth round. Modeling based on conversation rounds can be expressed as shown in the following formula (2-2), but is not limited thereto.

[0071] Formula (2-2)

[0072] Through the above-mentioned embodiments of the present invention, a method for determining a mask coefficient is provided. The mask matrix determined based on this method can simulate the memory habit of being impressed by the content of a conversation that has just occurred, but having a vague memory of the content of a conversation that occurred a long time ago, and perform corresponding information masking on the time series conversation samples. The constructed conversation samples can enable the model to automatically capture effective conversation history information.

[0073] In the second method, people randomly forget some information during daily conversations. To mimic this forgetting mechanism, an information loss mechanism can be designed to discard information from random rounds of historical conversations, and after encoding, let the model compare it with the complete conversation encoding to find valid information.

[0074] According to an embodiment of the present invention, corresponding to the second method, the operation S202 may include: determining the forgotten probability generated for each round of conversation content to obtain multiple forgotten probabilities, and determining a mask matrix according to the multiple forgotten probabilities.

[0075] According to an embodiment of the present invention, after obtaining a plurality of forgotten probabilities, the plurality of forgotten probabilities can be sorted accordingly according to the sorting of the conversation contents in the time-series conversation sequence, and the sorted plurality of forgotten probabilities can be directly solved and determined as a mask matrix. In this case, each forgotten probability can be directly used as a mask coefficient of the corresponding conversation content in the time-series conversation sequence.

[0076] According to an embodiment of the present invention, determining the mask matrix according to multiple forgotten probabilities may further include: for each round of conversation content, in response to determining that the probability of conversation content being forgotten is greater than or equal to a preset forgetting threshold, determining the mask coefficient corresponding to the conversation content to be 0. In response to determining that the probability of conversation content being forgotten is less than the preset forgetting threshold, determining the mask coefficient corresponding to the conversation content to be 1. Determine the mask matrix according to multiple mask coefficients corresponding to the conversation contents of multiple rounds determined by 0 or 1.

[0077] For example, corresponding to the second method, a mask matrix randomly generated in units of dialogue rounds can be used to perform information masking on the w+1 rounds of temporal conversation sequence shown in formula (1), and the result shown in formula (3-1) can be obtained.

[0078] Formula (3-1)

[0079] In formula (3-1), The session samples may be generated based on the mask matrix determined in the second manner. It can be a mask coefficient for controlling the conversation content of the nth round. Modeling based on conversation rounds can be expressed as shown in the following formula (3-2), but is not limited thereto.

[0080] Formula (3-2)

[0081] In formula (3-2), It can be the forgotten probability randomly generated for the session content when constructing the mask matrix. It can be the probability that the content of this round of conversation is discarded, that is, the above-mentioned preset forgetting threshold, and its value can be a preset hyperparameter.

[0082] Through the above-mentioned embodiments of the present invention, another method for determining the mask coefficient is provided. The mask matrix determined based on this method can simulate the forgetting mechanism and perform corresponding information masking on the time series conversation samples. The constructed conversation samples can enable the model to automatically capture effective conversation history information.

[0083] In the third approach, in the course of daily conversations, some information may resonate strongly with the speaker, which is different for each speaker. To model this mechanism, the information of random rounds of historical conversations can be enhanced.

[0084] According to an embodiment of the present invention, corresponding to the second manner, the operation S202 may include: determining the enhanced probability generated for each round of conversation content to obtain multiple enhanced probabilities, and determining a mask matrix according to the multiple enhanced probabilities.

[0085] According to an embodiment of the present invention, after obtaining a plurality of enhanced probabilities, the plurality of enhanced probabilities can be sorted accordingly according to the sorting of the conversation contents in the time-series conversation sequence, and the sorted plurality of enhanced probabilities can be directly solved and determined as a mask matrix. In this case, each enhanced probability can be directly used as a mask coefficient of the corresponding conversation content in the time-series conversation sequence.

[0086] According to an embodiment of the present invention, determining a mask matrix according to multiple enhanced probabilities may further include: for each round of conversation content, in response to determining that the probability of conversation content being enhanced is greater than or equal to a preset enhancement threshold, determining a mask coefficient corresponding to the conversation content according to the sum of the enhanced probability and 1. In response to determining that the probability of conversation content being enhanced is less than the preset enhancement threshold, determining that the value of the mask coefficient corresponding to the conversation content is 1. Determine a mask matrix according to multiple mask coefficients corresponding to multiple rounds of conversation content determined by 1 or the sum.

[0087] For example, corresponding to the third method, a mask matrix randomly generated in units of dialogue rounds can be used to perform information masking on the w+1 rounds of sequential conversation sequence shown in formula (1), and the result shown in formula (4-1) can be obtained.

[0088] Formula (4-1)

[0089] In formula (4-1), The session samples may be generated based on the mask matrix determined in the third way. It can be a mask coefficient for controlling the conversation content of the nth round. Modeling based on conversation rounds can be expressed as shown in the following formula (4-2), but is not limited to this.

[0090] Formula (4-2)

[0091] In formula (4-2), It can be the enhanced probability randomly generated for the session content when constructing the mask matrix. It may be the probability that the content of this round of conversation is enhanced, that is, the above-mentioned preset enhancement threshold, and its value may be a preset hyperparameter.

[0092] Through the above-mentioned embodiments of the present invention, another method for determining the mask coefficient is provided. The mask matrix determined based on this method can simulate the conversation resonance mechanism and perform corresponding information enhancement processing on the time series conversation samples. The constructed conversation samples can enable the model to automatically capture effective conversation history information.

[0093] It should be noted that the method for determining the mask matrix may not be limited to those listed above, but may also include other determination methods, which are not limited here.

[0094] According to an embodiment of the present invention, through one or more of the methods described above, different degrees of conversation masking or conversation enhancement processing can be performed on the temporal conversation sequence based on the mask matrix to construct conversation samples that can be used to train a conversation emotion recognition model.

[0095] Figure 3 The flowchart of the training method of the conversation emotion recognition model according to an embodiment of the present invention is schematically shown.

[0096] like Figure 3 As shown, the method includes operations S301 to S303.

[0097] In operation S301, a time series conversation sequence is input into a conversation emotion recognition model to obtain a first conversation emotion prediction result of the time series conversation sequence, wherein the time series conversation sequence has a conversation emotion label.

[0098] In operation S302, the conversation sample is input into a conversation emotion recognition model to obtain a second conversation emotion prediction result of the conversation sample.

[0099] In operation S303, a conversation emotion recognition model is trained according to the first conversation emotion prediction result, the second conversation emotion prediction result, and the conversation emotion label to obtain a trained conversation emotion recognition model.

[0100] According to an embodiment of the present invention, the conversation sample is obtained by processing the time-series conversation sequence using the above-mentioned conversation sample generation method. The conversation emotion tag can represent the real emotion of the speaker expressing the latest conversation content in the time-series conversation sequence, and the emotion can include happiness, sadness, excitement, anger, etc., but is not limited to this.

[0101] According to an embodiment of the present invention, the conversation emotion recognition model may include various deep learning models or neural network models, such as a pre-trained language model based on Transformer, etc., but is not limited thereto. The pre-trained language model based on Transformer uses an attention mechanism and can focus on semantically related conversation information.

[0102] Figure 4 A schematic diagram schematically shows a training process of a Transformer-based pre-trained language model according to an embodiment of the present invention.

[0103] like Figure 4As shown, when it is necessary to train the pre-trained language model 400, a time-series conversation sequence 410 may be first obtained. Then, based on the aforementioned conversation sample generation method, the time-series conversation sequence 410 may be subjected to information enhancement processing to obtain a first conversation sample 420, a second conversation sample 430, and a third conversation sample 440, etc., and may not be limited thereto. The time-series conversation sequence 410 may have a historical conversation content 411 and a target conversation content 412, and may also have a conversation emotion label representing the real emotion of the target conversation content 412.

[0104] According to an embodiment of the present invention, see Figure 4 As shown, the time-series conversation sequence 410, the first conversation sample 420, the second conversation sample 430, and the third conversation sample 440 may first be input into the pre-trained language model 400. Then, the pre-trained language model 400 may be used to encode the conversation samples, and the encoding results of each conversation sample may be predicted based on, for example, formula (5) to obtain a first conversation emotion prediction result 450 and a second conversation emotion prediction result 460. Each conversation emotion prediction result may include the prediction probability of each type of emotion predicted for each conversation sample.

[0105] Formula (5)

[0106] In formula (5), It can represent the emotion prediction result of the first session. W, b can be the model parameters that need to be learned.

[0107] According to an embodiment of the present invention, after obtaining the first conversation emotion prediction result of a time-series conversation sequence and the second conversation emotion prediction result of each conversation sample, a loss value can be constructed based on the first conversation emotion prediction result, the second conversation emotion prediction result and the conversation emotion label, and the parameters of the conversation emotion recognition model can be adjusted according to the loss value.

[0108] According to an embodiment of the present invention, the operation S203 may further include: determining a first loss according to the first session emotion prediction result and the session emotion label, determining a second loss according to the second session emotion prediction result and the session emotion label, and adjusting parameters of the session emotion recognition model according to the first loss and the second loss.

[0109] For example, we can first construct the CE Loss of the sample prediction probability of the first conversation emotion prediction result of the time series conversation sequence and its label, that is, the first loss mentioned above, and the KL Loss of the sample prediction probability of the second conversation emotion prediction result of each type of conversation sample and the label of the time series conversation sequence, that is, the second loss mentioned above, and establish a loss function based on KL Loss and CE Loss. For example, we can obtain the loss function shown in formula (6).

[0110] Formula (6)

[0111] In formula (6), Can represent conversation emotion labels. The first conversation emotion prediction result may be represented. It can represent the second conversation emotion prediction result output after encoding and predicting the conversation sample constructed based on the mask matrix determined in the first way, for example. It can represent the second conversation emotion prediction result output after encoding and predicting the conversation sample constructed based on the mask matrix determined, for example, in the second manner described above. It can represent the second conversation emotion prediction result output after encoding and predicting the conversation sample constructed based on the mask matrix determined, for example, in the third way mentioned above. It is a pre-set hyperparameter that can be used to control the ratio of the two losses.

[0112] Then, by optimizing KL Loss, the prediction results of various types of conversation samples and time-series conversation sequence samples can be brought closer. At the same time, the CE Loss of the prediction probability of the original samples can be optimized to realize the process of training the conversation emotion recognition model.

[0113] Through the above-mentioned embodiments of the present invention, a solution for effectively modeling context in conversation emotion recognition is proposed, so that the model can implicitly capture real and effective conversation history information. In this solution, a modeling method that integrates context can be adopted to splice historical conversations with current sentences in the order of conversation advancement, and different tags are added to distinguish them. For historical conversations, by adopting a modeling method that controls the visibility of different information, different conversation samples are constructed to better integrate historical conversation information, while enabling the model to automatically capture effective historical conversation information and improve the training accuracy of the model. This method has good portability and can be applied to various pre-trained models.

[0114] Figure 5 The flowchart of the conversation emotion recognition method according to an embodiment of the present invention is schematically shown.

[0115] like Figure 5 As shown, the method includes operations S501~S502.

[0116] In operation S501, a conversation content to be identified and a portion of conversation content before the conversation content to be identified are obtained.

[0117] In operation S502, part of the conversation content and the conversation content to be recognized are input into a conversation emotion recognition model to obtain a conversation emotion recognition result of the conversation content to be recognized.

[0118] According to an embodiment of the present invention, the conversation emotion recognition model is trained using the training method of the aforementioned conversation emotion recognition model. The conversation content to be recognized may include shortened conversation content such as "yes" or "good". Part of the conversation content may be conversation content that is closer to the conversation content to be recognized, or may be conversation content that is older than the conversation content to be recognized, which is not limited here.

[0119] Through the above-mentioned embodiments of the present invention, an effective multi-sample comparison conversation emotion recognition method is proposed, which can distinguish the conversation emotions in multiple rounds of dialogue.

[0120] Figure 6 A block diagram of a conversation sample generating device according to an embodiment of the present invention is schematically shown.

[0121] like Figure 6 As shown, the conversation sample generating device 600 includes a first acquisition module 610 , a first determination module 620 and a second determination module 630 .

[0122] The first acquisition module 610 is used to acquire a time-series conversation sequence, wherein the time-series conversation sequence includes conversation contents of multiple rounds arranged in chronological order.

[0123] The first determination module 620 is used to determine a mask matrix generated for a time-series conversation sequence, wherein the mask matrix includes a plurality of mask coefficients, and the mask coefficients have a one-to-one correspondence with the conversation content.

[0124] The second determination module 630 is used to determine the session sample according to the mask matrix, the time-series session sequence and the one-to-one correspondence.

[0125] According to an embodiment of the present invention, the first determining module includes a first determining unit, a second determining unit and a third determining unit.

[0126] The first determining unit is used to determine, for each round of conversation content, the round number to which the conversation content belongs in the time-series conversation sequence.

[0127] The second determining unit is used to determine a mask coefficient corresponding to the conversation content according to the corresponding round number and the total round number of the time-series conversation sequence.

[0128] The third determining unit is used to determine the mask matrix according to a plurality of mask coefficients corresponding to the conversation contents of a plurality of rounds.

[0129] According to an embodiment of the present invention, the first determining module includes a fourth determining unit and a fifth determining unit.

[0130] The fourth determining unit is used to determine the forgotten probability generated for the conversation content of each round to obtain multiple forgotten probabilities.

[0131] The fifth determining unit is used to determine a mask matrix according to multiple forgotten probabilities.

[0132] According to an embodiment of the present invention, the fifth determining unit includes a first determining subunit, a second determining subunit and a third determining subunit.

[0133] The first determining subunit is configured to determine, for each round of conversation content, that the mask coefficient corresponding to the conversation content is 0 in response to determining that the probability of the conversation content being forgotten is greater than or equal to a preset forgetting threshold.

[0134] The second determining subunit is configured to determine that the mask coefficient corresponding to the conversation content is 1 in response to determining that the probability of the conversation content being forgotten is less than a preset forgetting threshold.

[0135] The third determination subunit is used to determine the mask matrix according to a plurality of mask coefficients determined by 0 or 1 and corresponding to the conversation contents of a plurality of rounds.

[0136] According to an embodiment of the present invention, the first determining module includes a sixth determining unit and a seventh determining unit.

[0137] The sixth determining unit is used to determine the enhanced probability generated for each round of conversation content to obtain multiple enhanced probabilities.

[0138] The seventh determining unit is used to determine a mask matrix according to a plurality of enhanced probabilities.

[0139] According to an embodiment of the present invention, the seventh determining unit includes a fourth determining subunit, a fifth determining subunit and a sixth determining subunit.

[0140] The fourth determination subunit is used to determine, for each round of conversation content, a mask coefficient corresponding to the conversation content according to the sum of the enhancement probability and 1 in response to determining that the enhancement probability of the conversation content is greater than or equal to a preset enhancement threshold.

[0141] The fifth determining subunit is configured to determine that the value of the mask coefficient corresponding to the conversation content is 1 in response to determining that the probability of the conversation content being enhanced is less than a preset enhancement threshold.

[0142] The sixth determination subunit is used to determine the mask matrix according to a plurality of mask coefficients corresponding to the conversation contents of a plurality of rounds determined by 1 or a sum value.

[0143] According to an embodiment of the present invention, the second determining module includes an obtaining unit and an eighth determining unit.

[0144] The obtaining unit is used to perform dot product calculation on the conversation contents of multiple rounds and multiple mask coefficients according to a one-to-one correspondence to obtain a dot product calculation result.

[0145] The eighth determining unit is used to determine the session sample according to the click calculation result.

[0146] According to an embodiment of the present invention, the conversation sample generating device further includes a third determining module and a fourth determining module.

[0147] The third determination module is used to determine the target conversation content and the historical conversation content, wherein the historical conversation content is located before the target conversation content, and the historical conversation content includes conversation contents of a preset number of rounds.

[0148] The fourth determination module is used to determine the time-series conversation sequence according to the historical conversation content and the target conversation content.

[0149] Figure 7 A block diagram of a training device for a conversation emotion recognition model according to an embodiment of the present invention is schematically shown.

[0150] like Figure 7 As shown, the training device 700 for the conversation emotion recognition model includes a first acquisition module 710 , a second acquisition module 720 and a training module 730 .

[0151] The first obtaining module 710 is used to input the time series conversation sequence into the conversation emotion recognition model to obtain a first conversation emotion prediction result of the time series conversation sequence, wherein the time series conversation sequence has a conversation emotion label.

[0152] The second acquisition module 720 is used to input the conversation sample into the conversation emotion recognition model to obtain the second conversation emotion prediction result of the conversation sample, wherein the conversation sample is obtained by processing the time-series conversation sequence using the conversation sample generation device of the present invention.

[0153] The training module 730 is used to train the conversation emotion recognition model according to the first conversation emotion prediction result, the second conversation emotion prediction result and the conversation emotion label to obtain a trained conversation emotion recognition model.

[0154] According to an embodiment of the present invention, the training module includes a ninth determining unit, a tenth determining unit and an adjusting unit.

[0155] The ninth determining unit is used to determine the first loss according to the first conversation emotion prediction result and the conversation emotion label.

[0156] The tenth determining unit is used to determine the second loss according to the second conversation emotion prediction result and the conversation emotion label.

[0157] An adjustment unit is used to adjust parameters of the conversation emotion recognition model according to the first loss and the second loss.

[0158] Figure 8 A block diagram of a conversation emotion recognition device according to an embodiment of the present invention is schematically shown.

[0159] like Figure 8 As shown, the conversation emotion recognition device 800 includes a second acquisition module 810 and a third acquisition module 820 .

[0160] The second acquisition module 810 is used to acquire the conversation content to be identified and a portion of conversation content before the conversation content to be identified.

[0161] The third acquisition module 820 is used to input part of the conversation content and the conversation content to be recognized into the conversation emotion recognition model to obtain the conversation emotion recognition result of the conversation content to be recognized, wherein the conversation emotion recognition model is trained using the training device of the conversation emotion recognition model of the present invention.

[0162] According to the embodiments of the present invention, any one or more of the modules, units, and sub-units, or at least part of the functions of any one of them, can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, units, and sub-units can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, units, and sub-units can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, units, and sub-units can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0163] For example, any multiple of the first acquisition module 610, the first determination module 620 and the second determination module 630, or the first acquisition module 710, the second acquisition module 720 and the training module 730, or the second acquisition module 810 and the third acquisition module 820 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present invention, at least one of the first acquisition module 610, the first determination module 620 and the second determination module 630, or the first acquisition module 710, the second acquisition module 720 and the training module 730, or the second acquisition module 810 and the third acquisition module 820 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the first acquisition module 610, the first determination module 620 and the second determination module 630, or the first acquisition module 710, the second acquisition module 720 and the training module 730, or the second acquisition module 810 and the third acquisition module 820 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0164] It should be noted that the conversation sample generation device, the conversation emotion recognition model training device and the conversation emotion recognition device in the embodiments of the present invention correspond to the conversation sample generation method, the conversation emotion recognition model training method and the conversation emotion recognition method in the embodiments of the present invention. The description of the conversation sample generation device, the conversation emotion recognition model training device and the conversation emotion recognition part specifically refers to the conversation sample generation method, the conversation emotion recognition model training method and the conversation emotion recognition method part, and will not be repeated here.

[0165] Fig. 9 A block diagram of an electronic device suitable for implementing at least one of a conversation sample generation method, a conversation emotion recognition model training method, and a conversation emotion recognition method according to an embodiment of the present invention is schematically shown. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0166] like Fig. 9 As shown, the electronic device 900 according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0167] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0168] According to an embodiment of the present invention, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The system 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0169] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0170] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0171] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.

[0172] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 902 and / or the RAM 903 described above and / or one or more memories other than the ROM 902 and the RAM 903 .

[0173] An embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method provided by the embodiment of the present invention.

[0174] When the computer program is executed by the processor 901, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0175] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0176] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0177] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teaching of the present invention, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0178] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination. The scope of the present invention is defined by the attached claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for generating a conversation sample, comprising: Acquire a time-series conversation sequence, wherein the time-series conversation sequence includes conversation contents of multiple rounds arranged in chronological order, the time-series conversation sequence has a conversation emotion label, the time-series conversation sequence is determined according to historical conversation contents and target conversation contents, the historical conversation contents are located before the target conversation contents, the historical conversation contents include conversation contents of a preset number of rounds, and the conversation emotion label represents the real emotion of the target conversation contents; Determining a mask matrix generated for the time series conversation sequence according to at least one of a probability of being forgotten and a probability of being enhanced of the conversation content of each round, including: Determine the forgotten probability generated for each round of conversation content to obtain multiple forgotten probabilities; for each round of conversation content, in response to determining that the forgotten probability of the conversation content is greater than or equal to a preset forgetting threshold, determine that the mask coefficient corresponding to the conversation content is 0; in response to determining that the forgotten probability of the conversation content is less than the preset forgetting threshold, determine that the mask coefficient corresponding to the conversation content is 1; and determine the mask matrix according to the multiple mask coefficients corresponding to the conversation contents of the multiple rounds determined by 0 or 1; Determine the enhanced probability generated for each round of conversation content to obtain multiple enhanced probabilities; for each round of conversation content, in response to determining that the enhanced probability of the conversation content is greater than or equal to a preset enhancement threshold, determine a mask coefficient corresponding to the conversation content according to the sum of the enhanced probability and 1; in response to determining that the enhanced probability of the conversation content is less than the preset enhancement threshold, determine that the value of the mask coefficient corresponding to the conversation content is 1; and determine the mask matrix according to the multiple mask coefficients corresponding to the conversation contents of the multiple rounds determined by 1 or the sum; The mask matrix includes a plurality of mask coefficients, and the mask coefficients have a one-to-one correspondence with the conversation content; and A session sample is determined according to the mask matrix, the time-series session sequence and the one-to-one correspondence.

2. The method according to claim 1, wherein: The determining of a mask matrix generated for the timing session sequence comprises: For each round of conversation content, determine the round number to which the conversation content belongs in the time-series conversation sequence; Determining a mask coefficient corresponding to the conversation content according to the round number and the total round number of the time-series conversation sequence; and The mask matrix is ​​determined according to a plurality of mask coefficients corresponding to the conversation contents of the plurality of rounds.

3. The method according to claim 1 or 2, wherein: The determining of the session sample according to the mask matrix, the time-series session sequence and the one-to-one correspondence includes: According to the one-to-one correspondence, performing dot product calculations on the conversation contents of the multiple rounds and the multiple mask coefficients to obtain a dot product calculation result; and The session sample is determined according to the dot product calculation result.

4. A method for training a conversation emotion recognition model, comprising: Inputting a time series conversation sequence into a conversation emotion recognition model to obtain a first conversation emotion prediction result of the time series conversation sequence, wherein the time series conversation sequence has a conversation emotion label; Inputting a conversation sample into the conversation emotion recognition model to obtain a second conversation emotion prediction result of the conversation sample, wherein the conversation sample is obtained by processing the time series conversation sequence using the method according to any one of claims 1 to 3; and The conversation emotion recognition model is trained according to the first conversation emotion prediction result, the second conversation emotion prediction result and the conversation emotion label to obtain a trained conversation emotion recognition model.

5. The method according to claim 4, wherein: The training of the conversation emotion recognition model according to the first conversation emotion prediction result, the second conversation emotion prediction result and the conversation emotion label includes: Determining a first loss according to the first conversation emotion prediction result and the conversation emotion label; determining a second loss according to the second conversation emotion prediction result and the conversation emotion label; and According to the first loss and the second loss, the parameters of the conversation emotion recognition model are adjusted.

6. A conversation emotion recognition method, comprising: Acquire the conversation content to be identified and a portion of the conversation content before the conversation content to be identified; as well as The partial conversation content and the conversation content to be recognized are input into a conversation emotion recognition model to obtain a conversation emotion recognition result of the conversation content to be recognized, wherein the conversation emotion recognition model is trained using the method according to claim 4 or 5.

7. A conversation sample generating device, comprising: A first acquisition module is used to acquire a time-series conversation sequence, wherein the time-series conversation sequence includes conversation contents of multiple rounds arranged in chronological order, the time-series conversation sequence has a conversation emotion label, the time-series conversation sequence is determined according to historical conversation contents and target conversation contents, the historical conversation contents are located before the target conversation contents, the historical conversation contents include conversation contents of a preset number of rounds, and the conversation emotion label represents the real emotion of the target conversation contents; A first determination module is used to determine a mask matrix generated for the time series conversation sequence according to at least one of a probability of being forgotten and a probability of being enhanced of the conversation content of each round, wherein the mask matrix includes a plurality of mask coefficients, and the mask coefficients have a one-to-one correspondence with the conversation content, and the first determination module includes: a fourth determining unit, configured to determine the forgotten probability generated for the conversation content of each round to obtain a plurality of forgotten probabilities; a first determining subunit, configured to determine, for the conversation content of each round, in response to determining that the forgotten probability of the conversation content is greater than or equal to a preset forgetting threshold, that the mask coefficient corresponding to the conversation content is 0; a second determining subunit, configured to determine that, in response to determining that the forgotten probability of the conversation content is less than the preset forgetting threshold, that the mask coefficient corresponding to the conversation content is 1; and a third determining subunit, configured to determine the mask matrix according to the plurality of mask coefficients corresponding to the conversation contents of the plurality of rounds determined by 0 or 1; a sixth determination unit, for determining the enhanced probability generated for each round of conversation content to obtain a plurality of enhanced probabilities; a fourth determination subunit, for determining, for each round of conversation content, a mask coefficient corresponding to the conversation content according to the sum of the enhanced probability and 1 in response to determining that the enhanced probability of the conversation content is greater than or equal to a preset enhancement threshold; a fifth determination subunit, for determining that the value of the mask coefficient corresponding to the conversation content is 1 in response to determining that the enhanced probability of the conversation content is less than the preset enhancement threshold; and a sixth determination subunit, for determining the mask matrix according to a plurality of mask coefficients corresponding to the conversation contents of the plurality of rounds determined by 1 or the sum; and The second determination module is used to determine the session sample according to the mask matrix, the time-series session sequence and the one-to-one correspondence.

8. A training device for a conversation emotion recognition model, comprising: A first acquisition module is used to input a time series conversation sequence into a conversation emotion recognition model to obtain a first conversation emotion prediction result of the time series conversation sequence, wherein the time series conversation sequence has a conversation emotion label; A second acquisition module is used to input a conversation sample into the conversation emotion recognition model to obtain a second conversation emotion prediction result of the conversation sample, wherein the conversation sample is obtained by processing the time-series conversation sequence using the device according to claim 7; and A training module is used to train the conversation emotion recognition model according to the first conversation emotion prediction result, the second conversation emotion prediction result and the conversation emotion label to obtain a trained conversation emotion recognition model.

9. A conversation emotion recognition device, comprising: A second acquisition module is used to acquire the conversation content to be identified and a portion of conversation content located before the conversation content to be identified; as well as The third acquisition module is used to input the partial conversation content and the conversation content to be recognized into a conversation emotion recognition model to obtain a conversation emotion recognition result of the conversation content to be recognized, wherein the conversation emotion recognition model is trained using the device according to claim 8.

10. An electronic device, comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

11. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 6.

Images