Emotion recognition method and model training method, apparatus, device, and storage medium

Abstract

The application discloses an emotion recognition method, a model training method, a device, equipment and a storage medium. The method comprises the following steps: recognizing a first to-be-recognized text by using a first model to obtain a first recognition result, the first to-be-recognized text comprising external stimulus information given to a to-be-recognized intelligent agent at a first time, and the first recognition result representing a first emotional state of the to-be-recognized intelligent agent caused by the external stimulus information; recognizing a second to-be-recognized text by using a second model to obtain a second recognition result, the second to-be-recognized text comprising information for describing a second emotional state of the to-be-recognized intelligent agent at the first time, and the second recognition result representing the second emotional state; and determining an emotional state of the to-be-recognized intelligent agent at a current time according to attribute information, the first recognition result and the second recognition result, the attribute information representing a personality characteristic of the to-be-recognized intelligent agent, and the current time being a time after the second time. The method can improve the accuracy of the emotion recognition result.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an emotion recognition method, model training method, device, equipment and storage medium. Background Technology

[0002] Emotions influence people's words and actions in life, and the recognition of human emotions is a hot topic of discussion and research. In the field of Artificial Intelligence (AI), the cognitive, decision-making, and reasoning abilities of intelligent agents have developed rapidly. By endowing intelligent agents with human-like emotional expressions, it is possible to build human-computer interaction (HCI) environments.

[0003] In practical applications, the emotional state of an agent can be obtained by recognizing its emotions. Traditional techniques typically predict the agent's emotional state after the current moment based on its emotional state at moments prior to the current moment. However, the emotional state of an agent is usually influenced by various factors, leading to inaccuracies in the emotion recognition results obtained using traditional techniques.

[0004] Therefore, there is an urgent need for an emotion recognition method that can improve the accuracy of emotion recognition results. Summary of the Invention

[0005] This application provides an emotion recognition method, a model training method, an apparatus, a device, and a storage medium, which can improve the accuracy of emotion recognition results.

[0006] The first aspect of this application provides an emotion recognition method, which includes: using a first model to recognize a first text to be recognized to obtain a first recognition result, wherein the first text to be recognized includes external stimulus information given to the intelligent agent to be recognized at a first moment, and the first recognition result represents a first emotional state generated by the external stimulus information to the intelligent agent to be recognized; using a second model to recognize a second text to be recognized to obtain a second recognition result, wherein the second text to be recognized includes information for describing a second emotional state of the intelligent agent to be recognized at a first moment, and the second recognition result represents the second emotional state; and determining the emotional state of the intelligent agent to be recognized at the current moment based on attribute information, the first recognition result, and the second recognition result, wherein the attribute information represents the personality characteristics of the intelligent agent to be recognized, and the current moment is a moment after the second moment.

[0007] A second aspect of this application provides a model training method, the method comprising: acquiring a first training set, wherein the first training set includes a first sample and a real label of the first sample, the first sample includes external stimulus information given to an agent at a first preset historical time, and the real label of the first sample represents the emotional state generated by the agent at a second preset historical time after the first preset historical time due to the external stimulus information given to the agent; and adjusting the parameters of a first initial encoder and a first initial classifier included in a first initial model using the first training set to obtain the first model.

[0008] A third aspect of this application provides a model training method, the method comprising: obtaining a second training set, wherein the second training set includes a second sample and a real label of the second sample, the second sample includes information describing the emotional state of an agent at a first preset historical moment, and the real label of the second sample represents the emotional state of the agent at the first preset historical moment; and adjusting the parameters of a second initial encoder and a second classifier included in a second initial model using the second training set to obtain the second model.

[0009] A fourth aspect of this application provides an emotion recognition device, comprising: a first recognition unit, configured to recognize a first text to be recognized using a first model to obtain a first recognition result, wherein the first text to be recognized includes external stimulus information given to the intelligent agent to be recognized at a first moment, and the first recognition result indicates a first emotional state generated by the intelligent agent to be recognized at a second moment after the first moment due to the external stimulus information; a second recognition unit, configured to recognize a second text to be recognized using a second model to obtain a second recognition result, wherein the second text to be recognized includes information describing a second emotional state of the intelligent agent to be recognized at the first moment, and the second recognition result indicates the second emotional state; and a determining unit, configured to determine the emotional state of the intelligent agent to be recognized at a current moment based on attribute information, the first recognition result, and the second recognition result, wherein the attribute information represents the personality traits of the intelligent agent to be recognized, and the current moment is a moment after the second moment.

[0010] A fifth aspect of this application provides a model training apparatus, comprising: an acquisition unit for acquiring a first training set, wherein the first training set includes a first sample and a real label of the first sample, the first sample including external stimulus information given to an agent at a first preset historical time, and the real label of the first sample representing the emotional state generated by the agent at a second preset historical time after the first preset historical time due to the external stimulus information given to the agent; and a training unit for adjusting the parameters of a first initial encoder and a first initial classifier included in a first initial model using the first training set to obtain the first model.

[0011] A sixth aspect of this application provides a model training apparatus, comprising: an acquisition unit for acquiring a second training set, wherein the second training set includes second samples and real labels of the second samples, the second samples including information describing the emotional state of an agent at a first preset historical moment, and the real labels of the second samples representing the emotional state of the agent at the first preset historical moment; and a training unit for adjusting the parameters of a second initial encoder and a second classifier included in a second initial model using the second training set to obtain the second model.

[0012] A seventh aspect of this application also provides an execution device, including: a processor; and a memory for storing a data processing program. After the execution device is powered on and runs the program through the processor, it executes the method described above.

[0013] An eighth aspect of this application also provides a training device, including: a processor; and a memory for storing a data processing program. After the training device is powered on and runs the program through the processor, it executes the method described above.

[0014] A ninth aspect of this application also provides a computer-readable storage medium storing one or more computer instructions, characterized in that the instructions are executed by a processor to implement the emotion recognition method described in any of the above technical solutions.

[0015] A ninth aspect of this application also provides a computer-readable storage medium storing one or more computer instructions, characterized in that the instructions are executed by a processor to implement the model training method described in any of the above technical solutions.

[0016] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments disclosed in this application, nor is it intended to limit the scope of this application's disclosure. Other features disclosed in this application will become readily apparent from the following description.

[0017] The technical solution of the emotion recognition method provided in this application includes: using a first model to recognize a first text to be recognized to obtain a first recognition result, wherein the first text to be recognized includes external stimulus information given to the intelligent agent to be recognized at a first moment, and the first recognition result represents a first emotional state generated by the external stimulus information to the intelligent agent to be recognized; using a second model to recognize a second text to be recognized to obtain a second recognition result, wherein the second text to be recognized includes information for describing a second emotional state of the intelligent agent to be recognized at a first moment, and the second recognition result represents the second emotional state; and determining the emotional state of the intelligent agent to be recognized at the current moment based on attribute information, the first recognition result, and the second recognition result, wherein the attribute information represents the personality characteristics of the intelligent agent to be recognized, and the current moment is a moment after the second moment. In the aforementioned emotion recognition method, the emotional state of the agent to be recognized at the current moment is determined based on attribute information, the first recognition result, and the second recognition result. That is, when determining the emotional state of the agent to be recognized at the current moment, not only is the emotional state of the agent to be recognized at the first moment before the current moment (i.e., the second recognition result) considered, but also the emotional state generated by the agent to be recognized after being given external stimulus information at the first moment (i.e., the first recognition result), as well as the attribute information of the agent to be recognized. In this way, the accuracy of the emotion recognition result can be improved. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of an application scenario applicable to the emotion recognition method provided in the embodiments of this application.

[0020] Figure 2 This is a schematic diagram of an emotion recognition method provided in an embodiment of this application.

[0021] Figure 3 is a schematic diagram of a two-dimensional emotion coordinate system provided in an embodiment of this application.

[0022] Figure 4 The above Figure 2 A schematic diagram illustrating the training process for obtaining the first model involved in the described emotion recognition method.

[0023] Figure 5 The above Figure 2 A schematic diagram illustrating the training process for obtaining the second model involved in the described emotion recognition method.

[0024] Figure 6 This is a schematic diagram of the emotional vector of an intelligent agent provided in an embodiment of this application.

[0025] Figure 7 This is a schematic diagram of another emotion recognition method provided in the embodiments of this application.

[0026] Figure 8 This is a schematic diagram of an emotion recognition device provided in an embodiment of this application.

[0027] Figure 9 This is a schematic diagram of a training device provided in an embodiment of this application.

[0028] Figure 10 This is a schematic diagram of an execution device provided in an embodiment of this application.

[0029] Figure 11 This is a schematic diagram of a training device provided in an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the technical solutions of this application, the application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. However, this application can be implemented in many other ways different from those described above. Therefore, based on the embodiments provided in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0031] It should be noted that the terms "first," "second," "third," etc., in the claims, specification, and drawings of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. Such data are interchangeable where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown or described herein. Furthermore, the terms "comprising," "having," and their variations are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0032] It should be noted that the terms "emotion" and "feeling" used in the emotion recognition method provided in this application embodiment can be used interchangeably.

[0033] To facilitate understanding, the technical terms that may be involved in the embodiments of this application will be briefly introduced first.

[0034] 1. Agent

[0035] The concept of intelligent agents first appeared in Professor Minsky's 1986 book, *The Society of Mind*, which posits that certain individuals in society, through negotiation, can find solutions to problems; these individuals are considered intelligent agents. Intelligent agents are used to describe hardware, software, or other entities with adaptive and autonomous capabilities, aiming to recognize and simulate human intelligence. In 1994, Minsky further clarified the concept of intelligent agents in the journal *Communications of the ACM*. This clarification considers both society and individuals. From society's perspective, an intelligent agent is an individual capable of performing specific tasks; society is not concerned with how it works. From the individual's perspective, however, they are required to possess certain abilities; otherwise, they will not be accepted by society.

[0036] 2. Sentiments

[0037] Emotions are experiences linked to a person's social needs. The term "emotion" is often used to describe feelings that have stable and profound social significance. As a feeling, emotions possess strong stability, depth, and persistence.

[0038] 3. Emotions

[0039] Emotions are experiences linked to a person's physiological needs; that is, emotions are psychological activities mediated by the subject's desires and needs. When objective things or situations meet the subject's desires and needs, they can evoke positive, affirmative emotions. When objective things do not meet the subject's desires or needs, they can evoke negative, negative emotions. Emotions consist of three parts: unique subjective experience, external expression, and physiological arousal. Emotions possess adaptive, motivational, organizational, and social functions.

[0040] Although the definitions of emotion and feeling are not exactly the same, they are inseparable. Therefore, people often use the terms "emotion" and "feeling" interchangeably. That is to say, the terms "emotion" and "feeling" used in the emotion recognition method provided in this application embodiment can be used interchangeably.

[0041] The application scenarios and methods of emotion recognition applicable to embodiments of this application will be described in detail below with reference to the accompanying drawings. It is understood that, where there is no conflict between the various embodiments provided in this application, the following embodiments and features can be combined with each other. Furthermore, the timing sequence of steps in the following method embodiments is merely an example and not a strict limitation.

[0042] First, the application scenarios of the emotion recognition method applicable to the embodiments of this application will be introduced with reference to the accompanying drawings.

[0043] Figure 1This is a schematic diagram illustrating an application scenario of the emotion recognition method provided in the embodiments of this application. For example... Figure 1 As shown, this application scenario includes at least one terminal 101 and at least one server 102. The server 102 and the terminal 101 can communicate via a network 103 to transmit data. The network 103 can be a wired network or a wireless network; this application does not specifically limit its use.

[0044] Applications can be installed and run on terminal 101. These applications can be any application capable of providing emotion recognition services. For example, the applications can be, but are not limited to, any of the following: smart assistant applications, video applications, news applications, social networking applications, interactive entertainment applications, browser applications, shopping applications, content sharing applications, virtual reality (VR) applications, or augmented reality (AR) applications. In this embodiment, the device type of terminal 101 is not specifically limited. For example, terminal 101 can be, but is not limited to, any of the following devices: personal computer, smartphone, tablet computer, desktop computer, smart speaker, smartwatch, or wearable device, etc.

[0045] Server 102 provides background services to clients of applications in terminal 101. For example, server 102 can be a background server for the aforementioned applications. Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. Optionally, server 102 can simultaneously provide background services to applications in multiple terminals 101.

[0046] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can be any network, including but not limited to local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0047] It should be understood that the above Figure 1 The application scenarios shown are for illustrative purposes only and do not constitute any limitation on the application scenarios to which the emotion recognition method provided in this application embodiment is applicable. Optionally, the above... Figure 1 The application scenarios shown can also include a greater number of servers 102, a greater number of terminals 101, and a greater number of networks 103. Optionally, the above... Figure 1 The terminal 101 shown can also be replaced by a server other than server 102.

[0048] Next, combined Figures 2 to 7 The emotion recognition method provided in the embodiments of this application will be described in detail.

[0049] Figure 2 This is a schematic diagram illustrating an emotion recognition method provided in an embodiment of this application. The emotion recognition method provided in this embodiment can be executed by an execution device. It is understood that the execution device can be implemented as software, or a combination of software and hardware. For example, the execution device in this embodiment can be, but is not limited to, a server or a terminal device used by a user. Figure 2 As shown, the emotion recognition method provided in this application includes steps S210 to S230. Steps S210 to S230 will be described in detail below.

[0050] S210, the first model is used to identify the first text to be identified and a first identification result is obtained. The first text to be identified includes external stimulus information given to the intelligent agent to be identified at a first moment. The first identification result indicates the first emotional state generated by the intelligent agent to be identified at a second moment after the first moment due to the external stimulus information.

[0051] The first model described in S210 above is a trained model. In other words, executing the method described in S210 above is the application stage of the first model. Specifically, the first model is used to recognize the first text to be recognized in order to obtain the first recognition result.

[0052] Below, we will first provide a detailed introduction to the first text to be identified and the first identification result described in S210 above.

[0053] The first text to be identified includes external stimulus information given to the intelligent agent at a first moment. This external stimulus information can be understood as information that can influence the emotional state of the intelligent agent. Specifically, after the external stimulus information is given to the intelligent agent at the first moment, the intelligent agent will generate a first emotional state corresponding to that external stimulus information at a second moment. The time difference between the first and second moments is not specifically limited. Generally, after a stimulus is given to the intelligent agent, the agent can quickly generate an emotion corresponding to that stimulus. That is, in this scenario, the time difference between the first and second moments can be very short, for example, one second.

[0054] In this embodiment, the content of the external stimulus information given to the intelligent agent to be identified is not specifically limited. In other words, any information that can affect the emotional state of the intelligent agent to be identified can be used as external stimulus information. In some implementations, the external stimulus information includes at least one of the following: information evaluating the intelligent agent to be identified, or environmental information about the location of the intelligent agent to be identified. That is, the external stimulus information can be information evaluating the intelligent agent to be identified, or environmental information about the location of the intelligent agent to be identified; or, the external stimulus information can include: information evaluating the intelligent agent to be identified, and environmental information about the location of the intelligent agent to be identified. The environmental information about the location of the intelligent agent to be identified can be location environment information, weather environment information, etc., and is not specifically limited thereto. For example, when the external stimulus information is the external physical environment information of the intelligent agent to be identified, the first text to be identified can be expressed as: "The physical environment of the intelligent agent to be identified is very cold." For example, when the external stimulus information is information evaluating the intelligent agent to be identified, the first text to be identified can be expressed as: "The intelligent agent to be identified is wearing nice clothes today." The first identification result indicates the first emotional state of the intelligent agent to be identified caused by the external stimulus information. Generally, when external stimuli are positive, the first emotional state is positive; when external stimuli are negative, the first emotional state is negative.

[0055] The first model described in S210 above is a trained model. Optionally, in some implementations, a training method for obtaining the first model can be executed before executing S210 above. Next, the training method for obtaining the first model provided by the embodiments of this application will be introduced.

[0056] Optionally, in some implementations, before executing S210 above, i.e., before using the first model to recognize the first text to be recognized and obtaining the first recognition result, the method further includes: adjusting the parameters of the first initial encoder and the parameters of the first initial classifier included in the first initial model using the first training set to obtain the first model. The first training set includes first samples and the true labels of the first samples. The first samples include external stimulus information given to the agent at a first preset historical time. The true labels of the first samples represent the emotional state generated by the agent at a second preset historical time after the first preset historical time due to the external stimulus information given to the agent. There is no specific limitation on the time difference between the first preset historical time and the second preset historical time. Generally speaking, after giving a stimulus to the agent to be recognized, the agent can quickly generate an emotion corresponding to that stimulus. That is, in this scenario, the time difference between the first preset historical time and the second preset historical time can be very short, for example, 0.5 seconds.

[0057] In the training method described above, which uses a first training set to train a first initial model to obtain a first model, the first training set includes a first sample and the true label of the first sample. This means the training process is a supervised model training process, which ensures the accuracy of the first model obtained through training. Furthermore, it can improve the accuracy of the first recognition result obtained by using the first model to recognize a first document to be recognized, thereby contributing to improved accuracy of sentiment recognition results.

[0058] The training data used in the above training process, namely the first training set, will be described in detail below. The first training set includes the first sample and the true label of the first sample.

[0059] The first sample includes external stimulus information given to the agent at a first preset historical moment. This external stimulus information can be, but is not limited to, positive or negative stimuli. In this embodiment, the content of the external stimulus information given to the agent is not specifically limited and can be set according to the application scenario. In other words, any information that can affect the agent's emotional state can be used as external stimulus information. The form of the external stimulus information given to the agent is not specifically limited and can be set according to the application scenario. In some implementations, the external stimulus information given to the agent can include, but is not limited to, any of the following: environmental information of the agent's location, or information evaluating the agent. The environmental information of the agent's location can be location environment information, weather environment information, etc., and is not specifically limited thereto.

[0060] The true label of the first sample represents the emotional state that the agent experiences at a second preset historical moment, following a first preset historical moment, due to external stimuli. It can be understood that the emotional state represented by the true label of the first sample is any one of the N types of emotional states output by the second classifier. Generally, if the external stimuli are positive, the first sample corresponds to a positive emotional state; if the external stimuli are negative, the first sample corresponds to a negative emotional state.

[0061] In this embodiment, the number of samples included in the first training set is not specifically limited. That is, the first training set may include multiple first samples and multiple real labels for the first samples, and the multiple first samples are all different. Here, any two different first samples can be understood as the following: the agents corresponding to these two different first samples are different, the external stimulus information corresponding to these two different first samples is different, or the preset historical time corresponding to these two different first samples is different.

[0062] Next, we will introduce in detail the first initial model and the first model involved in the above model training process.

[0063] The first initial model includes a first initial encoder and a first classifier. The first initial encoder has feature extraction capabilities; the first classifier is an N-classifier, where N is greater than or equal to a first preset threshold. For example, Figure 3A Figure (1) shows a schematic diagram of the first model obtained by training the first initial model using the first training set, and the structure of the first initial model. In the embodiments of this application, the result of the first initial encoding and the structure of the first classifier are not limited. That is, any model structure with feature extraction can be used as the first initial encoder, and any classifier with N-classification function can be used as the first initial classifier. For example, the first initial encoder can be, but is not limited to, the encoder included in the Transformer model, or a variant of the encoder included in the Transformer model. For example, the first initial classifier can be, but is not limited to, a random forest classifier or a Naive Bayes classifier.

[0064] The first model is obtained by training the parameters of the first initial model using a first training set. Based on this, the first model may include a first encoder and a first classifier. The first encoder is the encoder obtained by adjusting the parameters of the first initial encoder as described above; the first classifier is the classifier obtained by adjusting the parameters of the first initial classifier as described above. For example, Figure 3AFigure (1) shows a schematic diagram of the first model obtained by training the first initial model using the first training set, and the structure of the first model.

[0065] The previous section detailed the first training set, the first initial model, and the first model. Below, we will combine... Figure 4 The training method S400, which involves "adjusting the parameters of the first initial encoder and the first initial classifier using the first training set to obtain the first model," is described in detail below. See also... Figure 4 The training method 400 includes S410 to S440. Below, S410 to S440 will be described in detail.

[0066] S410, the first initial encoder is used to extract features from the first sample to obtain the feature vector of the first sample.

[0067] Executing S410 above, that is, using the first initial encoder to extract features from the first sample and obtain the feature vector of the first sample, may include the following steps: inputting the first sample into the first initial encoder, and the first initial encoder outputting the feature vector of the first sample.

[0068] S420: The first initial classifier is used to classify the feature vector of the first sample to obtain the predicted label of the first sample.

[0069] In this embodiment, the recognition result output by the first initial classifier is any one of N types of emotional states, wherein the first recognition result is any one of N types of emotional states, and N is greater than or equal to a first preset threshold. In the above implementation, the first initial classifier is an N-classifier. Based on this, the predicted label of the first sample described in S420 corresponds to one of the N types of emotional states. The value of N and the value of the first preset threshold are not specifically limited. It is understood that in the above implementation, the larger the value of N, the finer the granularity of the emotional recognition result, which can further improve the accuracy of the emotional recognition result. In some implementations, N can be equal to 24, where... Figure 3B This illustrates these 24 types of emotional states. Below, we will combine... Figure 3B The recognition results output by the first initial classifier are introduced. Figure 3B This illustration shows a schematic diagram of a two-dimensional emotion coordinate system provided in an embodiment of this application. See also... Figure 3B As shown, the horizontal axis of the two-dimensional emotion coordinate system represents the degree of positive or negative emotion. A value greater than zero and a larger absolute value indicates a deeper positive emotion, while a value less than zero and a larger absolute value indicates a deeper negative emotion. See also... Figure 3BAs shown, the vertical axis of the two-dimensional emotion coordinate system represents the degree of internalization and externalization of the agent's emotion. Similarly, the further away from the origin and the larger the absolute value, the deeper the degree; the closer to the origin, the shallower the degree. See also... Figure 3B , Figure 3B The illustrated two-dimensional emotional coordinates include 24 emotional states, specifically the following emotions: "anger," "irritability," "disgust," "abhorrence," "sadness," "fear," "depression," "regret," "anger," "complaint," "mild negativity," "worry," "fright," "interest," "doubt," "neutral," "surprise," "coquettishness," "mild positivity," "surprise," "happiness," "admiration," "like," and "trust." Traditional techniques typically use a 7-classifier to identify emotional states, outputting one of the following results: "fear," "happiness," "surprise," "anger," "sadness," "disgust," or "other." In the implementation provided in this application, the first initial classifier is a 24-classifier. This allows for a finer granularity in the identification results output by the first classifier in the trained first model, improving the accuracy of emotion recognition.

[0070] Performing S420 above, that is, using the first initial classifier to classify the feature vector of the first sample and obtain the predicted label of the first sample, may include the following steps: inputting the feature vector of the first sample into the first initial classifier, and the first initial classifier outputting the predicted label of the first sample.

[0071] S430, adjust the parameters of the first initial encoder and the parameters of the first initial classifier based on the difference between the true label and the predicted label of the first sample.

[0072] The difference between the true label and the predicted label of the first sample can be represented by a loss function. In this embodiment, the loss function can be a classification loss function. The form of the classification loss function is not specifically limited and can be selected according to actual needs. For example, the classification loss function can be, but is not limited to, negative log-likelihood loss or cross-entropy loss.

[0073] S440, once the training reaches the first preset training condition, stop adjusting the parameters of the first initial encoder and the first initial classifier to obtain the first model.

[0074] The first preset training condition is not specifically limited and can be set according to the actual scenario. In some implementations, the first preset training condition may include at least one of the following conditions: the number of training iterations of the first initial model meets a preset number of training iterations, the training time of the first initial model meets a preset training time, or the loss result of the first initial model (i.e., the value of the loss function of the first initial model) is less than a preset loss threshold. The preset number of training iterations, preset training time, and preset loss threshold are not specifically limited and can be set according to actual needs.

[0075] It should be noted that the training steps described in S410 to S440 above, namely "training the first initial model using the first training set to obtain the first model", are merely illustrative and do not constitute any limitation on the embodiments of this application. That is to say, other training steps besides those described in S410 to S440 above can also be performed on the first initial model using the first training set to obtain the first model.

[0076] Optionally, in some implementations, the step of obtaining the first training set can be performed before executing S210. In this embodiment, the method for obtaining the first training set is not specifically limited. In some implementations, obtaining the first training set may include the following steps: obtaining sample sentences labeled with different external stimuli; preprocessing the sample sentences labeled with different external stimuli to obtain the first training set, wherein the number of samples included in the first training set is less than the number of sample sentences labeled with different external stimuli. In the above implementations, preprocessing the large number of sample sentences labeled with different external stimuli can effectively reduce data redundancy. This is beneficial for improving the training efficiency of training the first initial model using a non-redundant first training set. The preprocessing method is not specifically limited and can be set according to the actual application scenario and requirements. For example, the preprocessing method may include, but is not limited to: text deduplication, Chinese word segmentation, or stop word removal.

[0077] S220, the second model is used to identify the second text to be identified, and a second identification result is obtained. The second text to be identified includes information describing the second emotional state of the agent to be identified at the first moment, and the second identification result represents the second emotional state.

[0078] The second model described in S220 above is a trained model. In other words, executing the method described in S220 is the application phase of the second model. Specifically, the second model is used to recognize the second text to be recognized, thereby obtaining a second recognition result.

[0079] Below, we will first provide a detailed introduction to the second text to be identified and the second identification result described in S220 above.

[0080] The second text to be identified includes information describing the second emotional state of the agent to be identified at a first moment. The information describing the emotional state of the agent to be identified at the first moment is not specifically limited. In some implementations, the information describing the emotional state of the agent to be identified at the first moment may be the words spoken by the agent at the first moment. The words spoken by the agent at the first moment can be generated by a language model (LM) based on the agent's emotional state at the first moment. In this embodiment, the type of language model is not specific; for example, the language model may be, but is not limited to, a statistical language model (SLM) or a recurrent neural network (RNN). Optionally, in other implementations, the information describing the emotional state of the agent to be identified at the first moment may be the text written by the agent at the first moment. For example, the second sample may be represented as: "I am in a good mood today," where "I" refers to the agent.

[0081] The second recognition result represents the second emotional state. It can be understood that the emotional state represented by the second recognition result is any one of the M types of emotional states output by the second classifier.

[0082] The second model described in S220 above is a trained model. Optionally, in some implementations, a training method for obtaining the second model can be executed before executing S220 above. Next, the training method for obtaining the second model provided by the embodiments of this application will be introduced.

[0083] Optionally, in some implementations, before performing the above S220, that is, before using the second model to recognize the second text to be recognized and obtaining the second recognition result, the method further includes: using the second training set to adjust the parameters of the second initial encoder and the parameters of the second classifier included in the second initial model to obtain the second model, wherein the second training set includes the second sample and the real labels of the second sample, the second sample includes information for describing the emotional state of the agent at the first preset historical moment, and the real label of the second sample represents the emotional state of the agent at the first preset historical moment.

[0084] The training method described above, which uses a second training set to train the second initial model to obtain the second model, includes the second sample and its true labels. This means the training process is supervised, ensuring the accuracy of the obtained second model. Furthermore, it can improve the accuracy of the second recognition result obtained by using the second model to recognize the second document to be identified, thereby improving the accuracy of the sentiment recognition result.

[0085] The training data used in the above training process, namely the second training set, will be described in detail below. The second training set includes the second sample and the true labels of the second sample.

[0086] The second sample includes information describing the agent's emotional state at a first preset historical moment. The first preset historical moment is not specifically limited and can be set according to the actual scenario. Similarly, the information describing the agent's emotional state at the first preset historical moment is also not specifically limited. In some implementations, the information describing the agent's emotional state at the first preset historical moment can be what the agent said at that moment. Optionally, in other implementations, the information describing the agent's emotional state at the first preset historical moment can be the text written by the agent at that moment. For example, the second sample could be represented as: "I'm in a good mood today," where "I" refers to the agent.

[0087] The true label of the second sample represents the agent's emotional state at a first preset historical moment. For example, in the following scenario, the second sample can be represented as: "I am happy today," where "I" refers to the agent. In this scenario, the true label of the second sample can be "happy." It can be understood that the emotional state represented by the true label of the second sample is any one of the M types of emotional states output by the second classifier.

[0088] In this embodiment, the number of samples included in the second training set is not specifically limited. That is, the second training set may include multiple second samples and multiple real labels for the second samples, all of which are different. Any two different second samples can be understood as follows: the agents corresponding to these two different second samples are different; the state information of the agents corresponding to these two different second samples is different; or the preset historical time points corresponding to these two different second samples are different.

[0089] Next, we will introduce in detail the second initial model and the second model involved in the above model training process.

[0090] The second initial model includes a second initial encoder and a second classifier. The second initial encoder has feature extraction capabilities; the second classifier is an M-classifier, where M is greater than or equal to a second preset threshold. For example, Figure 3A Figure (2) shows a schematic diagram of the second initial model obtained by training the second initial model using the second training set, and the structure of the second initial model. In the embodiments of this application, the structure of the second initial encoder and the structure of the second initial classifier are not specifically limited. That is, any model structure with feature extraction can be used as the second initial encoder, and any classifier with M-classification function can be used as the second initial classifier. For example, the second initial encoder can be, but is not limited to, the encoder included in the Transformer model, or a variant of the encoder included in the Transformer model. For example, the second initial classifier can be, but is not limited to, a random forest classifier or a Naive Bayes classifier.

[0091] The second model is obtained by training the parameters of the second initial model using a second training set. Based on this, the second model may include a second encoder and a second classifier. The second encoder is the encoder obtained by adjusting the parameters of the second initial encoder as described above; the second classifier is the classifier obtained by adjusting the parameters of the second initial classifier as described above. For example, Figure 3A (2) shows a schematic diagram of the training process of the second initial model to obtain the second model by using the second training set.

[0092] The previous section detailed the second training set, the second initial model, and the second model. Below, we will combine... Figure 5 The training method 500, which involves "adjusting the parameters of the second initial encoder and the second classifier included in the second initial model using the second training set to obtain the second model," is described in detail above. See [link to relevant documentation]. Figure 5 The method 500 includes steps S510 to S540. Steps S510 to S540 will be described in detail below.

[0093] S510: Use the second initial encoder to extract features from the second sample to obtain the feature vector of the second sample.

[0094] Performing the above S510, that is, using the second initial encoder to extract features from the second sample and obtain the feature vector of the second sample, may include the following steps: inputting the second sample into the second initial encoder, and the second initial encoder outputting the feature vector of the second sample.

[0095] S520: The second initial classifier is used to classify the feature vector of the second sample to obtain the predicted label of the second sample.

[0096] In this embodiment, the recognition result output by the second initial classifier is any one of the M types of emotional states, wherein the second recognition result is any one of the M types of emotional states, and M is greater than or equal to a second preset threshold. In the above implementation, the second initial classifier is an M classifier. Based on this, the predicted label of the second sample described in S520 corresponds to one of the M types of emotional states. The value of M and the value of the second preset threshold are not specifically limited. It is understood that in the above implementation, the larger the value of M, the finer the granularity of the emotional recognition result, which can further improve the accuracy of the emotional recognition result. In some implementations, M can be equal to N, and both M and N are equal to 24. Figure 3B Twenty-four types of emotional states are shown. In traditional techniques, a 7-classifier is typically used to identify emotional states, and the output of the 7-classifier is one of the following: "fear," "happiness," "surprise," "anger," "sadness," "disgust," or "other." In the implementation provided in this application, the second initial classifier is a 24-classifier. This makes the granularity of the identification results output by the second classifier in the trained second model more refined, which is beneficial to improving the accuracy of emotion recognition results.

[0097] Performing S520 above, that is, using the second initial classifier to classify the feature vector of the second sample and obtain the predicted label of the second sample, may include the following steps: inputting the feature vector of the second sample into the second initial classifier, and the second initial classifier outputting the predicted label of the second sample.

[0098] S530, based on the difference between the true label of the second sample and the predicted label of the second sample, adjust the parameters of the second initial encoder and the second initial classifier.

[0099] The difference between the true label of the second sample and the predicted label of the second sample can be represented by a loss function. In this embodiment, the loss function can be a classification loss function. The form of the classification loss function is not specifically limited and can be selected according to actual needs. For example, the classification loss function can be, but is not limited to, negative log-likelihood loss or cross-entropy loss function.

[0100] S540, once the training reaches the second preset training conditions, stop adjusting the parameters of the second initial encoder and the second initial classifier to obtain the second model.

[0101] The second preset training condition is not specifically limited and can be set according to the actual scenario. In some implementations, the second preset training condition may include at least one of the following conditions: the number of training iterations of the second initial model meets a preset number of training iterations, the training time of the second initial model meets a preset training time, or the loss result of the second initial model (i.e., the value of the loss function of the second initial model) is less than a preset loss threshold. The preset number of training iterations, preset training time, and preset loss threshold are not specifically limited and can be set according to actual needs.

[0102] It should be noted that the training steps described in S510 to S540 above, namely "training the second initial model using the second training set to obtain the second model", are merely illustrative and do not constitute any limitation on the embodiments of this application. That is to say, other training steps besides those described in S510 to S540 above can also be performed on the second initial model using the second training set to obtain the second model.

[0103] It is understandable that the above Figure 5 The model training principles shown in S510 to S540 are the same as those described above. Figure 4 The training principles of models S410 to S440 shown are the same. The difference lies in... Figure 5 The second training set shown includes the second samples and Figure 4 The first training set shown includes different samples.

[0104] In the embodiments of this application, the execution order of S210 and S220 is not specifically limited. That is, in some implementations, S220 can be executed after S210. Optionally, in other implementations, S210 can be executed after S220.

[0105] S230, based on the attribute information, the first recognition result, and the second recognition result, determine the emotional state of the intelligent agent to be identified at the current moment, wherein the attribute information represents the personality characteristics of the intelligent agent to be identified, and the current moment is the moment after the second moment.

[0106] The attribute information, first recognition result, and second recognition result described in S230 above can be understood as information that influences the emotional state of the agent to be identified at the current moment. The purpose of performing S230 is to determine the emotional state of the agent to be identified at the current moment based on information from moments prior to the current moment. This information includes attribute information, the first recognition result, and the second recognition result. The personality traits of the agent to be identified are also referred to as the agent's persona. For example... Figure 6 This illustration shows a schematic diagram of an agent's persona emotion vector provided in an embodiment of this application. Figure 6 The human-like emotional vector of the agent shown can be used as attribute information of the aforementioned agent to be identified. For example... Figure 6 As shown, the agent's persona emotion vector can represent any of the following personas: a normal persona, a melancholic persona, or a slacker persona. The normal persona has a neutral emotion characteristic, and negative emotions have a higher priority than positive emotions. The melancholic persona has a sadness emotion characteristic, meaning sadness has a higher priority than anger. The slacker persona has an anger emotion characteristic, meaning anger has a higher priority than sadness.

[0107] The following describes the implementation method of "determining the emotional state of the intelligent agent to be identified at the current moment based on attribute information, the first identification result, and the second identification result" provided in the embodiments of this application.

[0108] In this embodiment, determining the emotional state of the intelligent agent to be identified at the current moment based on attribute information, a first identification result, and a second identification result includes: determining a target emotional intensity based on the attribute information and the first identification result, wherein the attribute information represents the personality trait of the intelligent agent to be identified as a target personality; determining the greater of the emotional intensity represented by the target emotional intensity and the negative emotional intensity represented by the second identification result; if the greater emotional intensity is greater than a preset confidence level, then determining that the emotional state of the intelligent agent to be identified at the current moment is consistent with the emotional tendency of the greater emotional intensity; wherein, if the positive emotional intensity represented by the first emotional state exceeds a target threshold, then the emotional state represented by the target emotional intensity is consistent with the emotional tendency represented by the first emotional state; if the positive emotional intensity represented by the first emotional state does not exceed the target threshold, then the emotional state represented by the target emotional intensity is opposite to the emotional tendency represented by the first emotional state; the target threshold is a threshold corresponding to the target personality, and the target personality is one of multiple candidate personalities, with different candidate personalities corresponding to different thresholds. No specific limitation is made to the above determination that the emotional state of the intelligent agent to be identified at the current moment is consistent with the emotional tendency of the greater emotional intensity. In some implementations, the larger of the two sentiment values ​​(represented by the target sentiment score and the negative sentiment score represented by the second recognition result) is determined as the greater sentiment score. In some implementations, the current emotional state of the agent to be identified is the sentiment with the greater sentiment score. For example, the greater sentiment score is "happy," and the current emotional state of the agent to be identified is "happy." Optionally, in some implementations, the current emotional state of the agent to be identified is a similar emotion to the greater sentiment score. For example, the greater sentiment score is "joyful," and the current emotional state of the agent to be identified is "happy." The value of the preset confidence level is not specifically limited and can be set according to actual needs. In the above implementations, the target sentiment score is first determined based on attribute information and the first recognition result generated based on external stimulus information. The sentiment state represented by the target sentiment score is determined based on the positive sentiment level represented by the first sentiment state corresponding to the first recognition result and a target threshold, where the target threshold is the threshold corresponding to the target personality. Then, the current emotional state of the agent to be identified is determined based on the target sentiment score and the second recognition result. In the above implementation, when determining the emotional state of the agent to be identified at the current moment, not only are the agent's second emotional state at the first moment and its first emotional state generated at the second moment after receiving external stimuli at the first moment considered, but the influence of the agent's personality traits on its emotional state is also taken into account. This improves the accuracy of the emotion recognition results. The number and type of multiple candidate personalities described in the above implementation are not specifically limited. That is, the number and type of multiple candidate personalities described in the above implementation can be set according to actual application requirements.Generally speaking, extroverted personalities are less affected by external stimuli, while neutral personalities are more affected. Therefore, in some implementations, multiple candidate personalities include neutral, extroverted, and introverted personalities, where the threshold for introverted personalities is greater than the threshold for neutral personalities, and the threshold for neutral personalities is greater than the threshold for extroverted personalities. It should be noted that the multiple candidate personalities described in the above implementations are merely examples and do not constitute any limitation on the embodiments of this application. For example, in some implementations, the multiple candidate personalities may also include an optimistic personality, where the threshold for optimistic personalities is greater than the threshold for extroverted personalities.

[0109] It should be understood that the above Figure 2 The emotion recognition method shown is for illustrative purposes only and does not constitute any limitation on the emotion recognition method provided in the embodiments of this application. For example, the agent described above can also be replaced by a user. Furthermore, the first model and the second model described above can also be the same model, which stores two sets of model parameters, wherein one set of model parameters is the model parameters possessed by the first model, and the other set of model parameters is the model parameters possessed by the second model.

[0110] In this embodiment, the emotion recognition method determines the emotional state of the target intelligent agent at the current moment based on attribute information, a first recognition result, and a second recognition result. That is, when determining the emotional state of the target intelligent agent at the current moment, it considers not only the emotional state of the target intelligent agent at a first moment before the current moment (i.e., the second recognition result), but also the emotional state generated by the target intelligent agent after being given external stimuli at the first moment (i.e., the first recognition result), as well as the attribute information of the target intelligent agent. This improves the accuracy of the emotion recognition result. Furthermore, the first recognition result is obtained using a trained first model, and the second recognition result is obtained using a trained second model. The first model includes an N-classifier, and the second model includes an M-classifier. When the values ​​of N and M are large, the first recognition result obtained based on the first model is more accurate, and the second recognition result obtained based on the second model is more accurate. Furthermore, the emotion recognition result determined based on the first and second recognition results is also more accurate.

[0111] Below, in conjunction with Figure 7 This application introduces another emotion recognition method provided by an embodiment. It is understood that... Figure 7 The emotion recognition method described above is... Figure 2 A specific example of the described emotion recognition method, Figure 7 The methods described are for illustrative purposes only and do not constitute any limitation on the emotion recognition methods provided in this application.

[0112] Figure 7 This is a schematic diagram illustrating another emotion recognition method provided in an embodiment of this application. The emotion recognition method provided in this embodiment can be executed by an execution device. It is understood that the execution device can be implemented as software, or a combination of software and hardware. For example, the execution device in this embodiment can be, but is not limited to, a server or a terminal device used by a user. Figure 7 As shown, the method includes steps S710 to S780. Steps S710 to S780 will be described in detail below.

[0113] S710, Obtain training dataset 1, wherein training dataset 1 includes training sample 1 and the real label of training sample 1. Training sample 1 includes text that stimulates the emotion of agent 1 in the external environment 1. The real label of training sample 1 represents the effect of the external environment 1 on the emotional state of agent 1 after stimulating agent 1.

[0114] The above training dataset 1 can be understood as the above... Figure 2 The following is a specific example of the first training set. Training dataset 1 includes training sample 1 and the ground truth labels for training sample 1. The training sample 1 and its ground truth labels are described below.

[0115] Training sample 1 includes text that stimulates agent 1 in the external environment 1 at the current moment. The external environment 1 is not specifically limited; it can be a natural environment or an artificial environment. In this embodiment, the external environment 1 may include at least one of the following: other agents besides agent 1, the natural environment in which agent 1 is located, or a user interacting with agent 1. The content of the text with the above-mentioned properties included in training sample 1 is not specifically limited and can be set according to actual needs.

[0116] The true label of training sample 1 represents the impact of external environment 1 on agent 1's emotional state after stimulation. Specifically, the impact of external environment 1 on agent 1's emotional state can be positive or negative. Below, examples are provided to illustrate training sample 1 and its true label. For instance, in some scenarios, external environment 1 can be the user interacting with agent 1. In this scenario, training sample 1 could be: "You're so smart." In this scenario, the true label of training sample 1 could be: "Happy." Table 1 below illustrates, for example, the impact of different stimulus categories corresponding to external environment 1 on agent 1's emotional state.

[0117] Table 1

[0118]

[0119] In Table 1 above, the impact on the emotional state of agent 1 is represented by numerical values, where a larger number corresponds to an emotion, the deeper the degree of that emotion. For example, taking the stimulus category corresponding to external environment 1 in Table 1 as dialogue-praise, agent 1 will be given a stimulus of "happiness" at a degree of "2". Similarly, taking the stimulus category corresponding to external environment 1 in Table 1 as dialogue-aggression, agent 1 will be given a stimulus of "anger" at a degree of "2" and a stimulus of "fear" at a degree of "2".

[0120] In this embodiment of the application, the number of training samples included in the training dataset 1 is not specifically limited. That is, the training dataset 1 may include at least one training sample 1 and at least one true label of training sample 1. Optionally, the training dataset 1 may also include multiple training samples 1 and multiple true labels of training samples 1, wherein any two training samples 1 among the multiple training samples 1 are different, that is, all the multiple training samples 1 are different.

[0121] In this embodiment of the application, the method for obtaining training dataset 1 described in S710 above is not specifically limited. For example, when the execution device performs... Figure 7 In the scenario illustrated by the emotion recognition method, the execution device can directly perform the data acquisition process to obtain the dataset acquisition process described in S710 above. For example, when the execution device performs... Figure 7 In the scenario of the emotion recognition method shown, the data acquisition process can also be performed by other devices that communicate with the execution device to obtain the training dataset 1 described in S710 above. After that, the execution device can obtain the training dataset 1 from the other devices that communicate with it.

[0122] S720, the initial external stimulus model is trained using training dataset 1 to obtain the external stimulus model, wherein the initial external stimulus model includes the initial encoder A and the initial emotion classifier A.

[0123] The above initial external stimulus model can be understood as the above Figure 2 A specific example of the first initial model in the described method; the initial encoder A and the initial sentiment classifier A mentioned above can be understood as the aforementioned... Figure 2 The first initial encoder and the first initial classifier in the described method; the external stimulus model described above can be understood as the above Figure 2 A specific example of the first model is shown.

[0124] The purpose of executing S720 is to train the initial external stimulus model using the training dataset 1 obtained in S710, thereby obtaining a trained external stimulus model. It is understood that the model structure of the initial external stimulus model described in S720 is the same as the model structure of the external stimulus model; the difference lies in the model parameters. It is understood that the external stimulus model includes encoder A and sentiment classifier A, where the difference between encoder A and the initial encoder A lies in the parameters and / or the values ​​of the parameters; the difference between classifier A and the initial classifier A lies in the parameters and / or the values ​​of the parameters. In other words, the structure of encoder A is the same as the structure of the initial encoder A. The structure of classifier A is the same as the structure of the initial classifier A.

[0125] The initial encoder A and the initial sentiment classifier A involved in the above S720 will be described in detail below.

[0126] The initial encoder A is used to extract features from training sample 1 included in training dataset 1 to obtain the feature vector of training sample 1. The feature vector of training sample 1 represents training sample 1. That is, training sample 1 can be generated based on the feature vector of training sample 1. In this embodiment, the type of the initial encoder A is not specifically limited; the initial encoder A only needs to have feature extraction functionality, and its type can be selected according to actual application requirements. For example, the initial encoder A included in the initial external stimulus model can be, but is not limited to, the encoder included in the Transformer model, or a variant of the encoder included in the Transformer model.

[0127] The initial sentiment classifier A is a 24-classifier. Specifically, the output of the initial sentiment classifier A is any one of 24 sentiment states. For example, Figure 3B Any one of the 24 emotional states described in the two-dimensional emotional coordinates shown can be used as the output of the initial emotional classifier A.

[0128] In this embodiment, performing S720 may specifically include S720-1 to S720-4. Next, S720 will be described in detail with reference to S720-1 to S720-4.

[0129] S720-1 uses the initial encoder A, which is included in the initial external stimulus model, to extract features from the training sample 1 included in the training dataset 1, and obtains the feature vector of the training sample 1.

[0130] S720-2 uses the initial emotion classifier A included in the initial external stimulus model to classify the feature vector of training sample 1 and obtain the predicted label of training sample 1.

[0131] The predicted label of training sample 1 represents the emotional state of agent 1 associated with training sample 1, as predicted by the initial external stimulus model when performing emotion recognition on training sample 1.

[0132] The implementation process of S720-1 and S720-2 described below is illustrated with examples. For instance, in some implementations, training sample 1 can be: "You are so smart," and the true label of training sample 1 can be "Happy." Based on this, the input data of the initial encoder A can be: "You are so smart," and the input data label can be: "Happy"; the output data of the initial encoder A can be the emotional stimulus feature vector of "You are so smart"; the input of the initial emotion classifier A can be the emotional stimulus feature vector of "You are so smart"; and the output of the initial emotion classifier A can be: "Happy."

[0133] S720-3, adjust the parameters of the initial external stimulus model based on the difference between the predicted label of training sample 1 and the true label of training sample 1 included in training dataset 1.

[0134] In this embodiment, the loss function A can be used to represent the difference between the predicted label of training sample 1 and the true label of training sample 1 included in training dataset 1. That is, by performing S720-3 above, the parameters of the initial external stimulus model are adjusted to minimize the loss function A, such that the difference between the predicted label of training sample 1 and the true label of training sample 1 included in training dataset 1 satisfies the preset training condition A. The loss function A can be a multi-class loss function, for example, it can be, but is not limited to, a multi-class cross-entropy loss function.

[0135] In performing the method described in S720-3, the parameters of the initial external stimulus model are adjusted, including: adjusting the parameters of the initial encoder A included in the initial external stimulus model; and adjusting the parameters of the initial emotion classifier A included in the initial external stimulus model.

[0136] S720-4: When the training reaches the preset training condition A, stop adjusting the parameters of the initial external stimulus model to obtain the external stimulus model.

[0137] The aforementioned preset training condition A can be understood as the above Figure 2 A specific example of the first preset training condition in the described method.

[0138] In this embodiment, the preset training condition A is not specifically limited, and can be set according to actual needs. In some implementations, the preset training condition A may include any of the following conditions: the number of training iterations of the initial external stimulus model is greater than a preset number of training iterations, the loss value corresponding to the output result of the initial external stimulus model is within a preset error range, or the output result of the initial external stimulus model reaches a preset recognition accuracy. The preset number of training iterations, preset error range, and preset recognition accuracy can be selected according to actual needs, and this embodiment does not specifically limit them.

[0139] Executing S710 and S720 above constitutes the process of training the initial external stimulus model using training dataset 1 to obtain a trained external stimulus model. It is understood that, in this embodiment, the external stimulus model obtained after executing S710 and S720 serves to determine the impact of the external environment on the agent's emotional state after the agent is stimulated by the external environment.

[0140] S730, Obtain training dataset 2, wherein training dataset 2 includes training sample 2 and the real label of training sample 2. Training sample 2 includes text used to represent the emotional state of agent 1 at the current moment, and the real label of training sample 2 represents the emotional state of agent 1 at the current moment.

[0141] The above training dataset 2 can be understood as the above... Figure 2 A specific example of the second training set in the described method; the above training sample 2 can be understood as the above Figure 2 A specific example of the second sample in the described method; the true label of the above training sample 2 can be understood as the above. Figure 2 A specific example of the true label of the second sample in the described method.

[0142] In this embodiment of the application, the training sample 2 includes text representing the emotional state of agent 1 after being stimulated by the external environment 1. Specifically, the text representing the emotions described by agent 1 after being stimulated by the external environment 1 can be considered as the emotional state of agent 1 after being stimulated by the external environment 1.

[0143] The true label of training sample 2 represents the emotional state described by agent 1 after being stimulated by external environment 1. There are no specific limitations on training sample 2 and its true label; they can be set according to the actual application scenario. For example, training sample 2 could be: "I am in a good mood today," and its true label could be: "happy." Or, training sample 2 could be: "I am in a bad mood today," and its true label could be: "sad."

[0144] In this embodiment of the application, the number of training samples included in the training dataset 2 is not specifically limited. That is, the training dataset 2 may include at least one training sample 2 and at least one true label for the training sample 2. Optionally, the training dataset 2 may also include multiple training samples 2 and multiple true labels for the training samples 2, wherein any two training samples 2 among the multiple training samples 2 are different, that is, all the multiple training samples 2 are different.

[0145] In this embodiment, the method for acquiring training dataset 2 described in S730 above is not specifically limited. For example, when the execution device performs... Figure 7 In the scenario illustrated by the emotion recognition method, the execution device can directly perform the data acquisition process to obtain the dataset acquisition process described in S730 above. For example, when the execution device performs... Figure 7 In the scenario of the emotion recognition method shown, the data acquisition process can also be performed by other devices that communicate with the execution device to obtain the training dataset 2 described in S730 above. After that, the execution device can obtain the training dataset 2 from the other devices that communicate with it.

[0146] S740, the initial emotion recognition model is trained using training dataset 2 to obtain the emotion recognition model, wherein the initial emotion recognition model includes the initial encoder B and the initial emotion recognition classifier B.

[0147] The initial emotion recognition model described above can be understood as the above... Figure 2 A specific example of the second initial model in the described method; the aforementioned initial encoder B and initial sentiment classifier B can be understood as the aforementioned... Figure 2 The described method includes a second initial encoder and a second initial classifier; the aforementioned emotion recognition model can be understood as the above. Figure 2 The following is a specific example of the second model. It can be understood that the emotion recognition model includes encoder B and emotion classifier B, where encoder B differs from the initial encoder B in the parameters and / or their values; classifier B differs from the initial classifier B in the parameters and / or their values. That is, the structure of encoder B is the same as the structure of the initial encoder B. The structure of classifier A is the same as the structure of the initial classifier B.

[0148] The purpose of executing S740 above is to train the initial emotion recognition model using the training dataset 3 obtained in S730 above, in order to obtain a trained emotion recognition model. It is understood that the model structure of the initial emotion recognition model described in S740 is the same as the model structure of the emotion recognition model; the difference lies in the model parameters of the initial emotion recognition model and the model parameters of the emotion recognition model.

[0149] The initial encoder B and the initial sentiment classifier B involved in the above S740 are described below.

[0150] The initial encoder B is used to extract features from training sample 2 included in training dataset 2 to obtain the feature vector of training sample 2.

[0151] The initial sentiment classifier B is a 24-classifier, and its output is any one of the 24 sentiment states. The 24 sentiment states corresponding to the output of the initial sentiment classifier B can be the same as the 24 sentiment states corresponding to the output of the initial sentiment classifier A mentioned above; for a more detailed explanation, please refer to the relevant description above.

[0152] In this embodiment of the application, the execution of S740 may specifically include S740-1 to S740-4. Next, S740 will be described in detail with reference to S740-1 to S740-4.

[0153] S740-1 uses the initial encoder B included in the initial emotion recognition model to extract features from the training samples 2 included in the training dataset 2, and obtains the feature vector of the training samples 2.

[0154] S740-2 uses the initial sentiment classifier B included in the initial sentiment recognition model to classify the feature vector of training sample 2 and obtain the predicted label of training sample 2.

[0155] The predicted label of training sample 2 represents the emotional state of agent 1 associated with training sample 2, as predicted by the initial emotion recognition model when performing emotion recognition on training sample 2.

[0156] The implementation process of S740-1 and S740-2 described below is illustrated with examples. For instance, in some implementations, training sample 2 can be "I'm in a good mood today," and the true label of training sample 1 can be "happy." Based on this, the input data of the initial encoder B can be "I'm in a good mood today," and the input data label can be "happy"; the output data of the initial encoder B can be the emotion feature vector of "I'm in a good mood today"; the input of the initial sentiment classifier B can be the emotion feature vector of "I'm in a good mood today"; and the output of the initial sentiment classifier B can be "happy."

[0157] S740-3, The parameters of the initial sentiment recognition model are adjusted based on the difference between the predicted labels of training sample 2 and the true labels of training sample 2 included in training dataset 2.

[0158] In this embodiment, the loss function B can be used to represent the difference between the predicted label of training sample 2 and the true label of training sample 2 included in training dataset 2. That is, by performing S720-3 above, the parameters of the initial sentiment recognition model are adjusted to minimize the loss function B, such that the difference between the predicted label of training sample 2 and the true label of training sample 2 included in training dataset 2 satisfies the preset training condition B. The loss function B can be a multi-class loss function, for example, it can be, but is not limited to, a multi-class cross-entropy loss function.

[0159] In executing the method described in S740-3, the parameters of the initial emotion recognition model are adjusted, including: adjusting the parameters of the initial encoder B included in the initial emotion recognition model; and adjusting the parameters of the initial emotion classifier B included in the initial emotion recognition model.

[0160] S740-4, Once the training reaches the preset training condition B, stop adjusting the parameters of the initial emotion recognition model to obtain the emotion recognition model.

[0161] The aforementioned preset training condition B can be understood as the above. Figure 2 A specific example of the second preset training condition in the described method.

[0162] In this embodiment, the preset training condition B is not specifically limited, and can be set according to actual needs. In some implementations, the preset training condition B may include any of the following conditions: the number of training iterations of the initial emotion recognition model is greater than a preset number of training iterations, the loss value corresponding to the output result of the initial emotion recognition model is within a preset error range, or the output result of the initial emotion recognition model reaches a preset recognition accuracy. The preset number of training iterations, preset error range, and preset recognition accuracy can be selected according to actual needs, and this embodiment does not specifically limit them.

[0163] Executing S730 and S740 above constitutes the process of training the initial emotion recognition model using training dataset 2 to obtain a trained emotion recognition model. It is understood that, in this embodiment, the emotion recognition model obtained after executing S730 and S740 serves to determine the agent's emotional state based on the agent's expressed emotional state.

[0164] The methods for generating external stimulus models and emotion recognition models provided in the embodiments of this application have been introduced above, with reference to S710 to S740. Next, the application stage based on the external stimulus models and emotion recognition models will be described.

[0165] S750, acquire the text to be identified 1, and process the text to be identified 1 using the external stimulus model to obtain the emotional stimulus result corresponding to the text to be identified 1. The text to be identified 1 is the text that affects the emotional state of agent 1 after the external environment 2 stimulates agent 1 at time 1.

[0166] The above-mentioned text to be identified 1 can be understood as the above-mentioned text. Figure 2 The first text to be identified in the described method.

[0167] The external stimulus model described in S750 above is a trained model obtained by training dataset 1. Based on this, by processing the text 1 to be identified using the external stimulus model, the emotional stimulus result corresponding to the text 1 to be identified can be obtained.

[0168] The text to be identified 1 is the text that affects the emotional state of agent 1 after the external environment 2 stimulates agent 1 at time 1. The text to be identified 1 may include the content that affects the emotional state of agent 1 after the external environment 2 stimulates agent 1 at time 1.

[0169] The emotional stimulus result corresponding to the text to be identified represents the emotional state of agent 1 after the external environment 2 stimulates agent 1 at time 1. This emotional state can be... Figure 3BAny one of the 24 emotional states shown.

[0170] In this embodiment, the external environment 2 is not specifically limited; that is, the external environment 2 can be a positive or negative environment. Generally, when agent 1 receives a positive external environment 2, agent 1's emotions will be positively stimulated. For example, the document to be identified 1 may specifically include the following content: "You are so smart." Based on this, the emotional stimulus result corresponding to the document to be identified 1 can be: "Happy".

[0171] S760, obtain the file to be identified 2, and use the emotion recognition model to process the file to be identified 2 to obtain the emotion recognition result corresponding to the text to be identified 2, wherein the text to be identified 2 is the text describing the emotional state of agent 1 at time 1.

[0172] The above-mentioned text to be identified, 2, can be understood as the above-mentioned text. Figure 2 The second text to be identified in the described method.

[0173] The sentiment recognition model described in S760 above is a trained model obtained by training on training dataset 2. Based on this, by processing the text 2 to be recognized using the sentiment recognition model, the sentiment recognition result corresponding to the text 2 to be recognized can be obtained.

[0174] The text to be identified, 2, is a description of the emotional state of agent 1 at time 1. Specifically, the words spoken by agent 1 at time 1 can be used to describe the emotional state of agent 1 at time 1. In this embodiment, the words spoken by agent 1 at time 1 can be generated by a language model based on the emotional state of agent 1 at time 1. The type of emotional state of agent 1 described by the text to be identified, 2, is not specifically limited; that is, the type of emotional state of agent 1 described by the text to be identified, 2, can be any... Figure 3B Any one of the 24 emotional states shown.

[0175] The emotion recognition result corresponding to the text to be recognized 2 represents the emotional state of agent 1 at time 1.

[0176] S770, obtain the personality emotion vector of agent 1, where the personality emotion vector of agent 1 represents the personality traits of agent 1.

[0177] The aforementioned emotional vector of agent 1 can be understood as the above... Figure 2 A specific example of the attribute information of the agent to be identified in the described method.

[0178] The persona emotion vector of agent 1 represents the personality traits of agent 1, where the personality traits of agent 1 include both intrinsic and extrinsic characteristics. For example, Figure 6 This illustration shows a schematic diagram of an agent's persona emotion vector provided in an embodiment of this application. Figure 6 The emotional vector of the agent shown can be used as the personality trait of agent 1 mentioned above. Here, for... Figure 6 For details not elaborated in detail, please refer to section S230 above. Figure 6 The description.

[0179] In this embodiment, the method for obtaining the persona emotion vector of agent 1 is not specifically limited. For example, in some scenarios, the device executing the emotion recognition method provided in this embodiment stores the persona emotion vector of agent 1 in its storage space. Thus, the device can obtain the persona emotion vector of agent 1 by accessing its own storage space. Alternatively, in other scenarios, the persona emotion vector of agent 1 is stored in the database of a remote server. Thus, the device executing the emotion recognition method provided in this embodiment can successfully read the persona emotion vector of agent 1 from the database of the remote server by interacting with the remote server.

[0180] S780, based on the emotional stimulus result corresponding to the text to be identified 1, the emotional recognition result corresponding to the text to be identified 2, and the personified emotional vector of agent 1, determine the emotional state of agent 1 at time 2.

[0181] In practical applications, human emotions are influenced by many factors, including the environment and the individual's own state. Emotional transfer is more complex, requiring careful definition of the quantity (quantitative) of emotional transfer and the distinction between quantitative and qualitative changes. Furthermore, human emotions are also influenced by background knowledge. For example, if I dislike playing basketball while Zhang San enjoys it, our emotional reactions to the lost basketball will differ significantly. Therefore, to improve the accuracy of emotion recognition results, this embodiment executes the method described in S780 above. This involves determining the emotional state of agent 1 at time 2 based on the emotional stimulus result received by agent 1 from external environment 2 at time 1, the emotion recognition result corresponding to agent 1's own emotional state described at time 1, and agent 1's personality emotion vector. In other words, the above implementation considers not only the impact of external stimuli on agent 1's emotional state at time 1, but also the emotional state expressed by agent 1 at time 1 and agent 1's personality traits. This makes the emotion recognition result of agent 1 at time 2, determined based on the above method, more accurate.

[0182] The emotional stimulus result corresponding to the text to be identified 1 and the emotional recognition result corresponding to the text to be identified 2 described in S780 above are both... Figure 3B Any one of the four emotional states shown. For ease of description, the emotional state corresponding to the emotional stimulus result of the text to be identified 1 is abbreviated as emotional state A, and the emotional state corresponding to the emotional recognition result of the text to be identified 2 is abbreviated as emotional state B. In this embodiment, the persona of agent 1 described by the persona emotional vector of agent 1 is not specifically limited. In other words, in this embodiment, the persona of agent 1 described by the persona emotional vector of agent 1 can be any of the following personas: a normal persona, a melancholic persona, or a slacker persona.

[0183] Next, based on the character settings of agent 1 described in implementation methods one through three, the method described in S780 above will be introduced. Specifically, agent 1 described in implementation method one is a normal character; agent 1 described in implementation method one is a melancholic character; and agent 1 described in implementation method one is a scoundrel character.

[0184] Implementation Method 1:

[0185] In implementation method one, the persona emotion vector of agent 1 indicates that agent 1 has a normal persona. That is to say, in this implementation method, the emotional state of agent 1 tends to be neutral.

[0186] In implementation method one, executing S780 above, that is, determining the emotional state of agent 1 at time 2 based on the emotional stimulus result corresponding to the text to be identified 1, the emotional recognition result corresponding to the text to be identified 2, and the personified emotional vector of agent 1, may include the following steps: if the difference between emotional state A and emotional state B is less than a preset difference, determine that the emotional state of agent 1 at time 2 is emotional state A or emotional state B; or, if the degree of positive emotional state A exceeds a preset threshold 1, and the degree of negative emotional state B does not exceed a preset threshold 2, determine that the emotional state of agent 1 at time 2 is emotional state A; or, if the degree of positive emotional state A does not exceed a preset threshold 1, and the degree of negative emotional state B exceeds a preset threshold 2, determine that the emotional state of agent 1 at time 2 is emotional state B.

[0187] In this embodiment, the values ​​of the preset threshold 1 and preset threshold 2 are not specifically limited. In some implementations, the values ​​of the preset threshold 1 and preset threshold 2 may both be equal to zero.

[0188] Implementation Method Two:

[0189] In implementation method two, the emotional vector of agent 1 represents that agent 1 is a melancholic and sentimental persona. In other words, in this implementation method, agent 1's emotional state is more inclined towards sadness and sentimentality.

[0190] In implementation method two, executing S780 above, that is, determining the emotional state of agent 1 at time 2 based on the emotional stimulus result corresponding to the text to be identified 1, the emotional recognition result corresponding to the text to be identified 2, and the personified emotional vector of agent 1, may include the following steps: if the degree of positive emotion corresponding to emotional state A exceeds a preset threshold 1', and the degree of negative emotion corresponding to emotional state B does not exceed a preset threshold 2', then the emotional state of agent 1 at time 2 is determined to be emotional state A; or, if the degree of positive emotion corresponding to emotional state A does not exceed a preset threshold 1', and the degree of negative emotion corresponding to emotional state B exceeds a preset threshold 2', then the emotional state of agent 1 at time 2 is determined to be emotional state A. In this embodiment, the preset threshold 1' may be greater than the preset threshold 1, and the preset threshold 2' may be greater than the preset threshold 2. In some implementations, the values ​​of the preset threshold 1 and the preset threshold 2 may both be equal to zero, and the values ​​of the preset threshold 1' and the preset threshold 2' may both be equal to 2.

[0191] Implementation method three:

[0192] In implementation method three, the persona emotion vector of agent 1 represents agent 1 as a scoundrel. In other words, in this implementation method, agent 1's emotional state is more inclined to be angry and furious.

[0193] In implementation method three, executing S780 above, that is, determining the emotional state of agent 1 at time 2 based on the emotional stimulus result corresponding to the text to be identified 1, the emotional recognition result corresponding to the text to be identified 2, and the personified emotional vector of agent 1, may include the following steps: if the degree of positive emotion corresponding to emotional state A exceeds a preset threshold #1, and the degree of negative emotion corresponding to emotional state B does not exceed a preset threshold #2, then the emotional state of agent 1 at time 2 is determined to be emotional state A; or, if the degree of positive emotion corresponding to emotional state A does not exceed a preset threshold #1, and the degree of negative emotion corresponding to emotional state B exceeds a preset threshold #2, then the emotional state of agent 1 at time 2 is determined to be emotional state B. In this embodiment of the application, the preset threshold #1 may be greater than the preset threshold 1', and the preset threshold #2 may be less than the preset threshold 2'. In some implementations, the values ​​of the aforementioned preset threshold 1 and preset threshold 2 can both be equal to zero; the values ​​of the aforementioned preset threshold 1' and preset threshold 2' can both be equal to 2; the value of the aforementioned preset threshold #1 can be equal to 3, and the value of the aforementioned preset threshold #2 can be equal to 1.5.

[0194] It should be understood that the above Figure 7 The illustrated emotion recognition method is for illustrative purposes only and does not constitute any limitation on the emotion recognition method provided in the embodiments of this application. For example, the text to be recognized 1 described in S750 above can also be replaced with speech to be recognized. Furthermore, semantic recognition is performed on the speech to be recognized to obtain the text to be recognized 1. As another example, both the initial emotion classifier A and the initial emotion classifier B are 24 classifiers. Optionally, in some other application scenarios, the initial emotion classifier A and the initial emotion classifier B can be more or fewer classifiers.

[0195] In this embodiment, the emotional state of agent 1 at time 2 is determined based on the emotional stimulus result corresponding to the stimulus received by agent 1 from external environment 2 at time 1, the emotional recognition result corresponding to the emotional state described by agent 1 at time 1, and the personified emotional vector of agent 1. That is, the above implementation not only considers the impact of external stimulus on the emotional state of agent 1 at time 1, but also considers the emotional state expressed by agent 1 at time 1, as well as the personality traits of agent 1. This makes the emotional recognition result of agent 1 at time 2 determined by the above method more accurate. The emotional stimulus result is obtained using a trained external stimulus model, and the emotional recognition result is obtained using a trained emotional recognition model. Specifically, the emotional classifier A in the external stimulus model is a 24-classifier, and the emotional classifier B in the emotional recognition model is a 24-classifier. This makes the obtained emotional stimulus model result and emotional recognition result more accurate, which is beneficial to improving the accuracy of the emotional recognition result.

[0196] The above, combined with Figures 1 to 7 This paper details the application scenarios and methods applicable to the emotion recognition method provided in this application. Below, we will combine... Figure 8 and Figure 11 This application introduces the emotion recognition device, training device, execution device, and training equipment. It should be understood that the emotion recognition method mentioned above corresponds to the emotion recognition device and execution device mentioned below. The model training method mentioned above corresponds to the training device and training equipment mentioned below. For details not described in detail below, please refer to the relevant descriptions in the above method embodiments.

[0197] Corresponding to the emotion recognition method provided in the embodiments of this application, the embodiments of this application provide an emotion recognition device.

[0198] Figure 8 This is a schematic diagram of the structure of an emotion recognition device provided in an embodiment of this application. Figure 8 As shown, the emotion recognition device includes a first recognition unit 801, a second recognition unit 802, and a determination unit 803. The function of each of the first recognition unit 801, the second recognition unit 802, and the determination unit 803 will be described in detail below.

[0199] The first identification unit 801 is configured to: identify a first text to be identified using a first model to obtain a first identification result, wherein the first text to be identified includes first external stimulus information given to the intelligent agent to be identified at a first moment, and the first identification result indicates that the first external stimulus information causes the intelligent agent to generate a first emotional state at a second moment after the first moment; the second identification unit 802 is configured to: identify a second text to be identified using a second model to obtain a second identification result, wherein the second text to be identified includes information describing the second emotional state of the intelligent agent to be identified at the first moment, and the second identification result indicates the second emotional state; the determining unit 803 is configured to: determine the emotional state of the intelligent agent to be identified at the current moment based on attribute information, the first identification result, and the second identification result, wherein the attribute information represents the personality traits of the intelligent agent to be identified, and the current moment is a moment after the second moment.

[0200] Corresponding to the model training method provided in the embodiments of this application, the embodiments of this application provide a training device.

[0201] Figure 9 This is a schematic diagram of the structure of a training device provided in an embodiment of this application. Figure 9 As shown, the training device includes an acquisition unit 901 and a training unit 902. The functions of the acquisition unit 901 and the training unit 902 will be described below.

[0202] The above Figure 9 The training apparatus shown can be used to train a first initial model using a first training set to obtain a first model. The roles of the acquisition unit 901 and the training unit 902 in this training method are described in detail below.

[0203] The acquisition unit 901 is used to: acquire a first training set, wherein the first training set includes a first sample and a real label of the first sample, the first sample includes external stimulus information given to the agent at a first preset historical moment, and the real label of the first sample represents the emotional state generated by the agent at a second preset historical moment after the first preset historical moment due to the external stimulus information given to the agent; the training unit 902 is used to: adjust the parameters of the first initial encoder and the parameters of the first initial classifier included in the first initial model using the first training set to obtain the first model.

[0204] The above Figure 9 The training apparatus shown can be used to train a second initial model using a second training set to obtain a second model. The roles of the acquisition unit 901 and the training unit 902 in this training method are described in detail below.

[0205] The acquisition unit 901 is used to: acquire a second training set, wherein the second training set includes a second sample and a real label of the second sample, the second sample includes information describing the emotional state of the agent at a first preset historical moment, and the real label of the second sample represents the emotional state of the agent at the first preset historical moment; the training unit 902 is used to: adjust the parameters of the second initial encoder and the parameters of the second classifier included in the second initial model using the second training set to obtain the second model.

[0206] Corresponding to the model training method provided in the embodiments of this application, the embodiments of this application also provide an execution device.

[0207] Figure 10 This is a schematic diagram of an execution device structure provided in an embodiment of this application. The execution device 1000 can be equipped with the above-mentioned... Figure 8 The emotion recognition device described in the corresponding embodiment is used to implement the above. Figure 2 The described method, and Figure 7 The illustrated model application phase describes the various steps of the method. Specifically, the execution device 1000 is implemented by one or more servers. The execution device 1000 can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 1022 (e.g., one or more processors) and memory 1032, and one or more storage media 1030 (e.g., one or more mass storage devices) for storing application programs 1042 or data 1044. The memory 1032 and storage media 1030 can be temporary or persistent storage. The program stored in the storage media 1030 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the training device. Furthermore, the CPU 1022 may be configured to communicate with the storage media 1030 and execute the series of instruction operations in the storage media 1030 on the execution device 1000.

[0208] The execution device 1000 may also include one or more power supplies 1026, one or more wired or wireless network interfaces 1050, one or more input / output interfaces 1058, and / or one or more operating systems 1041, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0209] Corresponding to the emotion recognition method provided in the embodiments of this application, the embodiments of this application also provide a training device.

[0210] Figure 11 This is a schematic diagram of the structure of a training device provided in an embodiment of this application. The training device 1100 can specifically be a virtual reality (VR) device, a mobile phone, a tablet, a laptop computer, a smart wearable device, a monitoring data processing device, or a radar data processing device, etc., and is not limited thereto. The training device 1100 may be equipped with... Figure 9 The training device described in the corresponding embodiment is used to implement Figure 4 The described methods and Figure 5 The described method, and Figure 7 The illustrated model training phase describes the various steps of the method. Specifically, the training device 1100 includes: a receiver 1101, a transmitter 1102, a processor 1103, and a memory 1104 (wherein the number of processors 1103 in the training device 1100 can be one or more). Figure 11 (Taking a processor as an example), processor 1103 may include application processor 11031 and communication processor 11032. In some embodiments of this application, receiver 1101, transmitter 1102, processor 1103 and memory 1104 may be connected via bus or other means.

[0211] Memory 1104 may include read-only memory and random access memory, and provides instructions and data to processor 1103. A portion of memory 1104 may also include non-volatile random access memory (NVRAM). Memory 1104 stores processor and operation instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0212] Processor 1103 controls the operation of the execution device. In specific applications, the various components of the execution device are coupled together through a bus system, which may include not only the data bus, but also power buses, control buses, and status signal buses. However, for clarity, all buses are referred to as the bus system in the diagram.

[0213] The methods disclosed in the embodiments of this application can be applied to or implemented by the processor 1103. The processor 1103 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 1103 or by instructions in software form. The processor 1103 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and may further include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor 1103 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 1104. Processor 1103 reads the information in memory 1104 and, in conjunction with its hardware, completes the steps of the above method.

[0214] Receiver 1101 can be used to receive input digital or character information, and to generate signal inputs related to the settings and function control of the execution device. Transmitter 1102 can be used to output digital or character information through the first interface; transmitter 1102 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; transmitter 1102 may also include a display device such as a display screen.

[0215] This application provides a computer-readable storage medium, which includes computer instructions. When executed by a processor, the computer instructions are used to implement the technical solution of any emotion recognition method in this application.

[0216] This application provides a computer-readable storage medium, which includes computer instructions. When executed by a processor, the computer instructions are used to implement the technical solution of any training method in this application.

[0217] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored on a computer-readable medium and includes several instructions to cause a computing device (which may be a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments disclosed in this application.

[0218] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0219] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0220] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage media, or any other non-transferable media that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0221] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0222] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

Claims

1. An emotion recognition method, characterized in that, The method includes: The first model is used to identify the first text to be identified, and a first identification result is obtained. The first text to be identified includes external stimulus information given to the intelligent agent to be identified at a first moment. The first identification result indicates that the external stimulus information causes the intelligent agent to generate a first emotional state at a second moment after the first moment. The second model is used to identify the second text to be identified, and a second identification result is obtained. The second text to be identified includes information describing the second emotional state of the agent to be identified at the first moment. The second identification result represents the second emotional state. Based on the attribute information, the first identification result, and the second identification result, the emotional state of the intelligent agent to be identified at the current moment is determined, wherein the attribute information represents the personality characteristics of the intelligent agent to be identified, and the current moment is the moment after the second moment; Determining the emotional state of the agent to be identified at the current moment based on attribute information, the first identification result, and the second identification result includes: Based on the attribute information and the first recognition result, the target sentiment score is determined, wherein the attribute information indicates that the personality traits of the agent to be identified are the target personality. Determine the larger of the sentiment scores represented by the target sentiment score and the negative sentiment score represented by the second recognition result; If the greater sentiment score is greater than the preset confidence level, then it is determined that the emotional state of the agent to be identified at the current moment is consistent with the sentiment tendency of the greater sentiment score. Wherein, if the positive emotional level represented by the first emotional state exceeds the target threshold, then the emotional state represented by the target emotional level is consistent with the emotional tendency represented by the first emotional state; if the positive emotional level represented by the first emotional state does not exceed the target threshold, then the emotional state represented by the target emotional level is opposite to the emotional tendency represented by the first emotional state; the target threshold is the threshold corresponding to the target personality, and the target personality is one of multiple candidate personalities, with different candidate personalities corresponding to different thresholds.

2. The method according to claim 1, characterized in that, Before using the first model to recognize the first text to be recognized and obtaining the first recognition result, the method further includes: The parameters of the first initial encoder and the first initial classifier, which are included in the first initial model, are adjusted using the first training set to obtain the first model. The first training set includes a first sample and the real label of the first sample. The first sample includes external stimulus information given to the agent at a first preset historical time. The real label of the first sample represents the emotional state generated by the agent at a second preset historical time after the first preset historical time due to the external stimulus information given to the agent.

3. The method according to claim 2, characterized in that, The step of adjusting the parameters of the first initial encoder and the first initial classifier using the first training set to obtain the first model includes: The first initial encoder is used to extract features from the first sample to obtain the feature vector of the first sample. The first initial classifier is used to classify the feature vector of the first sample to obtain the predicted label of the first sample. Based on the difference between the true label and the predicted label of the first sample, the parameters of the first initial encoder and the parameters of the first initial classifier are adjusted. Once the training reaches the first preset training condition, stop adjusting the parameters of the first initial encoder and the first initial classifier to obtain the first model.

4. The method according to claim 2 or 3, characterized in that, The first initial classifier outputs an identification result of any one of N types of emotional states, wherein the first identification result is any one of the N types of emotional states, and N is greater than or equal to a first preset threshold.

5. The method according to any one of claims 1 to 3, characterized in that, Before using the second model to recognize the second text to obtain the second recognition result, the method further includes: The parameters of the second initial encoder and the second initial classifier, which are included in the second initial model, are adjusted using the second training set to obtain the second model. The second training set includes second samples and the real labels of the second samples. The second samples include information describing the emotional state of the agent at a first preset historical moment. The real labels of the second samples represent the emotional state of the agent at the first preset historical moment.

6. The method according to claim 5, characterized in that, The step of adjusting the parameters of the second initial encoder and the second initial classifier, which are included in the second initial model, using the second training set to obtain the second model includes: The second sample is used to extract features from the second sample using the second initial encoder to obtain the feature vector of the second sample; The feature vector of the second sample is classified using the second initial classifier to obtain the predicted label of the second sample; Based on the difference between the true label and the predicted label of the second sample, the parameters of the second initial encoder and the parameters of the second initial classifier are adjusted. Once the training reaches the second preset training condition, stop adjusting the parameters of the second initial encoder and the second initial classifier to obtain the second model.

7. The method according to claim 6, characterized in that, The recognition result output by the second initial classifier is any one of the M types of emotional states, wherein the second recognition result is any one of the M types of emotional states, and M is greater than or equal to a second preset threshold.

8. The method according to claim 1, characterized in that, The multiple candidate personalities include: neutral personality, extroverted personality, and introverted personality, wherein the threshold corresponding to the introverted personality is greater than the threshold corresponding to the neutral personality, and the threshold corresponding to the neutral personality is greater than the threshold corresponding to the extroverted personality.

9. The method according to any one of claims 1 to 3, characterized in that, The external stimulus information includes at least one of the following: evaluation information of the intelligent agent to be identified, or environmental information of the location of the intelligent agent to be identified.

10. An emotion recognition device, characterized in that, The device includes: The first recognition unit is used to recognize the first text to be recognized using the first model and obtain the first recognition result. The first text to be recognized includes external stimulus information given to the intelligent agent to be recognized at a first moment. The first recognition result indicates that the external stimulus information causes the intelligent agent to generate a first emotional state at a second moment after the first moment. The second recognition unit is used to recognize the second text to be recognized using the second model and obtain the second recognition result, wherein the second text to be recognized includes information describing the second emotional state of the agent to be recognized at the first moment, and the second recognition result represents the second emotional state; The determining unit is configured to determine the emotional state of the intelligent agent to be identified at the current moment based on the attribute information, the first identification result, and the second identification result, wherein the attribute information represents the personality traits of the intelligent agent to be identified, and the current moment is the moment after the second moment; Determining the emotional state of the agent to be identified at the current moment based on attribute information, the first identification result, and the second identification result includes: Based on the attribute information and the first recognition result, the target sentiment score is determined, wherein the attribute information indicates that the personality traits of the agent to be identified are the target personality. Determine the larger of the sentiment scores represented by the target sentiment score and the negative sentiment score represented by the second recognition result; If the greater sentiment score is greater than the preset confidence level, then it is determined that the emotional state of the agent to be identified at the current moment is consistent with the sentiment tendency of the greater sentiment score. Wherein, if the positive emotional level represented by the first emotional state exceeds the target threshold, then the emotional state represented by the target emotional level is consistent with the emotional tendency represented by the first emotional state; if the positive emotional level represented by the first emotional state does not exceed the target threshold, then the emotional state represented by the target emotional level is opposite to the emotional tendency represented by the first emotional state; the target threshold is the threshold corresponding to the target personality, and the target personality is one of multiple candidate personalities, with different candidate personalities corresponding to different thresholds.

11. An execution device, characterized in that, include: The memory and the processor are coupled; The memory is used to store one or more computer instructions; The processor is configured to execute one or more computer instructions to implement the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium storing one or more computer instructions thereon, characterized in that, The instruction is executed by the processor to implement the method as described in any one of claims 1 to 9.

Images