Emotion recognition method and device based on artificial intelligence, computer equipment and medium
A teacher-student network training strategy improves the efficiency and accuracy of voice emotion recognition by optimizing the student model for faster and more precise emotion recognition in financial technology applications.
Patent Information
- Application Number
- CN202411148585.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-12-03
AI Technical Summary
Existing voice emotion recognition methods based on deep learning in financial technology companies suffer from low recognition efficiency and accuracy.
An emotion recognition method utilizing a teacher-student network training strategy, involving a pre-built teacher model and student model, with specific components like speech signal feature extractors, gated recurrent units, and projection layers, to optimize the student model for improved emotion recognition.
The method enhances speech emotion recognition efficiency and accuracy by constructing a simplified target emotion recognition model that processes user input quickly and accurately.
Smart Images

Figure CN119068914A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence development technology and the field of financial technology, and in particular to artificial intelligence-based emotion recognition methods, devices, computer equipment and storage media. Background Art
[0002] Emotional intelligence is a research direction of artificial intelligence. With the development of artificial intelligence, emotional intelligence uses artificial intelligence technology to detect language and non-verbal signals, from voice changes to facial expressions, and can identify, interpret and respond to more channels for people to communicate emotions. From smart home systems to cars, from digital marketing to retail and even finance, emotional intelligence is widely used in various industries. For example, in the customer service scenario in the field of financial technology, with the assistance of instant emotion recognition technology that can instantly feedback user satisfaction, customer service can add the real-time results of emotion recognition to the consideration scope of the speech recommendation model, which is extremely necessary and practical in solving customer questions, guiding customer transactions, and providing after-sales service. It can effectively improve service efficiency and service quality in the financial field.
[0003] When it comes to customer emotion recognition, most of the methods currently used by financial technology companies are voice emotion recognition methods based on deep learning. This method first extracts the short-time Fourier transform spectrum or Mel spectrum from the original voice audio, and then uses common image processing models such as convolutional neural networks for feature extraction. However, the performance of this voice emotion recognition method is poor, resulting in low recognition efficiency and low voice emotion recognition accuracy. Summary of the invention
[0004] The purpose of the embodiments of the present application is to propose an emotion recognition method, device, computer equipment and storage medium based on artificial intelligence to solve the technical problems that the methods used by existing financial technology companies, such as the speech emotion recognition methods based on deep learning, have poor performance, resulting in low recognition efficiency, and low accuracy in speech emotion recognition.
[0005] In order to solve the above technical problems, the embodiment of the present application provides an emotion recognition method based on artificial intelligence, which adopts the following technical solutions:
[0006] Calling a pre-built teacher model and student model; wherein the teacher model includes a first speech signal feature extractor, a first gated recurrent unit, a second gated recurrent unit, a pre-trained language model, a first projection layer, and a first prediction layer; the student model includes a second speech signal feature extractor, a third gated recurrent unit, a second projection layer, and a second prediction layer;
[0007] Acquire pre-collected voice data, and pre-process the voice data to obtain voice sample data;
[0008] Training the teacher model based on the speech sample data, obtaining a joint embedding and training output in the teacher model, and generating a first loss corresponding to the teacher model based on the training output;
[0009] Training the student model based on the speech sample data to obtain a specified speech embedding in the student model;
[0010] generating a second loss corresponding to the student model based on the joint embedding and the speech embedding;
[0011] generating a comprehensive loss based on the first loss and the second loss;
[0012] Optimizing the student model based on the comprehensive loss to obtain a corresponding target emotion recognition model;
[0013] Based on the target emotion recognition model, emotion recognition processing is performed on the target speech data input by the user to generate an emotion recognition result corresponding to the user.
[0014] Furthermore, the step of training the teacher model based on the speech sample data to obtain the joint embedding and training output in the teacher model specifically includes:
[0015] Inputting the speech sample data into the teacher model;
[0016] Extracting features from the speech sample data by using the first speech signal feature extractor to obtain corresponding first speech features;
[0017] Processing the first speech feature through the first gated recurrent unit to obtain a corresponding first speech embedding;
[0018] Converting the speech sample data into text data, and encoding the text data using the pre-trained language model to obtain corresponding text features;
[0019] Processing the text feature by the second gated recurrent unit to obtain a corresponding text embedding;
[0020] Concatenating the speech embedding and the text embedding to obtain a corresponding concatenated embedding;
[0021] Projecting the concatenated embedding through the first projection layer to obtain the joint embedding;
[0022] The joint embedding is predicted by the first prediction layer to obtain the training output.
[0023] Furthermore, the step of generating a first loss corresponding to the teacher model based on the training output specifically includes:
[0024] Obtaining an emotion label corresponding to the speech sample data;
[0025] Generating a consistency correlation coefficient based on the training output and the sentiment label;
[0026] Acquire emotional dimension data corresponding to the voice sample data;
[0027] Determining a weight coefficient corresponding to the emotional dimension data;
[0028] Based on the consistency correlation coefficient and the weight coefficient, the first loss corresponding to the teacher model is generated.
[0029] Furthermore, the step of generating a consistency correlation coefficient based on the training output and the emotion label specifically includes:
[0030] Calculating the covariance between the training output and the emotion label;
[0031] Calculating a first variance of the training output, and calculating a second variance of the emotion label;
[0032] Calculating a first average value of the training outputs, and calculating a second average value of the emotion labels;
[0033] Calling a preset calculation formula to calculate the covariance, the first variance, the second variance, the first average value, and the second average value to obtain a corresponding calculation result;
[0034] The calculation result is used as the consistency correlation coefficient.
[0035] Furthermore, the step of training the student model based on the speech sample data to obtain the specified speech embedding in the student model specifically includes:
[0036] Inputting the speech sample data into the student model;
[0037] Extracting features from the speech sample data by using the second speech signal feature extractor to obtain corresponding second speech features;
[0038] Processing the second speech feature through the third gated recurrent unit to obtain a corresponding second speech embedding;
[0039] The second speech embedding is projected through the second projection layer to obtain the specified speech embedding.
[0040] Furthermore, the step of generating a comprehensive loss based on the first loss and the second loss specifically includes:
[0041] Obtaining a specified weight corresponding to the second loss;
[0042] Calculating a product between the specified weight and the second loss;
[0043] calculating a sum of the product and the first loss;
[0044] The sum is taken as the comprehensive loss.
[0045] Furthermore, after the step of optimizing the student model based on the comprehensive loss to obtain the corresponding target emotion recognition model, the method further includes:
[0046] Constructing verification data based on the voice sample data;
[0047] Performing performance verification processing on the target emotion recognition model based on the verification data to obtain performance indicator data of the target emotion recognition model;
[0048] Performing data analysis on the performance indicator data to generate a performance analysis result corresponding to the target emotion recognition model;
[0049] Based on the performance analysis results, corresponding adjustments are made to the target emotion recognition model.
[0050] In order to solve the above technical problems, the embodiment of the present application also provides an emotion recognition device based on artificial intelligence, which adopts the following technical solution:
[0051] A calling module, used to call a pre-built teacher model and student model; wherein the teacher model includes a first speech signal feature extractor, a first gated recurrent unit, a second gated recurrent unit, a pre-trained language model, a first projection layer and a first prediction layer; the student model includes a second speech signal feature extractor, a third gated recurrent unit, a second projection layer and a second prediction layer;
[0052] A preprocessing module, used to obtain pre-collected voice data and preprocess the voice data to obtain voice sample data;
[0053] A first processing module, configured to train the teacher model based on the speech sample data, obtain a joint embedding and training output in the teacher model, and generate a first loss corresponding to the teacher model based on the training output;
[0054] A second processing module, configured to train the student model based on the speech sample data to obtain a specified speech embedding in the student model;
[0055] A first generating module, configured to generate a second loss corresponding to the student model based on the joint embedding and the speech embedding;
[0056] A second generating module, configured to generate a comprehensive loss based on the first loss and the second loss;
[0057] An optimization module, used for optimizing the student model based on the comprehensive loss to obtain a corresponding target emotion recognition model;
[0058] The recognition module is used to perform emotion recognition processing on the target speech data input by the user based on the target emotion recognition model, and generate an emotion recognition result corresponding to the user.
[0059] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:
[0060] The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the artificial intelligence-based emotion recognition methods proposed in the embodiments of the present application are implemented.
[0061] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:
[0062] The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of any one of the artificial intelligence-based emotion recognition methods proposed in the embodiments of the present application are implemented.
[0063] Compared with the prior art, the embodiments of the present application have the following beneficial effects:
[0064] The present application first calls a pre-built teacher model and a student model; then obtains pre-collected speech data, and pre-processes the speech data to obtain speech sample data; then trains the teacher model based on the speech sample data, obtains the joint embedding and training output in the teacher model, and generates a first loss corresponding to the teacher model based on the training output; and trains the student model based on the speech sample data to obtain a specified speech embedding in the student model; subsequently generates a second loss corresponding to the student model based on the joint embedding and the speech embedding; and generates a comprehensive loss based on the first loss and the second loss; further optimizes the student model based on the comprehensive loss to obtain a corresponding target emotion recognition model; finally, performs emotion recognition processing on the target speech data input by the user based on the target emotion recognition model to generate an emotion recognition result corresponding to the user. This application constructs a corresponding target emotion recognition model by introducing a teacher-student network training strategy corresponding to the teacher model and the student model. Compared with the teacher model, the structure of the target emotion recognition model is relatively simple, so that the target emotion recognition model can be used to perform emotion recognition processing on the target speech data input by the user. The model processing speed is fast, which can effectively improve the efficiency of speech emotion recognition processing and improve the recognition accuracy of speech emotion. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0066] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;
[0067] Figure 2 A flowchart of an embodiment of an emotion recognition method based on artificial intelligence according to the present application;
[0068] Figure 3 is a structural schematic diagram of an embodiment of an emotion recognition device based on artificial intelligence according to the present application;
[0069] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION
[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.
[0071] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0072] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.
[0073] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0074] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.
[0075] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, etc.
[0076] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .
[0077] It should be noted that the artificial intelligence-based emotion recognition method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the artificial intelligence-based emotion recognition device is generally set in the server / terminal device.
[0078] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0079] Continue to refer Figure 2 , showing a flow chart of an embodiment of an artificial intelligence-based emotion recognition method according to the present application. According to different needs, the order of the steps in the flow chart can be changed, and some steps can be omitted. The artificial intelligence-based emotion recognition method provided in the embodiment of the present application can be applied to any scenario that requires emotion recognition about voice, and the artificial intelligence-based emotion recognition method can be applied to products in these scenarios, for example, emotion recognition of user input voice in business scenarios where intelligent customer service in the financial insurance field provides services. The services provided by intelligent customer service may include financial product recommendation services, such as auto insurance product recommendation services, life insurance product recommendation services, accident insurance product recommendation services, and so on. The artificial intelligence-based emotion recognition method comprises the following steps:
[0080] Step S201, calling the pre-built teacher model and student model.
[0081] In this embodiment, the electronic device (e.g. Figure 1 The server / terminal device shown in the figure) can obtain the pre-built teacher model and student model through a wired connection or a wireless connection. It should be pointed out that the above-mentioned wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future. Among them, the teacher model includes a first speech signal feature extractor, a first gated recurrent unit, a second gated recurrent unit, a pre-trained language model, a first projection layer and a first prediction layer; the student model includes a second speech signal feature extractor, a third gated recurrent unit, a second projection layer and a second prediction layer.
[0082] Step S202: Acquire pre-collected voice data, and pre-process the voice data to obtain voice sample data.
[0083] In this embodiment, the above-mentioned voice data may be voice data input by users when interacting with intelligent customers in business scenarios provided by intelligent customer service in the financial insurance field. Among them, the above-mentioned preprocessing may include data cleaning of the voice data, such as filtering erroneous data, deduplication and other processing. In addition, when the voice data is preprocessed to obtain voice sample data, the voice sample data may be emotionally annotated by manual annotation or automatic machine annotation, that is, matching emotional tags are added to the voice sample data to divide the voice sample data into three emotional categories, which may specifically include positive tags, neutral tags and negative tags. Among them, positive tags can be represented by the number 1, neutral tags can be represented by the number 0, and negative tags can be represented by the number -1.
[0084] Step S203: train the teacher model based on the speech sample data, obtain the joint embedding and training output in the teacher model, and generate a first loss corresponding to the teacher model based on the training output.
[0085] In this embodiment, loss is an indicator used in machine learning to measure the difference between the model prediction result and the true label. During the training process, the model minimizes the training loss by continuously adjusting parameters to improve its performance. Among them, the specific implementation process of training the teacher model based on the speech sample data, obtaining the joint embedding and training output in the teacher model, and generating the first loss corresponding to the teacher model based on the training output will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated on here. In addition, after obtaining the trained teacher model, the teacher model can be used to transfer knowledge to the student model according to the knowledge distillation mechanism to transfer the knowledge learned by the teacher model to the student model. Knowledge distillation is a process of transferring the knowledge of a complex model (usually referred to as a "teacher" model) to a simple model (usually referred to as a "student" model). Its main purpose is to simulate the behavior of a complex model by training a simple and efficient model, so as to achieve high-performance prediction on new tasks or new data sets. Knowledge distillation can help the student model quickly grasp the key points of knowledge, establish a knowledge framework, and improve learning efficiency. It transforms a large amount of knowledge into a concise form, removes redundant information and tedious details, and extracts core concepts and basic principles, so that the student model can better understand and master knowledge. Specifically, a teacher model can be pre-trained, and then the predicted output of the teacher model and the real data label can be combined to supervise the training of the student model.
[0086] Step S204: training the student model based on the speech sample data to obtain a specified speech embedding in the student model.
[0087] In this embodiment, the specific implementation process of training the student model based on the speech sample data to obtain the specified speech embedding in the student model will be described in further detail in subsequent specific embodiments of the present application and will not be elaborated on here.
[0088] Step S205, generating a second loss corresponding to the student model based on the joint embedding and the speech embedding.
[0089] In this embodiment, the formula corresponding to the second loss specifically includes: in, is the second loss, e AT For the joint embedding, e A The speech embedding is constrained by using the joint embedding in the teacher model. By minimizing the L2 loss between the joint embedding and the speech embedding, the student model is able to output a speech embedding that is as close as possible to the joint embedding of the teacher model. L2 loss, also known as mean squared error (MSE) loss, is used to measure the difference between the model prediction value and the true value, and the performance of the model is evaluated by calculating the average of the squares of the difference between the predicted value and the true value.
[0090] Step S206: generating a comprehensive loss based on the first loss and the second loss.
[0091] In this embodiment, the specific implementation process of generating the comprehensive loss based on the first loss and the second loss will be further described in detail in subsequent specific embodiments of the present application and will not be elaborated on here.
[0092] Step S207, optimizing the student model based on the comprehensive loss to obtain a corresponding target emotion recognition model.
[0093] In this embodiment, after obtaining the comprehensive loss, the model parameters of the student model can be reversely optimized according to the comprehensive loss. Specifically, multiple rounds of optimization can be performed, and the model training is stopped when the optimization cutoff condition is reached, thereby obtaining the target emotion recognition model. Among them, the setting of the optimization cutoff condition is not specifically limited. For example, it can be a training cutoff condition, it can be that the number of training iterations reaches a preset number, or it can be that the comprehensive loss is lower than a preset loss threshold. The above-mentioned target emotion recognition model is the student model that has completed the optimization processing after the final training. In addition, the target emotion recognition model obtained removes the text transcription technology in the teacher model, but the emotion recognition effects of the two are equivalent, so that the recognition processing speed can be effectively improved when the target emotion recognition model is used for emotion recognition processing in the subsequent process.
[0094] Step S208: performing emotion recognition processing on the target speech data input by the user based on the target emotion recognition model to generate an emotion recognition result corresponding to the user.
[0095] In this embodiment, by inputting the target speech data input by the user into the target emotion recognition model, the target emotion recognition model performs emotion recognition processing on the target speech data, and the target emotion recognition model can identify the probability that the speech data input by the user is each type of emotion, and output the emotion category with the highest probability. For example, the output emotion category can be a positive, neutral label, or a negative label.
[0096] This application constructs a corresponding target emotion recognition model by introducing a teacher-student network training strategy corresponding to the teacher model and the student model. Compared with the teacher model, the structure of the target emotion recognition model is relatively simple, so that the target emotion recognition model can be used to perform emotion recognition processing on the target speech data input by the user. The model processing speed is fast, which can effectively improve the efficiency of speech emotion recognition processing and improve the recognition accuracy of speech emotion.
[0097] In some optional implementations, the step of training the teacher model based on the speech sample data to obtain the joint embedding and training output in the teacher model in step S203 includes the following steps:
[0098] The speech sample data is input into the teacher model.
[0099] In this embodiment, when the teacher model is trained using the speech sample data, the speech sample data is first used as a model input and input into the teacher model.
[0100] The first speech signal feature extractor performs feature extraction on the speech sample data to obtain a corresponding first speech feature.
[0101] In this embodiment, the first speech signal feature extractor may specifically use Wav2vec 2.0. By using Wav2vec 2.0 to perform feature extraction on the speech sample data, the first sound feature in the speech sample data may be extracted, thereby obtaining the first speech feature.
[0102] The first speech feature is processed by the first gated recurrent unit to obtain a corresponding first speech embedding.
[0103] In this embodiment, by using a gated recurrent unit (GRU), context information corresponding to the first speech feature can be learned and a corresponding first speech embedding can be output. The first gated recurrent unit includes 2 layers and 128 hidden units. The 128-dimensional vector outputted by the last unit in the first gated recurrent unit is used as the first speech embedding.
[0104] The speech sample data is converted into text data, and the text data is encoded using the pre-trained language model to obtain corresponding text features.
[0105] In this embodiment, the speech sample data can be transcribed into text using ASR technology or the above-mentioned Wav2vec 2.0 to obtain the above-mentioned text data. The above-mentioned pre-trained language model can specifically adopt the BERT (Bidirectional Encoder Representation from Transformers) model. The BERT model is a pre-trained model for language representation. It adopts the bidirectional encoder representation of Transformer, emphasizing the generation of a deep bidirectional language representation model, instead of using the traditional unidirectional language model or the shallow splicing method of two unidirectional language models for pre-training as in the past. By encoding the text data using the BERT model, corresponding text features can be generated.
[0106] The text features are processed by the second gated recurrent unit to obtain corresponding text embedding.
[0107] In this embodiment, the context information corresponding to the text feature is learned by using the second gated recurrent unit, and the corresponding text embedding is output. The second gated recurrent unit includes 2 layers and 128 hidden units. The 128-dimensional vector outputted by the last unit in the second gated recurrent unit is used as the above text embedding.
[0108] The speech embedding and the text embedding are concatenated to obtain a corresponding concatenated embedding.
[0109] The concatenated embedding is projected through the first projection layer to obtain the joint embedding.
[0110] In this embodiment, the first projection layer may specifically adopt a fully connected layer, and the concatenated embedding is processed by the fully connected layer to output an embedding vector of the joint representation corresponding to the first speech embedding and the text embedding, that is, the above-mentioned joint embedding.
[0111] The joint embedding is predicted by the first prediction layer to obtain the training output.
[0112] In this embodiment, the first prediction layer may specifically adopt a Softmax layer, which is used in combination with a fully connected layer to convert the output of the fully connected layer into a probability distribution so that the sum of the probabilities of all categories is 1. The joint embedding is processed for emotion recognition by using the first prediction layer, and an emotion prediction result corresponding to the joint embedding is output, that is, the training output.
[0113] After the speech sample data is input into the teacher model, the present application processes the speech sample data by using the first speech signal feature extractor, the first gated recurrent unit, the second gated recurrent unit, the pre-trained language model, the first projection layer and the first prediction layer included in the teacher model, so as to quickly and accurately output the joint embedding corresponding to the speech sample data and the training output of the teacher model, thereby ensuring the data accuracy of the obtained joint embedding and training output.
[0114] In some optional implementations of this embodiment, the step of generating a first loss corresponding to the teacher model based on the training output in step S203 includes the following steps:
[0115] Obtain an emotion label corresponding to the speech sample data.
[0116] In this embodiment, in the process of preprocessing the speech data to obtain the speech sample data, the speech sample data can be emotionally labeled by manual labeling or automatic machine labeling, that is, matching emotional labels are added to the speech sample data to classify the speech sample data into three emotional categories, which may specifically include positive labels, neutral labels, and negative labels. Among them, the positive label can be represented by the number 1, the neutral label can be represented by the number 0, and the negative label can be represented by the number -1.
[0117] A consistency correlation coefficient is generated based on the training output and the sentiment label.
[0118] In this embodiment, the specific implementation process of generating the consistency correlation coefficient based on the training output and the emotion label will be further described in detail in subsequent specific embodiments of the present application, and will not be elaborated on here.
[0119] Acquire emotional dimension data corresponding to the voice sample data.
[0120] In this embodiment, when the voice sample data is obtained, emotional dimension data matching the voice sample data will also be generated based on manual processing. The emotional dimension data includes three emotional dimensions corresponding to Arousal, Valence, and Dominance. Arousal represents the activation level or intensity of the emotional experience. Valence represents the positive or negative direction of the emotion, that is, the positivity or negativity of the emotion. Dominance represents the degree of dominance or control of the individual in the emotional experience.
[0121] Determine a weight coefficient corresponding to the sentiment dimension data.
[0122] In this embodiment, there is no specific limitation on the selection of the weight coefficient of the emotional dimension data, and it can be set according to actual business usage requirements.
[0123] Based on the consistency correlation coefficient and the weight coefficient, the first loss corresponding to the teacher model is generated.
[0124] In this embodiment, the formula corresponding to the first loss specifically includes: in, represents the first loss, j represents the speech sample data j annotated with emotional dimension data, CCC j represents the consistency correlation coefficient corresponding to the speech sample data j, α j Represents the weight coefficient of the emotional dimension data corresponding to the speech sample data j.
[0125] The present application obtains the emotion label corresponding to the speech sample data, and generates a consistency correlation coefficient based on the training output and the emotion label; then obtains the emotion dimension data corresponding to the speech sample data, and determines the weight coefficient corresponding to the emotion dimension data; subsequently, based on the consistency correlation coefficient and the weight coefficient, generates the first loss corresponding to the teacher model, so as to realize the use of the generated consistency correlation coefficient and the obtained weight coefficient corresponding to the emotion dimension data, so as to realize the quick and convenient generation of the first loss corresponding to the teacher model, thereby improving the generation efficiency of the obtained first loss.
[0126] In some optional implementations, the construction method of the first loss corresponding to the teacher model may also include: calculating the first loss corresponding to the teacher model through a training loss function, that is, the first loss is a training loss value calculated by the loss function. The training loss function may, for example, be a cross entropy loss, a logarithmic loss, or the like.
[0127] In some optional implementations, generating a consistency correlation coefficient based on the training output and the emotion tag comprises the following steps:
[0128] The covariance between the training output and the sentiment label is calculated.
[0129] In this embodiment, the covariance between the training output and the emotion label may be calculated by using a general covariance calculation formula.
[0130] A first variance of the training output is calculated, and a second variance of the sentiment label is calculated.
[0131] In this embodiment, the first variance of the emotion tag and the second variance of the training output may be calculated by using a general variance calculation formula.
[0132] A first average of the training outputs is calculated, and a second average of the sentiment labels is calculated.
[0133] In this embodiment, a general average value calculation formula may be used to calculate the first average value of the emotion label, and the second average value of the training output may be calculated.
[0134] A preset calculation formula is called to calculate the covariance, the first variance, the second variance, the first average value and the second average value to obtain a corresponding calculation result.
[0135] In this embodiment, the above calculation formula specifically includes: Among them, CCC is the consistency correlation coefficient, is the training output, y is the emotion label of the speech sample data, is the covariance, is the first variance of the training output, is the second variance of the sentiment label, is the first average value of the training output, μ y is the second average value of the emotion tag. The consistency correlation coefficient is obtained by inputting the covariance, the first variance, the second variance, the first average value and the second average value into the above calculation formula for calculation.
[0136] The calculation result is used as the consistency correlation coefficient.
[0137] The present application calculates and processes the covariance between the training output and the emotion label, the first variance of the training output, the second variance of the emotion label, the first average value of the training output and the second average value of the emotion label based on a calculation formula, so as to quickly and accurately generate the required consistency correlation coefficient and ensure the accuracy of the obtained consistency correlation coefficient.
[0138] In some optional implementations, step S204 includes the following steps:
[0139] The speech sample data is input into the student model.
[0140] In this embodiment, when the student model is trained using the speech sample data, the speech sample data is used as a model input and input into the student model.
[0141] The second speech signal feature extractor performs feature extraction on the speech sample data to obtain corresponding second speech features.
[0142] In this embodiment, the second speech signal feature extractor may specifically use Wav2vec 2.0. By using Wav2vec 2.0 to perform feature extraction on the speech sample data, the second sound feature in the speech sample data may be extracted, thereby obtaining the second speech feature.
[0143] The second speech feature is processed by the third gated recurrent unit to obtain a corresponding second speech embedding.
[0144] In this embodiment, the context information corresponding to the second speech feature is learned by using a third gated recurrent unit, and the corresponding second speech embedding is output. The second gated recurrent unit includes 2 layers and 128 hidden units. The 128-dimensional vector outputted by the last unit in the third gated recurrent unit is used as the second speech embedding.
[0145] The second speech embedding is projected through the second projection layer to obtain the specified speech embedding.
[0146] In this embodiment, the second projection layer may specifically adopt a fully connected layer, and the second speech embedding is processed by the fully connected layer to output the corresponding specified speech embedding. In addition, the student model also includes a second prediction layer, and the second prediction layer may specifically adopt a Softmax layer, which is used in combination with the fully connected layer to convert the output of the fully connected layer into a probability distribution so that the sum of the probabilities of all categories is 1. The second speech embedding is processed for emotion recognition by using the second prediction layer, and the emotion prediction result corresponding to the second speech embedding is output.
[0147] After the speech sample data is input into the student model, the present application processes the speech sample data by using the second speech signal feature extractor, the third gated recurrent unit and the second projection layer included in the student model, so as to quickly and accurately output the specified speech embedding corresponding to the speech sample data, thereby ensuring the data accuracy of the obtained specified speech embedding.
[0148] In some optional implementations of this embodiment, step S206 includes the following steps:
[0149] Get the specified weight corresponding to the second loss.
[0150] In this embodiment, there is no specific limitation on the selection of the above-mentioned designated weights, which can be set according to actual business usage requirements.
[0151] A product between the specified weight and the second loss is calculated.
[0152] In this embodiment, the designated weight and the second loss may be multiplied to obtain the product of the designated weight and the second loss.
[0153] A sum between the product and the first loss is calculated.
[0154] In this embodiment, the product and the first loss may be summed to obtain a sum value of the product and the first loss.
[0155] The sum is taken as the comprehensive loss.
[0156] The present application obtains the designated weight corresponding to the second loss; then calculates the product between the designated weight and the second loss; subsequently calculates the sum of the product and the first loss, and uses the sum as the comprehensive loss, so as to quickly and accurately generate the corresponding comprehensive loss according to the first loss and the second loss, thereby improving the efficiency of generating the comprehensive loss and ensuring the accuracy of the obtained comprehensive loss.
[0157] In some optional implementations of this embodiment, after step S207, the electronic device may further perform the following steps:
[0158] Verification data is constructed based on the voice sample data.
[0159] In this embodiment, a specified proportion of data can be randomly selected from the voice sample data and used as the verification data. The value of the specified proportion is not specifically limited and can be determined according to actual usage requirements, for example, it can be set to 0.3.
[0160] Based on the verification data, performance verification processing is performed on the target emotion recognition model to obtain performance indicator data of the target emotion recognition model.
[0161] In this embodiment, the verification data is respectively input into the target emotion recognition model, so that the verification data is processed by the target emotion recognition model for emotion recognition, and then the target emotion recognition model is subjected to performance verification processing according to the emotion recognition result output by the target emotion recognition model, that is, the target emotion recognition model is subjected to performance evaluation to obtain performance index data of the target emotion recognition model. Among them, the above-mentioned performance index data may include specific data values corresponding to the calculated accuracy, F1 value, recall rate and other indicators of the target emotion recognition model used to evaluate the model performance.
[0162] Perform data analysis on the performance indicator data to generate a performance analysis result corresponding to the target emotion recognition model.
[0163] In this embodiment, the above-mentioned performance indicator data specifically include accuracy, F1 value, and recall rate. The calculated accuracy can be compared with a preset accuracy threshold, the F1 value can be compared with a preset specified threshold, and the recall rate can be compared with a preset recall rate threshold. If the accuracy is greater than the corresponding accuracy threshold, the F1 value is greater than the corresponding specified threshold, and the recall rate is greater than the corresponding recall rate threshold, then it is determined that the target emotion recognition model has passed the performance verification, and a first performance analysis result indicating that the target emotion recognition model has passed the performance verification is generated. If at least one indicator data among all the performance indicator data is less than the corresponding indicator threshold, then it is determined that the target emotion recognition model has not passed the performance verification, and a second performance analysis result indicating that the target emotion recognition model has not passed the performance verification is generated.
[0164] Based on the performance analysis results, corresponding adjustments are made to the target emotion recognition model.
[0165] In this embodiment, if the content of the performance analysis result is that the target emotion recognition model has not passed the performance verification, a preset model improvement strategy can be used to improve the target emotion recognition model. Among them, the above-mentioned model improvement strategy may include strategies such as adjusting model parameters / training parameters, changing layer structure, modifying model architecture or data enhancement strategy. In addition, if the content of the performance analysis result is that the target emotion recognition model has passed the performance verification, the target emotion recognition model is determined to be a model that meets the required performance requirements, and there is no need to adjust the target emotion recognition model subsequently.
[0166] This application constructs verification data based on the speech sample data; then performs performance verification processing on the target emotion recognition model based on the verification data to obtain performance indicator data of the target emotion recognition model; then performs data analysis on the performance indicator data to generate a performance analysis result corresponding to the target emotion recognition model; and then performs corresponding adjustment processing on the target emotion recognition model based on the performance analysis result. After completing the construction of the target emotion recognition model, this application will also intelligently use the verification data constructed based on the speech sample data to perform performance verification processing on the target emotion recognition model, and will perform corresponding adjustment processing on the target emotion recognition model according to the performance analysis result of analyzing the performance indicator data of the obtained target emotion recognition model, so as to ensure that the target emotion recognition model finally obtained can meet the corresponding performance test expectations, which is conducive to ensuring the accuracy of the subsequent use of the target emotion recognition model to perform emotion recognition processing on the target speech data input by the user, thereby improving the accuracy of the generated emotion recognition result corresponding to the user.
[0167] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
[0168] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned emotion recognition results, the above-mentioned emotion recognition results can also be stored in a node of a blockchain.
[0169] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.
[0170] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0171] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0172] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).
[0173] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.
[0174] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of an emotion recognition device based on artificial intelligence, and the embodiment of the device is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0175] like Figure 3As shown, the emotion recognition device 300 based on artificial intelligence described in this embodiment includes: a calling module 301, a preprocessing module 302, a first processing module 303, a second processing module 304, a first generating module 305, a second generating module 306, an optimization module 307 and a recognition module 308. Among them:
[0176] The calling module 301 is used to call the pre-built teacher model and student model; wherein the teacher model includes a first speech signal feature extractor, a first gated recurrent unit, a second gated recurrent unit, a pre-trained language model, a first projection layer and a first prediction layer; the student model includes a second speech signal feature extractor, a third gated recurrent unit, a second projection layer and a second prediction layer;
[0177] The preprocessing module 302 is used to obtain pre-collected voice data and pre-process the voice data to obtain voice sample data;
[0178] A first processing module 303 is used to train the teacher model based on the speech sample data, obtain the joint embedding and training output in the teacher model, and generate a first loss corresponding to the teacher model based on the training output;
[0179] A second processing module 304 is used to train the student model based on the speech sample data to obtain a specified speech embedding in the student model;
[0180] A first generating module 305, configured to generate a second loss corresponding to the student model based on the joint embedding and the speech embedding;
[0181] A second generating module 306, configured to generate a comprehensive loss based on the first loss and the second loss;
[0182] An optimization module 307, configured to optimize the student model based on the comprehensive loss to obtain a corresponding target emotion recognition model;
[0183] The recognition module 308 is used to perform emotion recognition processing on the target speech data input by the user based on the target emotion recognition model to generate an emotion recognition result corresponding to the user.
[0184] In this embodiment, the operations performed by the above modules or units respectively correspond one-to-one to the steps of the artificial intelligence-based emotion recognition method in the aforementioned implementation, and will not be repeated here.
[0185] In some optional implementations of this embodiment, the first processing module 303 includes:
[0186] A first input submodule, used for inputting the speech sample data into the teacher model;
[0187] A first extraction submodule, configured to perform feature extraction on the speech sample data through the first speech signal feature extractor to obtain a corresponding first speech feature;
[0188] A first processing submodule, configured to process the first speech feature through the first gated recurrent unit to obtain a corresponding first speech embedding;
[0189] An encoding submodule, used to convert the speech sample data into text data, and use the pre-trained language model to encode the text data to obtain corresponding text features;
[0190] A second processing submodule, configured to process the text feature through the second gated recurrent unit to obtain a corresponding text embedding;
[0191] A concatenation submodule, configured to concatenate the speech embedding with the text embedding to obtain a corresponding concatenated embedding;
[0192] A third processing submodule, configured to project the concatenated embedding through the first projection layer to obtain the joint embedding;
[0193] A prediction submodule is used to perform prediction processing on the joint embedding through the first prediction layer to obtain the training output.
[0194] In some optional implementations of this embodiment, the first processing module 303 further includes:
[0195] A first acquisition submodule is used to acquire an emotion label corresponding to the speech sample data;
[0196] A first generating submodule, used for generating a consistency correlation coefficient based on the training output and the emotion label;
[0197] A second acquisition submodule is used to acquire the emotion dimension data corresponding to the voice sample data;
[0198] A first determination submodule, used to determine a weight coefficient corresponding to the emotion dimension data;
[0199] The second generating submodule is used to generate the first loss corresponding to the teacher model based on the consistency correlation coefficient and the weight coefficient.
[0200] In some optional implementations of this embodiment, the first generating submodule includes:
[0201] A first calculation unit, used for calculating the covariance between the training output and the emotion label;
[0202] A second calculation unit, used for calculating a first variance of the training output and a second variance of the emotion label;
[0203] A third calculation unit, used for calculating a first average value of the training output and a second average value of the emotion label;
[0204] a fourth calculation unit, configured to call a preset calculation formula to calculate the covariance, the first variance, the second variance, the first average value, and the second average value to obtain a corresponding calculation result;
[0205] A determination unit is used to use the calculation result as the consistency correlation coefficient.
[0206] In some optional implementations of this embodiment, the second processing module 304 includes:
[0207] A second input submodule, used for inputting the speech sample data into the student model;
[0208] A second extraction submodule, configured to perform feature extraction on the speech sample data through the second speech signal feature extractor to obtain corresponding second speech features;
[0209] a fourth processing submodule, configured to process the second speech feature through the third gated recurrent unit to obtain a corresponding second speech embedding;
[0210] The fifth processing submodule is used to perform projection processing on the second speech embedding through the second projection layer to obtain the specified speech embedding.
[0211] In some optional implementations of this embodiment, the second generating module 306 includes:
[0212] A third acquisition submodule, used to obtain a specified weight corresponding to the second loss;
[0213] A first calculation submodule, used for calculating the product between the specified weight and the second loss;
[0214] A second calculation submodule, used for calculating the sum of the product and the first loss;
[0215] The second determination submodule is used to use the sum value as the comprehensive loss.
[0216] In some optional implementations of this embodiment, the emotion recognition device based on artificial intelligence further includes:
[0217] A construction module, used to construct verification data based on the voice sample data;
[0218] A verification module, used to perform performance verification processing on the target emotion recognition model based on the verification data to obtain performance indicator data of the target emotion recognition model;
[0219] An analysis module, used to perform data analysis on the performance indicator data and generate a performance analysis result corresponding to the target emotion recognition model;
[0220] An adjustment module is used to perform corresponding adjustment processing on the target emotion recognition model based on the performance analysis result.
[0221] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.
[0222] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.
[0223] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.
[0224] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the emotion recognition method based on artificial intelligence, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.
[0225] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the artificial intelligence-based emotion recognition method.
[0226] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.
[0227] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned artificial intelligence-based emotion recognition method.
[0228] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0229] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.
Claims
1. An emotion recognition method based on artificial intelligence, characterized in that: The steps include: Calling a pre-built teacher model and student model; wherein the teacher model includes a first speech signal feature extractor, a first gated recurrent unit, a second gated recurrent unit, a pre-trained language model, a first projection layer, and a first prediction layer; the student model includes a second speech signal feature extractor, a third gated recurrent unit, a second projection layer, and a second prediction layer; Acquire pre-collected voice data, and pre-process the voice data to obtain voice sample data; Training the teacher model based on the speech sample data, obtaining a joint embedding and training output in the teacher model, and generating a first loss corresponding to the teacher model based on the training output; Training the student model based on the speech sample data to obtain a specified speech embedding in the student model; generating a second loss corresponding to the student model based on the joint embedding and the speech embedding; generating a comprehensive loss based on the first loss and the second loss; Optimizing the student model based on the comprehensive loss to obtain a corresponding target emotion recognition model; Based on the target emotion recognition model, emotion recognition processing is performed on the target speech data input by the user to generate an emotion recognition result corresponding to the user.
2. The emotion recognition method based on artificial intelligence according to claim 1, characterized in that: The step of training the teacher model based on the speech sample data to obtain the joint embedding and training output in the teacher model specifically includes: Inputting the speech sample data into the teacher model; Extracting features from the speech sample data by using the first speech signal feature extractor to obtain corresponding first speech features; Processing the first speech feature through the first gated recurrent unit to obtain a corresponding first speech embedding; Converting the speech sample data into text data, and encoding the text data using the pre-trained language model to obtain corresponding text features; Processing the text feature by the second gated recurrent unit to obtain a corresponding text embedding; Concatenating the speech embedding and the text embedding to obtain a corresponding concatenated embedding; Projecting the concatenated embedding through the first projection layer to obtain the joint embedding; The joint embedding is predicted by the first prediction layer to obtain the training output.
3. The emotion recognition method based on artificial intelligence according to claim 1, characterized in that: The step of generating a first loss corresponding to the teacher model based on the training output specifically includes: Obtaining an emotion label corresponding to the speech sample data; Generating a consistency correlation coefficient based on the training output and the sentiment label; Acquire emotional dimension data corresponding to the voice sample data; Determining a weight coefficient corresponding to the emotional dimension data; Based on the consistency correlation coefficient and the weight coefficient, the first loss corresponding to the teacher model is generated.
4. The emotion recognition method based on artificial intelligence according to claim 3 is characterized in that: The step of generating a consistency correlation coefficient based on the training output and the emotion label specifically includes: Calculating the covariance between the training output and the emotion label; Calculating a first variance of the training output, and calculating a second variance of the emotion label; Calculating a first average value of the training outputs, and calculating a second average value of the emotion labels; Calling a preset calculation formula to calculate the covariance, the first variance, the second variance, the first average value, and the second average value to obtain a corresponding calculation result; The calculation result is used as the consistency correlation coefficient.
5. The emotion recognition method based on artificial intelligence according to claim 1, characterized in that: The step of training the student model based on the speech sample data to obtain the specified speech embedding in the student model specifically includes: Inputting the speech sample data into the student model; Extracting features from the speech sample data by using the second speech signal feature extractor to obtain corresponding second speech features; Processing the second speech feature through the third gated recurrent unit to obtain a corresponding second speech embedding; The second speech embedding is projected through the second projection layer to obtain the specified speech embedding.
6. The emotion recognition method based on artificial intelligence according to claim 1, characterized in that: The step of generating a comprehensive loss based on the first loss and the second loss specifically includes: Obtaining a specified weight corresponding to the second loss; Calculating a product between the specified weight and the second loss; calculating a sum of the product and the first loss; The sum is taken as the comprehensive loss.
7. The emotion recognition method based on artificial intelligence according to claim 1, characterized in that: After the step of optimizing the student model based on the comprehensive loss to obtain the corresponding target emotion recognition model, the method further includes: Constructing verification data based on the voice sample data; Performing performance verification processing on the target emotion recognition model based on the verification data to obtain performance indicator data of the target emotion recognition model; Performing data analysis on the performance indicator data to generate a performance analysis result corresponding to the target emotion recognition model; Based on the performance analysis results, corresponding adjustments are made to the target emotion recognition model.
8. An emotion recognition device based on artificial intelligence, characterized in that: include: A calling module, used to call a pre-built teacher model and student model; wherein the teacher model includes a first speech signal feature extractor, a first gated recurrent unit, a second gated recurrent unit, a pre-trained language model, a first projection layer and a first prediction layer; the student model includes a second speech signal feature extractor, a third gated recurrent unit, a second projection layer and a second prediction layer; A preprocessing module, used to obtain pre-collected voice data and preprocess the voice data to obtain voice sample data; A first processing module, configured to train the teacher model based on the speech sample data, obtain a joint embedding and training output in the teacher model, and generate a first loss corresponding to the teacher model based on the training output; A second processing module, configured to train the student model based on the speech sample data to obtain a specified speech embedding in the student model; A first generating module, configured to generate a second loss corresponding to the student model based on the joint embedding and the speech embedding; A second generating module, configured to generate a comprehensive loss based on the first loss and the second loss; An optimization module, used for optimizing the student model based on the comprehensive loss to obtain a corresponding target emotion recognition model; The recognition module is used to perform emotion recognition processing on the target speech data input by the user based on the target emotion recognition model, and generate an emotion recognition result corresponding to the user.
9. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the artificial intelligence-based emotion recognition method as described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the artificial intelligence-based emotion recognition method as described in any one of claims 1 to 7.
