A method, device and computing device for recognizing speech emotions in dialogue scenes

Abstract

The present application discloses a method, device and computing device for speech emotion recognition in a conversational scenario. The method comprises: extracting MFCC features, first-order difference features and second-order difference features of training samples; reading feature files to be trained to form feature data combinations of data-labels; feeding the feature data combinations into an improved AlexNet network and a bidirectional BiGRU network, and aligning feature sequences through CTC, feeding the output of CTC into a Softmax classifier, and then performing back propagation training based on cross-entropy loss to obtain a recognition model; using the model to perform emotion classification on customer service and customer channel recordings. The device comprises a training feature extraction module, a data labeling module, a training module and a recognition module. The computing device comprises a memory, a processor and a computer program stored in the memory and executable by the processor, and the processor implements the method described in the present application when executing the computer program.

Description

Technical Field

[0001] The present application relates to the field of speech emotion recognition, and in particular to speech emotion recognition in consumer finance scenarios. Background Art

[0002] In the consumer finance scenario, a lot of hotlines, return visits, collection and other businesses are handled in the customer service call center every day. Customer service represents the image of the company. It is very important to improve the quality of customer service and effectively supervise the customer service attitude. In addition, real-time feedback on the emotional state of customers during the conversation is also the key to improving service quality. Traditionally, the method of providing feedback on customer service and customer conversation emotions is usually manual sampling, which is time-consuming, labor-intensive and costly. Therefore, the current consumer finance field urgently needs a system that can obtain the emotional state of customer service and customers in real time and relatively accurately during voice conversations.

[0003] In the current customer service dialogue scenarios in the financial field, the accuracy of speech emotion recognition is low due to the influence of factors such as telephone channel noise and dialects, and the recognition speed is also difficult to meet the real-time requirements, which greatly increases the difficulty of emotion recognition in financial customer service scenarios. The speech emotion recognition methods for financial customer service dialogue scenarios can generally be divided into two types: 1) Speech emotion recognition method based on traditional manual features + machine learning classifiers. This method first extracts Mel-frequency cepstral coefficients (MFCC) or PLP (Perceptual Linear Prediction) features from the speech, and then feeds the extracted features into machine learning classifiers such as GMM-UBM or SVM for emotion classification. The disadvantages of this method are that the traditional manual features are insufficient in processing information such as noise, and secondly, the simple machine learning classifier cannot effectively find a better sample classification space, resulting in a serious lack of recognition accuracy, which is difficult to meet the scene requirements of emotion recognition in customer service dialogues; 2) Speech emotion recognition method based on MFCC features + deep learning classification network. This method first extracts MFCC features from the speech signal, and then feeds the MFCC features into the deep learning classification network to process the information. The advantage of this method lies in the deep learning classification network's ability to efficiently process multi-level information, effectively utilizing the time domain and frequency domain characteristics of speech information, and can also classify and process information such as noise as background information, which can effectively improve the accuracy of speech emotion recognition.

[0004] At present, the accuracy and real-time performance of voice emotion recognition in customer service dialogue scenarios in the financial field need to be improved. When classifying emotions, the emotions of customers and agents can be divided into three types of emotions (positive, neutral, and negative). Among them, positive and neutral emotions generally have lower recognition accuracy due to the similarity of voice emotions. Although negative information is more obvious, when the negative information of customer service or customers is incorrectly recognized, it will have a greater business impact. In addition, if the recognition speed is low, it cannot meet the business needs of the scenario. Summary of the invention

[0005] The purpose of this application is to overcome the above problems or at least partially solve or alleviate the above problems.

[0006] According to one aspect of the present application, a method for recognizing speech emotions in a conversation scene is provided, comprising:

[0007] Extract the three-channel features of each speech file in the training set, namely, MFCC features, first-order difference features, and second-order difference features, to form a three-channel feature map, and store the three-channel feature map of each speech file in a feature file;

[0008] Read the feature files to be trained in batches to form a feature data combination of data and labels, where the labels are emotion classification labels;

[0009] The above feature data are combined and fed into the improved AlexNet network and the bidirectional BiGRU network in batches, and feature sequences are aligned through CTC. The feature sequences processed by CTC are fed into the Softmax classifier, and then back-propagation training is performed according to the cross entropy loss until the loss converges to obtain the speech emotion recognition model for the dialogue scene.

[0010] Extracting three-channel features from the customer service and customer channel recordings respectively, and sending the extracted three-channel features to the conversation scene speech emotion recognition model for emotion classification;

[0011] The structure of the improved AlexNet network is:

[0012] The first layer: a convolutional layer with a convolution kernel size of 9×9, a stride of 4, a total of 128 convolution kernels, and an activation function of LeakeyRelu;

[0013] The second layer: a convolution layer with a convolution kernel size of 1×1;

[0014] The third layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0015] The fourth layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0016] The fifth layer: a convolution layer with a convolution kernel size of 1×1;

[0017] The sixth layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0018] The seventh layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0019] The eighth layer: a convolution layer with a convolution kernel size of 1×1;

[0020] Ninth layer: fully connected layer, the number of neurons is 2048;

[0021] The tenth layer: fully connected layer, the number of neurons is 1024;

[0022] Eleventh layer: fully connected layer, the number of neurons is 3.

[0023] Optionally, the feature sequence processed by CTC is sent to a fully connected layer, and the output of the fully connected layer is sent to a Softmax classifier.

[0024] Optionally, the emotion classification labels include positive, negative and neutral.

[0025] Optionally, before extracting the three-channel features of each voice file in the training set, the method further includes:

[0026] The customer service recording data with completed emotion annotations is augmented, and the augmented data is divided into a training set and a test set.

[0027] According to another aspect of the present application, a device for recognizing speech emotions in a conversation scene is provided, comprising:

[0028] A training feature extraction module is configured to extract three-channel features of each speech file in the training set, namely, MFCC features, first-order difference features, and second-order difference features, to form a three-channel feature map, and store the three-channel feature map of each speech file in a feature file;

[0029] A data labeling module, which is configured to read the feature files to be trained in batches to form a feature data combination of data-label, wherein the label is a sentiment classification label;

[0030] A training module is configured to feed the above feature data into the improved AlexNet network and the bidirectional BiGRU network in batches, align the feature sequences through CTC, feed the feature sequences processed by CTC into the Softmax classifier, and then perform back propagation training according to the cross entropy loss until the loss converges to obtain a speech emotion recognition model for dialogue scenarios; and

[0031] A recognition module, which is configured to extract three-channel features from the customer service and customer channel recordings respectively, and send the extracted three-channel features to the conversation scene speech emotion recognition model for emotion classification;

[0032] The structure of the improved AlexNet network is:

[0033] The first layer: a convolutional layer with a convolution kernel size of 9×9, a stride of 4, a total of 128 convolution kernels, and an activation function of LeakeyRelu;

[0034] The second layer: a convolution layer with a convolution kernel size of 1×1;

[0035] The third layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0036] The fourth layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0037] The fifth layer: a convolution layer with a convolution kernel size of 1×1;

[0038] The sixth layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0039] The seventh layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0040] The eighth layer: a convolution layer with a convolution kernel size of 1×1;

[0041] Ninth layer: fully connected layer, the number of neurons is 2048;

[0042] The tenth layer: fully connected layer, the number of neurons is 1024;

[0043] Eleventh layer: fully connected layer, the number of neurons is 3.

[0044] Optionally, in the training module, the feature sequence processed by CTC is sent to a fully connected layer, and the output of the fully connected layer is sent to a Softmax classifier.

[0045] Optionally, the emotion classification labels include positive, negative and neutral.

[0046] Optionally, the device further comprises:

[0047] The sample augmentation module is configured to perform data augmentation on customer service recording data with completed emotion annotations, and divide the augmented data into a training set and a test set.

[0048] According to a third aspect of the present application, a computing device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the method described in the present application is implemented when the processor executes the computer program.

[0049] The method and device for recognizing speech emotions in dialogue scenarios of the present application, because the MFCC features and the combination of first-order difference features and second-order difference features are used in the feature extraction part, the coverage capability of scene features is effectively improved; and because the improved AlexNet feature mapping network can perform emotion feature mapping on speech data, the accuracy of model recognition can be improved.

[0050] Furthermore, the conversation scene speech emotion recognition method and device of the present application, because the bidirectional BiGRU sequence network and the CTC layer are sequentially connected after the improved AlexNet network, can improve the speech sequence feature processing capability and improve the training efficiency and operation efficiency of the model.

[0051] Based on the detailed description of the specific embodiments of the present application in combination with the accompanying drawings below, those skilled in the art will become more aware of the above and other objects, advantages and features of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Hereinafter, some specific embodiments of the present application will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0053] Figure 1 is a schematic flow chart of a method for recognizing speech emotions in a conversation scenario according to an embodiment of the present application;

[0054] Figure 2 It is a schematic diagram of the principle of a method for recognizing speech emotions in a conversation scenario according to an embodiment of the present application;

[0055] Figure 3 1 is a schematic diagram of an AlexNet network structure according to an embodiment of the present application, wherein (a) represents the AlexNet network structure of the prior art, and (b) represents the improved AlexNet network structure of the embodiment of the present application;

[0056] Figure 4 is a structural schematic diagram of a speech emotion recognition device for a conversation scene according to an embodiment of the present application;

[0057] Figure 5 is a schematic diagram of a computing device according to an embodiment of the present application;

[0058] Figure 6 It is a structural schematic diagram of a computer-readable storage medium according to an embodiment of the present application. DETAILED DESCRIPTION

[0059] Figure 1 and Figure 2 They are respectively a schematic flow chart and a principle diagram of a method for recognizing speech emotions in a conversation scenario according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the method for recognizing speech emotions in a dialogue scene may generally include:

[0060] Step S1: data augmentation is performed on the customer service recording data with completed emotion annotation, and the augmented data is divided into a training set and a test set; when the number of samples is small (for example, there are only 1,000 hours of financial customer service recording data), the sample data can be firstly emotion-annotated, and then data augmentation is performed by adding noise, speeding up the speech speed, or increasing data disturbance, and then the data is divided into a training set: a test set ratio of 7:3, and the division is performed while fully considering the speaker information, so that the speaker voices of the training set and the test set are separated;

[0061] Step S2, extracting three-channel features of each voice file in the training set, namely, MFCC features, first-order difference features and second-order difference features, forming a three-channel feature map, and storing the three-channel feature map of each voice file in a feature file;

[0062] Step S3, reading the data and the annotated labels of the feature file to be trained in batches to form a feature data combination of data and labels, wherein the labels are emotion classification labels, and the emotion classification labels include positive, negative and neutral;

[0063] Step S4, feed the above feature data into the improved AlexNet and bidirectional BiGRU networks in batches, align the feature sequences through CTC, send the feature sequences processed by CTC into the fully connected layer, and send the output of the fully connected layer into the Softmax classifier to complete the forward propagation process of the classification network, and then perform back propagation training according to the cross entropy loss until the loss converges to obtain a dialogue scene speech emotion recognition model; wherein the improved AlexNet network structure is as follows Figure 3 The speech emotion recognition model is verified using the test set.

[0064] like Figure 3 As shown, the structure of the improved AlexNet network is:

[0065] The first layer: a convolutional layer with a convolution kernel size of 9×9, a stride of 4, a total of 128 convolution kernels, and an activation function of LeakeyRelu;

[0066] The second layer: a convolution layer with a convolution kernel size of 1×1;

[0067] The third layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0068] The fourth layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0069] The fifth layer: a convolution layer with a convolution kernel size of 1×1;

[0070] The sixth layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0071] The seventh layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0072] The eighth layer: a convolution layer with a convolution kernel size of 1×1;

[0073] Ninth layer: fully connected layer, the number of neurons is 2048;

[0074] The tenth layer: fully connected layer, the number of neurons is 1024;

[0075] Eleventh layer: fully connected layer, the number of neurons is 3.

[0076] according to Figure 3 It can be seen that the improved AlexNet network of the embodiment of the present application makes the following improvements to the existing AlexNet network:

[0077] 1. Change the first convolution layer in the AlexNet network to a 9x9 convolution kernel with a step size of 4, for a total of 128 convolution kernels;

[0078] 2. Change all Relu activation functions in the AlexNet network to LeakeyRelu;

[0079] 3. Change all pooling layers in the AlexNet network to 1x1 convolutional layers with a step size of 1;

[0080] 4. Remove the fourth convolutional layer and all Local Response Norm layers in the AlexNet network;

[0081] 5. Change all 5x5 convolution kernels in the AlexNet network to 3x3 convolution kernels;

[0082] 6. Change the number of neurons in the last three fully connected layers of the AlexNet network to 2048, 1024, and 3 respectively.

[0083] Step S5: obtain the conversation recording between the customer service and the user, perform channel separation, separate the customer service channel and the user channel, extract three-channel features from the customer service channel recording and the customer channel recording respectively, and send the extracted three-channel features to the conversation scene speech emotion recognition model for emotion classification.

[0084] The above method has the following characteristics:

[0085] (1) Considering the fact that there are many specific financial speech feature words in financial customer service dialogue scenarios, the MFCC features are combined with the first-order difference features and the second-order difference features to form three-dimensional features. The feature coverage capability is enhanced, which is conducive to classification learning by deep learning classification networks;

[0086] (2) The AlexNet network was improved to form a corresponding deep learning feature mapping network in the financial customer service dialogue scenario. The improved AlexNet network has the characteristics of a small number of parameters and relatively strong feature mapping capabilities. The small number of parameters makes the feature processing speed faster and improves the model operation efficiency. The strong feature mapping capability improves the accuracy of emotion recognition. In addition, it can also be used in voice classification-related demand scenarios such as voice gender recognition and voiceprint recognition.

[0087] (3) A bidirectional BiGRU network is connected to the improved AlexNet network to encode the time series information and improve the speech sequence feature processing capability. Compared with the BiLSTM network, this effectively reduces the number of network parameters and speeds up the network operation without significantly affecting the network accuracy.

[0088] (4) The CTC layer is connected after the BiGRU network. CTC can effectively perform feature sequence alignment and form a strong and effective feature sequence alignment reinforcement with BiGRU, which effectively improves the network's feature sequence alignment processing capabilities. The combination with BiGRU effectively speeds up network training and operation.

[0089] The present application also provides a speech emotion recognition device for a dialogue scene, such as Figure 4 As shown, the device comprises:

[0090] Sample augmentation module 1, which is configured to perform data augmentation on customer service recording data with completed emotion annotations, and divide the augmented data into a training set and a test set;

[0091] A training feature extraction module 2 is configured to extract three-channel features of each voice file in the training set, namely, MFCC features, first-order difference features, and second-order difference features, to form a three-channel feature map, and store the three-channel feature map of each voice file in a feature file;

[0092] A data labeling module 3, which is configured to read the feature files to be trained in batches to form a feature data combination of data-label, wherein the label is an emotion classification label, and the emotion classification label includes positive, negative and neutral;

[0093] Training module 4 is configured to feed the above feature data combinations into the improved AlexNet network and the bidirectional BiGRU network in batches, perform feature sequence alignment through CTC, feed the feature sequence processed by CTC into the fully connected layer, feed the output of the fully connected layer into the Softmax classifier, and then perform back propagation training according to the cross entropy loss until the loss converges to obtain a speech emotion recognition model for dialogue scenarios; and

[0094] Recognition module 5, which is configured to extract three-channel features from the customer service and customer channel recordings respectively, and send the extracted three-channel features to the dialogue scene speech emotion recognition model for emotion classification;

[0095] The structure of the improved AlexNet network is:

[0096] The first layer: a convolutional layer with a convolution kernel size of 9×9, a stride of 4, a total of 128 convolution kernels, and an activation function of LeakeyRelu;

[0097] The second layer: a convolution layer with a convolution kernel size of 1×1;

[0098] The third layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0099] The fourth layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0100] The fifth layer: a convolution layer with a convolution kernel size of 1×1;

[0101] The sixth layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0102] The seventh layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu;

[0103] The eighth layer: a convolution layer with a convolution kernel size of 1×1;

[0104] Ninth layer: fully connected layer, the number of neurons is 2048;

[0105] The tenth layer: fully connected layer, the number of neurons is 1024;

[0106] Eleventh layer: fully connected layer, the number of neurons is 3.

[0107] The conversation scene speech emotion recognition device of the embodiment of the present application is capable of executing the steps of the conversation scene speech emotion recognition method of the embodiment of the present application, and its principles and effects are not repeated here.

[0108] The present application also provides a computing device, referring to Figure 5 The computing device includes a memory 1120, a processor 1110, and a computer program stored in the memory 1120 and executable by the processor 1110. The computer program is stored in a space 1130 for program code in the memory 1120. When the computer program is executed by the processor 1110, it is implemented to execute any one of the method steps 1131 according to the present invention.

[0109] The present application also provides a computer-readable storage medium. Figure 6 The computer-readable storage medium comprises a storage unit for program code, the storage unit being provided with a program 1131 ′ for executing the method steps according to the present invention, the program being executed by a processor.

[0110] The embodiment of the present application further provides a computer program product including instructions. When the computer program product is run on a computer, the computer is caused to execute the method steps according to the present invention.

[0111] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer loads and executes the computer program instructions, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0112] The professionals should also be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0113] A person skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing a processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state drive, a magnetic tape, a floppy disk, an optical disc, and any combination thereof.

[0114] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for recognizing speech emotions in a conversation scene, comprising: Extract the three-channel features of each speech file in the training set, namely, MFCC features, first-order difference features, and second-order difference features, to form a three-channel feature map, and store the three-channel feature map of each speech file in a feature file; Read the feature files to be trained in batches to form a feature data combination of data and labels, where the labels are emotion classification labels; The above feature data are combined and fed into the improved AlexNet network and the bidirectional BiGRU network in batches, and feature sequences are aligned through CTC. The feature sequences processed by CTC are fed into the Softmax classifier, and then back-propagation training is performed according to the cross entropy loss until the loss converges to obtain the speech emotion recognition model for dialogue scenarios. Extracting three-channel features from the customer service and customer voice channel recordings respectively, and sending the extracted three-channel features to the conversation scene speech emotion recognition model for emotion classification; The structure of the improved AlexNet network is: The first layer: a convolutional layer with a kernel size of 9×9, a stride of 4, a total of 128 convolution kernels, and an activation function of LeakeyRelu; The second layer: a convolution layer with a convolution kernel size of 1×1; The third layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu; The fourth layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu; The fifth layer: a convolution layer with a convolution kernel size of 1×1; The sixth layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu; The seventh layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu; The eighth layer: a convolution layer with a convolution kernel size of 1×1; Ninth layer: fully connected layer, the number of neurons is 2048; The tenth layer: fully connected layer, the number of neurons is 1024; Eleventh layer: fully connected layer, the number of neurons is 3.

2. The method according to claim 1, characterized in that The feature sequence processed by CTC is sent to the fully connected layer, and the output of the fully connected layer is sent to the Softmax classifier.

3. The method according to claim 2, characterized in that The emotion classification labels include positive, negative and neutral.

4. The method according to any one of claims 1 to 3, characterized in that Before extracting the three-channel features of each voice file in the training set, the method further includes: The customer service recording data with completed emotion annotations is augmented, and the augmented data is divided into a training set and a test set.

5. A speech emotion recognition device for a dialogue scene, comprising: A training feature extraction module is configured to extract three-channel features of each speech file in the training set, namely, MFCC features, first-order difference features, and second-order difference features, to form a three-channel feature map, and store the three-channel feature map of each speech file in a feature file; A data labeling module, which is configured to read the feature files to be trained in batches to form a feature data combination of data-label, wherein the label is a sentiment classification label; The training module is configured to feed the above feature data into the improved AlexNet network and the bidirectional BiGRU network in batches, align the feature sequences through CTC, feed the feature sequences processed by CTC into the Softmax classifier, and then perform back propagation training according to the cross entropy loss until the loss converges to obtain the speech emotion recognition model for the dialogue scene; and A recognition module, which is configured to extract three-channel features from the customer service and customer channel recordings respectively, and send the extracted three-channel features to the conversation scene speech emotion recognition model for emotion classification; The structure of the improved AlexNet network is: The first layer: a convolutional layer with a kernel size of 9×9, a stride of 4, a total of 128 convolution kernels, and an activation function of LeakeyRelu; The second layer: a convolution layer with a convolution kernel size of 1×1; The third layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu; The fourth layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu; The fifth layer: a convolution layer with a convolution kernel size of 1×1; The sixth layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu; The seventh layer: a convolutional layer with a kernel size of 3×3, a stride of 1, and an activation function of LeakeyRelu; The eighth layer: a convolution layer with a convolution kernel size of 1×1; Ninth layer: fully connected layer, the number of neurons is 2048; The tenth layer: fully connected layer, the number of neurons is 1024; Eleventh layer: fully connected layer, the number of neurons is 3.

6. The device according to claim 5, characterized in that In the training module, the feature sequence processed by CTC is sent to the fully connected layer, and the output of the fully connected layer is sent to the Softmax classifier.

7. The device according to claim 6, characterized in that The emotion classification labels include positive, negative and neutral.

8. The device according to any one of claims 5 to 7, characterized in that: Also includes: The sample augmentation module is configured to perform data augmentation on customer service recording data that has completed emotion annotation, and divide the augmented data into a training set and a test set.

9. A computing device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 3 is implemented.

Images