Emotion recognition method and device based on multi-classifier, electronic equipment and medium

Abstract

This application provides a method, apparatus, electronic device, and medium for emotion recognition based on a multi-classifier, belonging to the fields of artificial intelligence and fintech. The method includes: acquiring a first unlabeled speech dataset and a second speech dataset with original labels; predicting first, second, and third predicted labels using first, second, and third classifiers based on the first speech dataset; constructing target label data based on the first, second, and third predicted labels and the original labels; constructing a training speech dataset based on the target label data and the first speech dataset; training the first, second, and third classifiers using the training speech dataset; performing classifier selection on the first, second, and third preliminary classifiers based on the second speech dataset to obtain the target classifier with the strongest classification ability; and inputting the target speech data into the target classifier for emotion recognition, resulting in a more accurate target emotion category. This application can improve the accuracy of emotion recognition.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and financial technology, and in particular to an emotion recognition method and device, electronic device, and medium based on a multi-classifier. Background Technology

[0002] Speech emotion recognition (SER) is a technology used to automatically identify the emotional state of a speaker. It is widely used in banking, insurance, call centers, social robots, and healthcare. Traditional SER methods primarily rely on deep learning, extracting the spectrum from audio and then performing feature extraction based on this spectrum to identify emotions. Currently, datasets for deep learning-based SER are relatively scarce compared to those for computer vision and natural language processing. Furthermore, in practical applications, the emotion expressed by the same sentence is not constant across different contexts. Current methods, such as assigning one or more labels to a sentence to identify emotions, have low accuracy. Therefore, improving the accuracy of emotion recognition has become a pressing technical challenge. Summary of the Invention

[0003] The main objective of this application is to propose an emotion recognition method, apparatus, electronic device, and medium based on a multi-classifier, aiming to improve the accuracy of emotion recognition.

[0004] To achieve the above objectives, a first aspect of this application proposes a multi-classifier-based emotion recognition method, the method comprising:

[0005] Acquire sample speech data; wherein, the sample speech data includes a first speech dataset without labels and a second speech dataset with original labels;

[0006] A first predicted label is obtained by predicting the first classifier based on the first speech dataset; a second predicted label is obtained by predicting the second classifier based on the first speech dataset; and a third predicted label is obtained by predicting the third classifier based on the first speech dataset.

[0007] Target label data is constructed based on the first predicted label, the second predicted label, the third predicted label, and the original label;

[0008] A training speech dataset is constructed based on the target label data and the first speech dataset;

[0009] A first preliminary classifier is obtained by training the first classifier based on the training speech dataset; a second preliminary classifier is obtained by training the second classifier based on the training speech dataset; and a third preliminary classifier is obtained by training the third classifier based on the training speech dataset.

[0010] Based on the second speech dataset, the first preliminary classifier, the second preliminary classifier, and the third preliminary classifier are filtered to obtain the target classifier;

[0011] The acquired target speech data is input into the target classifier for emotion recognition to obtain the target emotion category.

[0012] In some embodiments, constructing target label data based on the first predicted label, the second predicted label, the third predicted label, and the original label includes:

[0013] Obtain the dimensions of the original labels to get the label dimensions;

[0014] The tag probability is calculated based on the first predicted tag, the second predicted tag, and the third predicted tag to obtain the interaction tag;

[0015] The target label data is constructed based on the interactive labels and the label dimensions.

[0016] In some embodiments, constructing the target tag data based on the interaction tag and the tag dimension includes:

[0017] The number of tag categories is obtained based on the interactive tags, thus obtaining the tag category count;

[0018] Compare the number of tag categories and the tag dimensions;

[0019] If the number of tag categories and the number of tag dimensions are equal, then the interactive tag is used as the target tag data;

[0020] If the number of tag categories is less than the number of tag dimensions, then the target tag data is constructed based on the interactive tags and a preset number of original tags; wherein the preset number is equal to the difference between the tag dimensions and the number of tag categories.

[0021] In some embodiments, the step of filtering the first preliminary classifier, the second preliminary classifier, and the third preliminary classifier based on the second speech dataset to obtain a target classifier includes:

[0022] The first preliminary classifier is validated based on the second speech dataset to obtain a first validation label;

[0023] The second preliminary classifier is validated based on the second speech dataset to obtain a second validation label;

[0024] The third preliminary classifier is validated based on the second speech dataset to obtain a third validation label;

[0025] Target verification tags are selected from the first verification tag, the second verification tag, and the third verification tag;

[0026] The target classifier is selected from the first preliminary classifier, the second preliminary classifier, and the third preliminary classifier based on the target verification label.

[0027] In some embodiments, the step of filtering the target verification label from the first verification label, the second verification label, and the third verification label includes:

[0028] First accurate data is obtained by obtaining the accuracy of the first verification tag, second accurate data is obtained by obtaining the accuracy of the second verification tag, and third accurate data is obtained by obtaining the accuracy of the third verification tag.

[0029] Select one data point from the first accurate data, the second accurate data, and the third accurate data as the target accurate data;

[0030] Based on the target accurate data, a verification label is selected from the first verification label, the second verification label, and the third verification label as the target verification label.

[0031] In some embodiments, the sample speech data further includes a third speech dataset with the original labels, and the method further includes: training a preset number of original classifiers based on the third speech dataset, specifically including:

[0032] Obtain the number of samples in the third speech dataset to obtain preliminary sample parameters;

[0033] The third speech dataset is input into the original classifier for prediction to obtain the original output.

[0034] Construct cross-entropy loss data based on the original labels, the preliminary sample parameters, and the original output;

[0035] The preset number of original classifiers are trained based on the cross-entropy loss data to obtain the first classifier, the second classifier, and the third classifier.

[0036] In some embodiments, training the first classifier to obtain a first preliminary classifier based on the training speech dataset, training the second classifier to obtain a second preliminary classifier based on the training speech dataset, and training the third classifier to obtain a third preliminary classifier based on the training speech dataset include:

[0037] Obtain the number of samples in the training speech dataset to obtain the training sample parameters;

[0038] The training speech dataset is input into the first classifier to make a prediction and obtain the first prediction data; the training speech dataset is input into the second classifier to make a prediction and obtain the second prediction data; and the training speech dataset is input into the third classifier to make a prediction and obtain the third prediction data.

[0039] Relative entropy loss data is constructed based on the target label data, the training sample parameters, the first prediction data, the second prediction data, and the third prediction data;

[0040] The first preliminary classifier is obtained by training the first classifier based on the relative entropy loss data, the second preliminary classifier is obtained by training the second classifier based on the relative entropy loss data, and the third preliminary classifier is obtained by training the third classifier based on the relative entropy loss data.

[0041] To achieve the above objectives, a second aspect of this application provides an emotion recognition device based on a multi-classifier, the device comprising:

[0042] A sample speech data acquisition module is used to acquire sample speech data; wherein, the sample speech data includes a first speech dataset without labels and a second speech dataset with original labels;

[0043] The first classifier training module is used to: predict a first predicted label by a first classifier based on the first speech dataset, predict a second predicted label by a second classifier based on the first speech dataset, and predict a third predicted label by a third classifier based on the first speech dataset.

[0044] The target label construction module is used to construct target label data based on the first predicted label, the second predicted label, the third predicted label, and the original label;

[0045] A training dataset construction module is used to construct a training speech dataset based on the target label data and the first speech dataset;

[0046] The second classifier training module is configured to: train the first classifier to obtain a first preliminary classifier based on the training speech dataset; train the second classifier to obtain a second preliminary classifier based on the training speech dataset; and train the third classifier to obtain a third preliminary classifier based on the training speech dataset.

[0047] The classifier filtering module is used to filter the first preliminary classifier, the second preliminary classifier, and the third preliminary classifier based on the second speech dataset to obtain the target classifier.

[0048] The emotion recognition module is used to input the acquired target speech data into the target classifier for emotion recognition and to obtain the target emotion category.

[0049] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0050] To achieve the above objectives, a fourth aspect of the present application provides a storage medium, which is a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0051] The present application proposes a multi-classifier-based emotion recognition method, apparatus, electronic device, and medium. It acquires a first unlabeled speech dataset and a second speech dataset with original labels. Based on the first speech dataset, a first classifier predicts a first predicted label; based on the first speech dataset, a second classifier predicts a second predicted label; and based on the first speech dataset, a third classifier predicts a third predicted label. Target label data is then constructed based on the first predicted label, the second predicted label, the third predicted label, and the original labels. A training speech dataset is constructed based on the target label data and the first speech dataset. The first classifier is trained using the training speech dataset to obtain a first preliminary classifier; the second classifier is trained using the training speech dataset to obtain a second preliminary classifier; and the third classifier is trained using the training speech dataset to obtain a third preliminary classifier. Finally, the first, second, and third preliminary classifiers are filtered using the second speech dataset to obtain a target classifier with strong classification capabilities. The acquired target speech data is then input into the target classifier for emotion recognition to obtain the target emotion category, thereby improving the accuracy of emotion recognition. Attached Figure Description

[0052] Figure 1 This is a flowchart of the emotion recognition method based on a multi-classifier provided in the embodiments of this application;

[0053] Figure 2 This is another flowchart of the emotion recognition method based on a multi-classifier provided in the embodiments of this application;

[0054] Figure 3 yes Figure 1 The flowchart for step 103 in the text;

[0055] Figure 4 yes Figure 3 The flowchart for step 303 in the document;

[0056] Figure 5 yes Figure 1 The flowchart for step 105 in the document;

[0057] Figure 6 yes Figure 1 The flowchart for step 106 in the document;

[0058] Figure 7 Yes, yes Figure 6 The flowchart for step 604 in the document;

[0059] Figure 8 This is a schematic diagram of the structure of the emotion recognition device based on a multi-classifier provided in an embodiment of this application;

[0060] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0064] First, let's analyze some of the terms used in this application:

[0065] Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0066] Natural Language Processing (NLP): NLP uses computers to process, understand, and utilize human language (such as Chinese and English). NLP is a branch of artificial intelligence and an interdisciplinary field of computer science and linguistics, often referred to as computational linguistics. NLP includes syntactic analysis, semantic analysis, and discourse understanding. It is commonly used in machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, intent recognition, information extraction and filtering, text classification and clustering, sentiment analysis, and opinion mining. It involves data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computation.

[0067] Speech Emotion Recognition (SER): Speech emotion recognition is a technology used to automatically identify the emotional state of a speaker. It is widely used in banking, insurance, call centers, social robots, and healthcare. Traditional speech emotion recognition methods are primarily based on deep learning, extracting the spectrum from audio and performing feature extraction based on this spectrum to identify emotions. Currently, datasets for deep learning-based speech emotion recognition are fewer compared to fields like computer vision and natural language processing. Furthermore, in practical applications, the emotion expressed by the same sentence is not constant across different contexts. Current methods for identifying emotions by assigning one or more labels to a sentence have low accuracy.

[0068] Based on this, embodiments of this application provide a method, apparatus, electronic device, and medium for emotion recognition based on a multi-classifier, aiming to improve the accuracy of emotion recognition.

[0069] The emotion recognition method, apparatus, electronic device, and medium based on multi-classifier provided in this application are specifically described through the following embodiments. First, the emotion recognition method based on multi-classifier in this application is described.

[0070] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0071] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0072] The emotion recognition method based on a multi-classifier provided in this application relates to the field of artificial intelligence technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the emotion recognition method based on a multi-classifier, but is not limited to the above forms.

[0073] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0074] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user voice data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to a confirmation page. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the normal operation of embodiments of this application obtained.

[0075] Figure 1 This is an optional flowchart of the emotion recognition method based on a multi-classifier provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps 101 to 107.

[0076] Step 101: Obtain sample speech data; wherein, the sample speech data includes a first speech dataset without labels and a second speech dataset with original labels;

[0077] Step 102: Based on the first speech dataset, predict the first classifier to obtain the first predicted label, based on the first speech dataset, predict the second classifier to obtain the second predicted label, and based on the first speech dataset, predict the third predicted label using the third classifier.

[0078] Step 103: Construct target label data based on the first predicted label, the second predicted label, the third predicted label, and the original label;

[0079] Step 104: Construct a training speech dataset based on the target label data and the first speech dataset;

[0080] Step 105: Train the first classifier based on the training speech dataset to obtain the first preliminary classifier; train the second classifier based on the training speech dataset to obtain the second preliminary classifier; and train the third classifier based on the training speech dataset to obtain the third preliminary classifier.

[0081] Step 106: Based on the second speech dataset, classifiers 1, 2, and 3 are selected to obtain the target classifier.

[0082] Step 107: Input the acquired target speech data into the target classifier for emotion recognition to obtain the target emotion category.

[0083] Steps 101 to 107 as illustrated in this embodiment involve acquiring a first unlabeled speech dataset and a second speech dataset with original labels. A first predicted label is obtained by predicting with a first classifier based on the first speech dataset. A second predicted label is obtained by predicting with a second classifier based on the first speech dataset. A third predicted label is obtained by predicting with a third classifier based on the first speech dataset. Target label data is constructed based on the first predicted label, second predicted label, third predicted label, and original label. A training speech dataset is constructed based on the target label data and the first speech dataset. A first preliminary classifier is trained using the training speech dataset. A second preliminary classifier is trained using the training speech dataset. A third preliminary classifier is trained using the training speech dataset. Classifier selection is performed using the first, second, and third preliminary classifiers based on the second speech dataset. The resulting target classifier is the one with the strongest classification ability. The acquired target speech data is input into the target classifier for emotion recognition to obtain the target emotion category, which can improve the accuracy of emotion recognition.

[0084] In step 101 of some embodiments, the sample speech data can be obtained from existing speech datasets, such as IEMOCAP data or CAISA data. The speech dataset is a dataset with precise single labels. IEMOCAP data, for example, could be 12 hours of audiovisual data from 10 actors, and could include five emotion categories: happiness, anger, sadness, frustration, and neutral. In fintech scenarios, the sample speech data could be speech data from policyholders seeking insurance advice or from users inquiring about financial planning; in the medical field, the sample speech data could also be speech data from patients seeking medical consultation. The original label is a precise label, such as at least one of the five emotion categories mentioned above: happiness, anger, sadness, frustration, and neutral. The first speech dataset of the sample speech data is a dataset without emotion category labels, which can be obtained by removing labels from an existing dataset with original labels. The second voice dataset of the sample voice data is a dataset with original labels. These original labels can be used to represent emotion categories, such as at least one of the five emotion categories mentioned above: happiness, anger, sadness, frustration, and neutrality. The original labels can also represent other emotion categories, such as anxiety, enthusiasm, etc. This application embodiment does not limit this.

[0085] In some embodiments, the sample speech data also includes a third speech dataset with original labels; see [link to documentation]. Figure 2 In some embodiments, before performing step 102, the emotion recognition method based on a multi-classifier further includes: training a preset number of original classifiers based on a third speech dataset, specifically including but not limited to steps 201 to 204:

[0086] Step 201: Obtain the number of samples in the third speech dataset to obtain preliminary sample parameters;

[0087] Step 202: Input the third speech dataset into the original classifier for prediction to obtain the original output;

[0088] Step 203: Construct cross-entropy loss data based on the original labels, preliminary sample parameters, and original output;

[0089] Step 204: Train a preset number of original classifiers based on the cross-entropy loss data to obtain a first classifier, a second classifier, and a third classifier.

[0090] In one application scenario, the third speech dataset is also a dataset with precise labels. The original labels of the third speech dataset can be the same as the original labels of the second speech dataset. The original labels of the third speech dataset can be used to represent at least one of the aforementioned five emotion categories: happiness, anger, sadness, frustration, and neutrality. The original labels of the third speech dataset can also represent other emotion categories, such as anxiety, enthusiasm, etc. This application embodiment does not limit this.

[0091] In some embodiments, the ratio of the first voice dataset, the second voice dataset, and the third voice dataset can be 1:3:1.

[0092] In some embodiments, the preset number is at least three. This application embodiment uses three preset number for illustration. The three original classifiers are: a first original classifier, a second original classifier, and a third original classifier. The first original classifier is an L3 Net pre-trained on AudioSet 60M. L3 Net is a method that learns embeddings through an audio-visual correspondence auxiliary task. L3 Net can determine whether a video image frame and a 1-second audio clip come from the same video and overlap temporally. The L3 Net network architecture consists of three parts: a visual sub-network for extracting visual features, an audio sub-network for extracting audio features, and a fusion layer that predicts the correspondence between the video and audio modalities. Since the matching of image and audio pairs can be automatically generated from the training data, for example, by obtaining images and audio from the same or different videos respectively, manual labeling is not required to train the model. The second original classifier is a VGGish model pre-trained on YouTube-8M. AudioSet is a dataset consisting of 10-second YouTube video audio tracks labeled by 2 million individuals. Its labels come from an ontology of over 600 audio event classes. A VGGish model was trained on this large YouTube dataset to generate 128-dimensional embeddings. The third original classifier is a wav2vec 2.0 pre-trained on LibriSpeech, which can utilize large amounts of unlabeled data for unsupervised learning.

[0093] In step 201 of some embodiments, the target sample parameter obtained by acquiring the number of samples in the third speech dataset can be represented as m.

[0094] In step 202 of some embodiments, the original output can be represented as Taking three original classifiers as an example, the resulting original outputs are also three: the first original output, the second original output, and the third original output.

[0095] In step 203 of some embodiments, the cross-entropy loss data constructed based on the original labels, preliminary sample parameters, and original output is the cross-entropy loss function, as shown in equation (1) below:

[0096]

[0097] Where, m represents the number of samples in the third speech dataset, i.e., the initial sample parameters; j represents the sample number in the third speech dataset, for example, j=2 represents the second sample in the third speech dataset; i is the number of the original classifier, for example, i=1 represents the first original classifier, i=2 represents the second original classifier, and i=3 represents the third original classifier; y j It is the original tag. This represents the output of the corresponding original classifier, for example, This represents the output of the first original classifier. This represents the output generated by inputting the second sample from the third speech dataset into the first original classifier. This represents the output of the second original classifier. This represents the output of the second sample from the third speech dataset, input into the second original classifier. This represents the output of the third original classifier. This represents the output generated by inputting the third sample from the third speech dataset into the third original classifier.

[0098] In step 204 of some embodiments, the first original classifier, the second original classifier, and the third original classifier are trained based on the cross-entropy loss data until the cross-entropy loss function converges, thus obtaining the first classifier, the second classifier, and the third classifier.

[0099] In step 102 of some embodiments, a first predicted label is obtained by predicting and voting on a first classifier based on an unlabeled first speech dataset; a second predicted label is obtained by predicting and voting on a second classifier; and a third predicted label is obtained by predicting and voting on a third classifier. In one application scenario, the original labels are: happy, calm, sad, angry; the first predicted label is happy, the second predicted label is happy, and the third predicted label is calm. The target label data obtained in step 103 is then represented as (2 / 3, 1 / 3, 0, 0). For details, please refer to the detailed description of step 103.

[0100] Please see Figure 3 In some embodiments, step 103 may include, but is not limited to, steps 301 to 303:

[0101] Step 301: Obtain the dimensions of the original tags to get the tag dimensions;

[0102] Step 302: Calculate the label probability based on the first predicted label, the second predicted label, and the third predicted label to obtain the interaction label;

[0103] Step 303: Construct target tag data based on interactive tags and tag dimensions.

[0104] In step 301 of some embodiments, taking the original labels as: happy, calm, sad, angry as an example, the dimension of the original labels is four-dimensional, that is, the label dimension is four-dimensional.

[0105] In step 302 of some embodiments, taking the example of "the first predicted label is happy, the second predicted label is happy, and the third predicted label is calm" in step 102 above, since both the first and second predicted labels are happy, the frequency of the label "happy" is 2. Since the third predicted label is calm, the frequency of the label "calm" is 1. The total frequency of all predicted labels is 2 + 1 = 3. Therefore, the probability of the label "happy" is 2 / 3, and the probability of the label "calm" is 1 / 3. The resulting interactive labels are 2 / 3 (representing happiness) and 1 / 3 (representing calm). The target label data obtained in step 303 is a fuzzy label.

[0106] Please see Figure 4 In some embodiments, step 303 may include, but is not limited to, steps 401 to 404:

[0107] Step 401: Obtain the number of tag categories based on the interactive tags to get the number of tag categories;

[0108] Step 402: Compare the number of tag categories and the tag dimensions;

[0109] Step 403: If the number of tag categories and the number of tag dimensions are equal, then the interactive tags will be used as the target tag data.

[0110] Step 404: If the number of tag categories is less than the number of tag dimensions, then construct the target tag data based on the interactive tags and a preset number of original tags; where the preset number is equal to the difference between the tag dimensions and the number of tag categories.

[0111] In step 401 of some embodiments, combining the examples of steps 301 to 302 above, the number of label categories is 2, and the label dimension is 4. In step 402, the number of label categories 2 and the label dimension 4 are compared. Since the number of label categories 2 is less than the label dimension 4, step 104 is executed. The preset quantity = label dimension 4 - label category 2 = 2. The interactive labels only involve "happy" and "calm". Compared with the original labels "happy, calm, sad, angry", there are 2 fewer labels: sad and angry. Therefore, they are filled with 0, and the target label data is represented as (2 / 3, 1 / 3, 0, 0). This target label data is a fuzzy label. In other embodiments, if the original labels are "happy, calm, sad" (i.e., the label dimension is 3), the first predicted label is happy, the second predicted label is sad, and the third predicted label is calm, then the number of label categories and the label dimension are equal. Step 403 is executed, and the target label data is (1 / 3, 1 / 3, 1 / 3).

[0112] In another application scenario, the original labels are: happy, calm, sad, angry; the first predicted label is happy, the second predicted label is sad, and the third predicted label is calm. The target label data obtained in step 103 is represented as (1 / 3, 1 / 3, 1 / 3, 0), which is a fuzzy label. In yet another application scenario, the original labels are: happy, calm, sad, angry; the first predicted label is angry, the second predicted label is calm, and the third predicted label is sad. The target label data obtained in step 103 is represented as (0, 1 / 3, 1 / 3, 1 / 3), which is a fuzzy label. In one application scenario, the target label data can be represented as Y = (ya, yb, yc, yd | ya + yb + yc + yd = 1), where the values ​​of ya, yb, yc, and yd range from [0, 1], ya represents happiness, yb represents calm, yc represents sadness, and yd represents anger.

[0113] In step 104 of some embodiments, a training speech dataset is constructed based on the target label data and the first speech dataset. The resulting training speech dataset includes the first speech dataset and the target label data, that is, the training speech dataset is obtained by adding the target label data as labels to the first speech dataset.

[0114] Please see Figure 5 In some embodiments, step 105 may include, but is not limited to, steps 501 to 504:

[0115] Step 501: Obtain the number of samples in the training speech dataset and obtain the training sample parameters;

[0116] Step 502: Input the training speech dataset into the first classifier to make predictions and obtain the first prediction data; input the training speech dataset into the second classifier to make predictions and obtain the second prediction data; input the training speech dataset into the third classifier to make predictions and obtain the third prediction data.

[0117] Step 503: Construct relative entropy loss data based on target label data, training sample parameters, first prediction data, second prediction data, and third prediction data;

[0118] Step 504: Train the first classifier based on the relative entropy loss data to obtain the first preliminary classifier; train the second classifier based on the relative entropy loss data to obtain the second preliminary classifier; and train the third classifier based on the relative entropy loss data to obtain the third preliminary classifier.

[0119] In step 501 of some embodiments, the training sample parameter obtained by obtaining the number of samples in the training speech dataset can be represented as n.

[0120] In step 502 of some embodiments, the first prediction data, the second prediction data, and the third prediction data can be represented as:

[0121] In step 503 of some embodiments, the target label data is the label data of the training speech dataset. The relative entropy loss data constructed based on the target label data, training sample parameters, first prediction data, second prediction data, and third prediction data is the relative entropy loss function, which is the KL divergence, as shown in the following equation (2):

[0122]

[0123] Where n represents the number of samples in the second speech dataset; b represents the sequence number of the sample in the second speech dataset, for example, b=1 represents the first sample in the second speech dataset; a is the sequence number of the preliminary classifier, for example, a=1 represents the first preliminary classifier, a=2 represents the second preliminary classifier, and a=3 represents the third preliminary classifier. Indicates interactive tags, This represents the output of the corresponding preliminary classifier, for example, This represents the output of the first preliminary classifier. This represents the output of the first sample from the second speech dataset input to the first preliminary classifier. This represents the output of the second preliminary classifier. This represents the output of the second preliminary classifier, which is the input of the first sample from the second speech dataset. This represents the output of the third preliminary classifier. This represents the output generated by inputting the first sample from the second speech dataset into the third preliminary classifier.

[0124] In step 504 of some embodiments, a first preliminary classifier is obtained by training a first classifier according to the relative entropy loss function, a second preliminary classifier is obtained by training a second classifier according to the relative entropy loss function, and a third preliminary classifier is obtained by training a third classifier according to the relative entropy loss function.

[0125] Steps 501 to 504 are used to mimic human interaction in order to train the first classifier, the second classifier, and the third classifier.

[0126] Please see Figure 6 In some embodiments, step 106 may include, but is not limited to, steps 601 to 605:

[0127] Step 601: Validate the first preliminary classifier based on the second speech dataset to obtain the first validation label;

[0128] Step 602: Validate the second preliminary classifier based on the second speech dataset to obtain the second validation label;

[0129] Step 603: Validate the third preliminary classifier based on the second speech dataset to obtain the third validation label;

[0130] Step 604: Filter out the target verification label from the first verification label, the second verification label, and the third verification label;

[0131] Step 605: Select the target classifier from the first preliminary classifier, the second preliminary classifier, and the third preliminary classifier based on the target verification label.

[0132] In step 601 of some embodiments, the second speech dataset is used to validate the first preliminary classifier. Specifically, the second speech dataset is input into the first preliminary classifier for prediction, and the first preliminary classifier outputs a first validation label.

[0133] In step 602 of some embodiments, the second speech dataset is used to validate the second preliminary classifier. Specifically, the second speech dataset is input into the second preliminary classifier for prediction, and the second preliminary classifier outputs a first validation label.

[0134] In step 603 of some embodiments, the second speech dataset is used to validate the third preliminary classifier. Specifically, the second speech dataset is input into the third preliminary classifier for prediction, and the third preliminary classifier outputs a first validation label.

[0135] In step 604 of some embodiments, a verification label is selected from the first verification label, the second verification label, and the third verification label as the target verification label, and the preliminary classifier corresponding to the target verification label is the classifier with the strongest classification ability among the three preliminary classifiers.

[0136] In step 605 of some embodiments, a preliminary classifier is selected from the first preliminary classifier, the second preliminary classifier, and the third preliminary classifier as the target classifier based on the target verification label. Specifically, the preliminary classifier corresponding to the target verification label is used as the target classifier.

[0137] In this embodiment, a first classifier is first trained on a first original classifier using a third speech dataset to obtain a first classifier, and then trained on the first classifier using a training speech dataset to obtain a first preliminary classifier. Similarly, a second classifier is first trained on a second original classifier using a third speech dataset to obtain a second classifier, and then trained on the second classifier using a training speech dataset to obtain a second preliminary classifier. Likewise, a third classifier is first trained on a third original classifier using a third speech dataset to obtain a third classifier, and then trained on the third classifier using a training speech dataset to obtain a third preliminary classifier. Then, the first preliminary classifier, the second preliminary classifier, and the third preliminary classifier are validated using the second speech dataset, and a target classifier is selected based on the validation results. This target classifier is the classifier with the strongest classification ability among the three classifiers. Compared to the traditional method of averaging the output results of multiple classifiers, this embodiment can improve the accuracy of emotion recognition.

[0138] According to the optimal interaction theory, a group rarely exceeds its best member. Therefore, in this embodiment, instead of using the average output of three classifiers, the classifier with the strongest classification ability is selected as the target classifier, which can improve the accuracy of emotion recognition.

[0139] Please see Figure 7 In some embodiments, step 604 may include, but is not limited to, steps 701 to 703:

[0140] Step 701: Obtain the accuracy of the first verification label to obtain the first accurate data, obtain the accuracy of the second verification label to obtain the second accurate data, and obtain the accuracy of the third verification label to obtain the third accurate data.

[0141] Step 702: Select one data point from the first accurate data, the second accurate data, and the third accurate data as the target accurate data;

[0142] Step 703: Select one verification label from the first verification label, the second verification label, and the third verification label as the target verification label based on the target accurate data.

[0143] In step 701 of some embodiments, the first verification label is compared with the actual label to determine the accuracy of the first verification label, obtaining first accurate data; similarly, the first verification label is compared with the actual label to determine the accuracy of the second verification label, obtaining second accurate data, and the third verification label is compared with the actual label to determine the accuracy of the third verification label, obtaining third accurate data. The first accurate data can be used to represent the accuracy of the first preliminary classifier for emotion recognition, the second accurate data can be used to represent the accuracy of the second preliminary classifier for emotion recognition, and the third accurate data can be used to represent the accuracy of the third preliminary classifier for emotion recognition.

[0144] In step 702 of some embodiments, one data point is selected from the first accurate data, the second accurate data, and the third accurate data as the target accurate data, which represents the data with the highest accuracy among the three accurate data points.

[0145] In step 703 of some embodiments, a verification label is selected from the first verification label, the second verification label, and the third verification label as the target verification label based on the target accurate data. That is, the verification label corresponding to the target accurate data is selected as the target verification label, so that the preliminary classifier corresponding to the target verification label is the classifier with the strongest classification ability among the three preliminary classifiers.

[0146] For example, if the accuracy of the first accurate data is higher than that of the second accurate data, and the accuracy of the second accurate data is higher than that of the third accurate data, it means that the accuracy of the first accurate data is the highest. In this case, the first verification label corresponding to the first accurate data is used as the target verification label, and the first preliminary classifier used to output the first verification label is used as the target classifier.

[0147] Step 107: The acquired target speech data is input into a target classifier for emotion recognition. The target classifier classifies the target speech data according to emotion to obtain a target emotion label, which is the target emotion category. For example, the target emotion label obtained after the target speech data is input into the target classifier for emotion recognition is "happy".

[0148] Emotion is a crucial human state, and accurately recognizing emotions can significantly improve the user experience of human-computer interaction. For example, in financial scenarios, intelligent voice customer service can adjust waiting music or upgrade customer service based on the customer's emotional state; smart furniture can adjust background music based on the resident's emotional state, and even directly provide emotional value; in insurance or wealth management sales, understanding user satisfaction with insurance or wealth management products can help platforms develop better sales strategies; in medical scenarios, intelligent consultation systems can respond appropriately to patients based on their emotional state, improving the patient's experience; and in the film and television industry, understanding audience emotions can help develop more compelling storylines and schedule specific program releases. In human-computer dialogue, understanding human emotional states helps intelligent robots respond appropriately and express reassurance and understanding in a timely manner, enhancing the user experience.

[0149] In practical applications, commonly used speech emotion datasets such as IEMOCAP and CAISA use precise single-labels, for example, [0, 1, 2, 3] representing happiness, anger, sadness, and calmness, respectively. Currently, there are no corresponding fuzzy labels. However, in the real world, there is always some fuzzy data between emotions; in some scenarios, emotions are complex, such as a mixture of happiness and sadness. The technical solution provided in this application's embodiments can obtain fuzzy labels, such as target label data (2 / 3, 1 / 3, 0, 0).

[0150] Inspired by optimal interaction theory, individuals with similar cognitive abilities perform better during interactions, and multiple individuals can improve individual performance by sharing information. This application proposes a method based on multi-classifier interactive learning to improve the accuracy of emotion recognition. N different classifiers are used to mimic N individuals with inconsistent perceptions of fuzzy emotions, and new soft labels (the probability distribution of emotions) are constructed, such as target label data (2 / 3, 1 / 3, 0, 0). Then, the classifier is retrained using these new soft labels (target label data) to engage in cognitive interactions with other classifiers. This process enables each classifier to learn better representations of fuzzy data from other classifiers, further improving its emotion recognition capabilities.

[0151] Please see Figure 8 This application also provides an emotion recognition device based on a multi-classifier, which can implement the above-mentioned emotion recognition method based on a multi-classifier. The device includes:

[0152] The sample speech data acquisition module is used to acquire sample speech data; wherein, the sample speech data includes a first speech dataset without labels and a second speech dataset with original labels;

[0153] The first classifier training module is used to: predict the first label by the first classifier based on the first speech dataset, predict the second label by the second classifier based on the first speech dataset, and predict the third label by the third classifier based on the first speech dataset.

[0154] The target label construction module is used to construct target label data based on the first predicted label, the second predicted label, the third predicted label, and the original label.

[0155] The training dataset building module is used to build a training speech dataset based on the target label data and the first speech dataset.

[0156] The second classifier training module is used to: train the first classifier to obtain a first preliminary classifier based on the training speech dataset, train the second classifier to obtain a second preliminary classifier based on the training speech dataset, and train the third classifier to obtain a third preliminary classifier based on the training speech dataset.

[0157] The classifier filtering module is used to filter the first preliminary classifier, the second preliminary classifier, and the third preliminary classifier based on the second speech dataset to obtain the target classifier.

[0158] The emotion recognition module is used to input the acquired target speech data into the target classifier for emotion recognition and to obtain the target emotion category.

[0159] The specific implementation of this emotion recognition device is basically the same as the specific implementation of the emotion recognition method based on multi-classifier described above, and will not be repeated here.

[0160] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned emotion recognition method based on a multi-classifier. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0161] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0162] The processor 801 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0163] The memory 802 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and is called and executed by the processor 801 using the multi-classifier-based emotion recognition method of the embodiments of this application.

[0164] The 803 input / output interface is used to implement information input and output.

[0165] The communication interface 804 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0166] Bus 805 transmits information between various components of the device (e.g., processor 801, memory 802, input / output interface 803, and communication interface 804);

[0167] The processor 801, memory 802, input / output interface 803, and communication interface 804 are connected to each other within the device via bus 805.

[0168] This application embodiment also provides a storage medium, which is a computer-readable storage medium, storing a computer program that, when executed by a processor, implements the above-described emotion recognition method based on a multi-classifier.

[0169] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0170] The emotion recognition method, apparatus, electronic device, and medium based on multi-classifiers provided in this application acquire a first unlabeled speech dataset and a second speech dataset with original labels. Based on the first speech dataset, a first classifier predicts a first predicted label; based on the first speech dataset, a second classifier predicts a second predicted label; and based on the first speech dataset, a third classifier predicts a third predicted label. Target label data is then constructed based on the first, second, and third predicted labels and the original labels. A training speech dataset is constructed based on the target label data and the first speech dataset. The first classifier is trained using the training speech dataset to obtain a first preliminary classifier; the second classifier is trained using the training speech dataset to obtain a second preliminary classifier; and the third classifier is trained using the training speech dataset to obtain a third preliminary classifier. Finally, the first, second, and third preliminary classifiers are filtered using the second speech dataset to obtain the target classifier with the strongest classification ability. The acquired target speech data is then input into the target classifier for emotion recognition to obtain the target emotion category, thereby improving the accuracy of emotion recognition. This application embodiment improves the accuracy of emotion recognition based on multi-classifier interactive learning.

[0171] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0172] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0173] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0174] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0175] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0176] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0177] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0178] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0179] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0180] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0181] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A sentiment recognition method based on a multi-classifier, characterized in that, The method includes: Acquire sample speech data; wherein, the sample speech data includes a first speech dataset without labels and a second speech dataset with original labels; A first predicted label is obtained by predicting the first classifier based on the first speech dataset; a second predicted label is obtained by predicting the second classifier based on the first speech dataset; and a third predicted label is obtained by predicting the third classifier based on the first speech dataset. Target label data is constructed based on the first predicted label, the second predicted label, the third predicted label, and the original label; A training speech dataset is constructed based on the target label data and the first speech dataset; A first preliminary classifier is obtained by training the first classifier based on the training speech dataset; a second preliminary classifier is obtained by training the second classifier based on the training speech dataset; and a third preliminary classifier is obtained by training the third classifier based on the training speech dataset. Based on the second speech dataset, the first preliminary classifier, the second preliminary classifier, and the third preliminary classifier are filtered to obtain the target classifier; The acquired target speech data is input into the target classifier for emotion recognition to obtain the target emotion category.

2. The method according to claim 1, characterized in that, The step of constructing target label data based on the first predicted label, the second predicted label, the third predicted label, and the original label includes: Obtain the dimensions of the original labels to get the label dimensions; The tag probability is calculated based on the first predicted tag, the second predicted tag, and the third predicted tag to obtain the interaction tag; The target label data is constructed based on the interactive labels and the label dimensions.

3. The method according to claim 2, characterized in that, The step of constructing the target tag data based on the interaction tag and the tag dimension includes: The number of tag categories is obtained based on the interactive tags, thus obtaining the tag category count; Compare the number of tag categories and the tag dimensions; If the number of tag categories and the number of tag dimensions are equal, then the interactive tag is used as the target tag data; If the number of tag categories is less than the number of tag dimensions, then the target tag data is constructed based on the interactive tags and a preset number of original tags; wherein the preset number is equal to the difference between the tag dimensions and the number of tag categories.

4. The method according to claim 1, characterized in that, The step of filtering the first preliminary classifier, the second preliminary classifier, and the third preliminary classifier based on the second speech dataset to obtain the target classifier includes: The first preliminary classifier is validated based on the second speech dataset to obtain a first validation label; The second preliminary classifier is validated based on the second speech dataset to obtain a second validation label; The third preliminary classifier is validated based on the second speech dataset to obtain a third validation label; Target verification tags are selected from the first verification tag, the second verification tag, and the third verification tag; The target classifier is selected from the first preliminary classifier, the second preliminary classifier, and the third preliminary classifier based on the target verification label.

5. The method according to claim 4, characterized in that, The step of filtering the target verification label from the first verification label, the second verification label, and the third verification label includes: First accurate data is obtained by obtaining the accuracy of the first verification tag, second accurate data is obtained by obtaining the accuracy of the second verification tag, and third accurate data is obtained by obtaining the accuracy of the third verification tag. Select one data point from the first accurate data, the second accurate data, and the third accurate data as the target accurate data; Based on the target accurate data, a verification label is selected from the first verification label, the second verification label, and the third verification label as the target verification label.

6. The method according to any one of claims 1 to 5, characterized in that, The sample speech data also includes a third speech dataset with the original labels, and the method further includes: training a preset number of original classifiers based on the third speech dataset, specifically including: Obtain the number of samples in the third speech dataset to obtain preliminary sample parameters; The third speech dataset is input into the original classifier for prediction to obtain the original output. Construct cross-entropy loss data based on the original labels, the preliminary sample parameters, and the original output; The preset number of original classifiers are trained based on the cross-entropy loss data to obtain the first classifier, the second classifier, and the third classifier.

7. The method according to any one of claims 1 to 5, characterized in that, The process of training the first classifier to obtain a first preliminary classifier based on the training speech dataset, training the second classifier to obtain a second preliminary classifier based on the training speech dataset, and training the third classifier to obtain a third preliminary classifier based on the training speech dataset includes: Obtain the number of samples in the training speech dataset to obtain the training sample parameters; The training speech dataset is input into the first classifier to obtain the first predicted data, the training speech dataset is input into the second classifier to obtain the second predicted data, and the training speech dataset is input into the third classifier to obtain the third predicted data. Relative entropy loss data is constructed based on the target label data, the training sample parameters, the first prediction data, the second prediction data, and the third prediction data; The first preliminary classifier is obtained by training the first classifier based on the relative entropy loss data, the second preliminary classifier is obtained by training the second classifier based on the relative entropy loss data, and the third preliminary classifier is obtained by training the third classifier based on the relative entropy loss data.

8. An emotion recognition device based on a multi-classifier, characterized in that, The device includes: A sample speech data acquisition module is used to acquire sample speech data; wherein, the sample speech data includes a first speech dataset without labels and a second speech dataset with original labels; The first classifier training module is used to: predict a first predicted label by a first classifier based on the first speech dataset, predict a second predicted label by a second classifier based on the first speech dataset, and predict a third predicted label by a third classifier based on the first speech dataset. The target label construction module is used to construct target label data based on the first predicted label, the second predicted label, the third predicted label, and the original label; A training dataset construction module is used to construct a training speech dataset based on the target label data and the first speech dataset; The second classifier training module is configured to: train the first classifier to obtain a first preliminary classifier based on the training speech dataset; train the second classifier to obtain a second preliminary classifier based on the training speech dataset; and train the third classifier to obtain a third preliminary classifier based on the training speech dataset. The classifier filtering module is used to filter the first preliminary classifier, the second preliminary classifier, and the third preliminary classifier based on the second speech dataset to obtain the target classifier. The emotion recognition module is used to input the acquired target speech data into the target classifier for emotion recognition and to obtain the target emotion category.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images