A voice processing method, device, equipment and storage medium

Abstract

A speech processing method, device, equipment and storage medium, the method comprising: obtaining speech data to be processed of a target application scenario, determining an emotion recognition result set of the speech data to be processed based on a speech emotion recognition model, the emotion recognition result set including the emotion category to which each speech segment of the speech data to be processed belongs; if the emotion recognition result set does not meet the scenario rule of the target application scenario (including the predefined proportion rule of each reference emotion category), clustering multiple speech segments to obtain multiple clusters, and speech segments in the same cluster correspond to one emotion category; determining a first cluster whose emotion category does not match the target application scenario from the multiple clusters, obtaining an emotion annotation label corresponding to the first cluster; determining an emotion evaluation result of the speech data to be processed based on the emotion annotation label corresponding to the first cluster and the emotion recognition result set. The present application can improve the accuracy of speech emotion recognition.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a speech processing method, apparatus, device and storage medium. Background Art

[0002] At present, most voice emotion analysis uses a general model. For example, some speakers are asked to pronounce with multiple emotions, and the model is trained with the collected pronunciation data to achieve the purpose of recognizing other people's voice emotions. In some cases where the recognition accuracy is not very high, some researchers will combine other information of the speaker (such as facial expressions, voice text, etc.) to get the results together, but this solution requires a relatively high cost, and everyone has their own set of voice emotion expression systems. Sometimes it is just a simple pronunciation that is louder, but the general model is likely to classify it as angry or excited. Therefore, how to improve the accuracy of voice emotion recognition is an urgent problem to be solved. Summary of the invention

[0003] The embodiments of the present application provide a speech processing method, apparatus, device and storage medium, which can improve the accuracy of speech emotion recognition.

[0004] On the one hand, an embodiment of the present application provides a speech processing method, the method comprising:

[0005] Acquire speech data to be processed of a target application scenario, and determine an emotion recognition result set of the speech data to be processed based on a speech emotion recognition model, wherein the emotion recognition result set includes an emotion category to which each speech segment in a plurality of speech segments of the speech data to be processed belongs;

[0006] If the emotion recognition result set does not meet the scenario rule of the target application scenario, clustering the multiple voice clips to obtain multiple clustering clusters, wherein the scenario rule includes a predefined proportion rule of each reference emotion category, and the voice clips in the same clustering cluster correspond to one emotion category;

[0007] Determine a first cluster whose emotion category does not match the target application scenario from the multiple clusters, and obtain an emotion annotation label corresponding to the first cluster;

[0008] Based on the emotion annotation label corresponding to the first cluster and the emotion recognition result set, an emotion evaluation result of the speech data to be processed is determined.

[0009] On the one hand, an embodiment of the present application provides a speech processing device, the device comprising:

[0010] An acquisition unit, used to acquire speech data to be processed of a target application scenario, and determine an emotion recognition result set of the speech data to be processed based on a speech emotion recognition model, wherein the emotion recognition result set includes an emotion category to which each speech segment in a plurality of speech segments of the speech data to be processed belongs;

[0011] A processing unit, configured to cluster the multiple voice segments to obtain multiple clusters if the emotion recognition result set does not meet the scenario rule of the target application scenario, wherein the scenario rule includes a predefined proportion rule of each reference emotion category, and the voice segments in the same cluster cluster correspond to one emotion category;

[0012] The processing unit is further configured to determine, from the plurality of clusters, a first cluster whose emotion category does not match the target application scenario, and obtain an emotion annotation label corresponding to the first cluster;

[0013] The processing unit is further configured to determine an emotion evaluation result of the speech data to be processed based on the emotion annotation label corresponding to the first cluster and the emotion recognition result set.

[0014] On the one hand, an embodiment of the present application provides a computer device, which includes a processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are interconnected, wherein the memory stores a computer program, and the processor is used to call the computer program to execute the speech processing method of any possible implementation method described above.

[0015] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the speech processing method of any possible implementation manner is implemented.

[0016] On the one hand, an embodiment of the present application further provides a computer program product, which includes a computer program or computer instructions, and the computer program or computer instructions are executed by a processor to implement the steps of the speech processing method provided in the embodiment of the present application.

[0017] On the one hand, an embodiment of the present application also provides a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the speech processing method provided in the embodiment of the present application.

[0018] In an embodiment of the present application, speech data to be processed of a target application scenario can be obtained, and an emotion recognition result set of the speech data to be processed can be determined based on a speech emotion recognition model, the emotion recognition result set including the emotion category to which each speech segment belongs in a plurality of speech segments of the speech data to be processed, if the emotion recognition result set does not meet the scenario rule of the target application scenario (including the predefined proportion rule of each reference emotion category), the plurality of speech segments are clustered to obtain a plurality of cluster clusters, the speech segments in the same cluster cluster correspond to an emotion category, the first cluster cluster whose emotion category does not match the target application scenario can be determined from the plurality of cluster clusters, and the emotion annotation label corresponding to the first cluster cluster is obtained, based on the emotion annotation label corresponding to the first cluster cluster and the emotion recognition result set, the emotion evaluation result of the speech data to be processed is determined. By adopting the method of the present application, it is possible to determine whether the recognition result of the speech emotion recognition model is accurate based on whether the emotion category to which each speech segment of the speech data to be processed belongs meets the objective scenario rule, and introduce some manual intervention when the recognition result of the speech emotion recognition model is inaccurate, thereby greatly improving the accuracy of speech emotion recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical methods of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 A schematic diagram of the system architecture of a speech processing system provided in an embodiment of the present application;

[0021] Figure 2 A flowchart of a speech processing method provided in an embodiment of the present application;

[0022] Figure 3 A flowchart of another speech processing method provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of the clustering effect provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of a cluster center provided in an embodiment of the present application;

[0025] Figure 6 A flowchart of another speech processing method provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of the structure of a speech processing device provided in an embodiment of the present application;

[0027] Figure 8 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical methods in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] See also Figure 1 , Figure 1 A schematic diagram of the system structure of a speech processing system provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the system includes a speech processing device 10 and a database 11. The speech processing device 10 and the database 11 can be connected to each other in a wired or wireless manner.

[0030] The speech processing device 10 may include one or more of a terminal and a server. That is, the speech processing method proposed in the embodiment of the present application may be executed by a terminal, or by a server, or jointly by a terminal and a server that can communicate with each other. Among them, the server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, an intelligent voice interaction device, a smart home appliance, a car terminal, etc., but is not limited thereto.

[0031] The database 11 may be a local database of the speech processing device 10 or a cloud database accessible to the speech processing device 10, or a local database of other computer devices. The database 11 may be used to store speech data, specifically, to-be-processed speech data.

[0032] The speech processing device 10 may be equipped with a speech emotion recognition model, and the interaction process between the speech processing device 10 and the database 11 is as follows:

[0033] The speech data to be processed of the target application scenario is obtained from the database 11, and the emotion recognition result set of the speech data to be processed is determined based on the speech emotion recognition model installed, and the emotion recognition result set includes the emotion category to which each speech segment belongs in the multiple speech segments of the speech data to be processed, and is specifically obtained by using the speech emotion recognition model to perform emotion recognition processing on each speech segment. If the emotion recognition result set does not meet the scene rule of the target application scenario, the multiple speech segments are clustered to obtain multiple cluster clusters, wherein the scene rule includes a predefined proportion rule of each reference emotion category, and the speech segments in the same cluster cluster correspond to one emotion category. A first cluster cluster whose emotion category does not match the target application scenario can be determined from the multiple cluster clusters. At this time, it can be considered that the speech emotion recognition model fails to accurately identify the emotion category to which each speech segment in the first cluster cluster belongs, and then the emotion annotation label corresponding to the first cluster cluster is determined by manual annotation, and the emotion annotation label can be used to indicate the true emotion category of each speech segment in the first cluster cluster. Based on the emotion annotation label corresponding to the first cluster cluster and the emotion recognition result set, the emotion evaluation result of the speech data to be processed is determined.

[0034] In one embodiment, the requesting terminal may send the speech data to be processed to the speech processing device 10. When the speech processing device 10 determines that the emotion recognition result set of the speech data to be processed meets the scenario rules of the target application scenario, it may use the emotion recognition result set of the speech data to be processed to determine the emotion evaluation result of the speech data to be processed. When it is determined that the emotion recognition result set of the speech data to be processed does not meet the scenario rules of the target application scenario, it may use the emotion annotation label and emotion recognition result set corresponding to the first cluster to determine the emotion evaluation result of the speech data to be processed. After obtaining the emotion evaluation result of the speech data to be processed, the speech processing device 10 may return the emotion evaluation result to the requesting terminal, which is conducive to improving the accuracy of speech emotion evaluation.

[0035] Existing methods cannot solve the needs of speech emotion recognition for different listeners or different dialogue scenarios. For example, for the same piece of speech data (interview recording), different listeners (human resources) may have different judgments on the emotions of the candidate; or in the analysis of sales recordings, the pronunciation is generally louder and the speech is faster (in order to attract the user's attention as much as possible), and the use of a general model is more likely to classify it as angry or excited. The solution of this application can meet the needs of speech emotion recognition in different dialogue scenarios by judging whether the emotion category of each speech segment in the speech data to be processed conforms to the objective scene rules. In addition, on the basis of the recognition of the speech emotion recognition model, a little manual intervention can be introduced, that is, the first cluster is manually labeled with emotions to meet the speech emotion recognition needs of different listeners and greatly improve the accuracy of speech emotion recognition.

[0036] See also Figure 2 , Figure 2 A flow chart of a speech processing method provided in an embodiment of the present application. The method can be applied to the speech processing device 10 in the above-mentioned speech processing system, and the method includes:

[0037] S201, obtaining speech data to be processed of a target application scenario, and determining an emotion recognition result set of the speech data to be processed based on a speech emotion recognition model, wherein the emotion recognition result set includes an emotion category to which each speech segment in a plurality of speech segments of the speech data to be processed belongs.

[0038] The speech processing solution proposed in this application is applicable to scenarios that require a longer recording analysis time (not a single sentence analysis), such as interview voice analysis, recording analysis of follow-up or unannounced visits, and recording analysis during telephone sales / customer service.

[0039] The target application scenario refers to the conversation scenario in which the interlocutor in the voice data to be processed is located. The conversation scenario may include an interview scenario, a sales scenario, a customer service scenario, a meeting scenario, etc., and this application does not limit this. The target application scenario can be determined by the conversation content in the voice data to be processed. For example, an interview scenario generally includes a self-introduction, and a sales scenario generally focuses on the product being sold.

[0040] The speech data to be processed is speech data that needs to be subjected to speech emotion recognition. The speech data to be processed can be segmented and processed according to a preset frame length, and multiple speech segments of the speech data to be processed can be obtained by segmenting the speech data to be processed into speech segments one by one. In one embodiment, the speech data to be processed can be preprocessed, and the preprocessing can specifically be pre-emphasis, which can emphasize the high-frequency part of the speech signal in the speech data to be processed and increase the high-frequency resolution, and then the speech data to be processed after pre-emphasis is segmented and processed to obtain multiple speech segments of the speech data to be processed.

[0041] The speech emotion recognition model is a general model for performing speech emotion recognition tasks. In one implementation, a limited category boundary can be determined, that is, it is set that all people's emotions can basically be distributed in N (positive integer) emotion categories. For example, the conventionally collected emotion categories are: {angry, afraid / nervous, happy, neutral, sad / wronged, surprised / excited}, of course, there can be more and more detailed, but it is a limited category. The training data can be collected from a limited number of M people (for example, 6 speakers, including 3 men and 3 women), and each person's training data contains speech samples of these N emotion categories. The classification model can be trained using the collected speech samples and the emotion categories corresponding to the speech samples. The classification model can be MLP (Multi-Layer Perceptron), SVM (Support Vector Machine), LSTM (Long Short-Term Memory), CNN (Convolutional Neural Networks), etc. After the classification model training is completed, the classification model obtained by training can be used as a speech emotion recognition model. In addition, the speech emotion recognition model can also be a general model that combines other information of the speaker to perform speech emotion recognition. This application does not limit the speech emotion recognition model.

[0042] The speech emotion recognition model can be used to determine the emotion category to which each speech segment belongs in multiple speech segments of the speech data to be processed, and the emotion recognition result set of the speech data to be processed can be formed based on the emotion category to which each speech segment belongs. For example, after preprocessing a speech of a long duration (hereinafter referred to as long speech A), the long speech A can be split into multiple sentences (one sentence is a speech segment), which include: sentence 1, sentence 2, sentence 3, sentence 4, sentence 5, sentence 6, sentence 7, sentence 8, ..., sentence m, and then the speech emotion recognition model is used to perform emotion recognition on each sentence, and the emotion category to which each sentence belongs can be obtained. Finally, the emotion recognition result set A of the speaker corresponding to the long speech A can be: {1: "neutral"; 2: "neutral"; 3: "neutral"; 4: "neutral"; 5: "excited"; 6: "neutral"; 7: "nervous"; 8: "neutral"; ...; m: "nervous"}, where 1, 2, ..., m are used to identify sentence 1, sentence 2, ..., sentence m, and it can be seen that the emotion recognition result set A includes the emotion category to which each sentence belongs.

[0043] S202. If the emotion recognition result set does not meet the scenario rules of the target application scenario, clustering is performed on the multiple voice clips to obtain multiple clustering clusters. The scenario rules include predefined proportion rules of each reference emotion category. The voice clips in the same clustering cluster correspond to one emotion category.

[0044] The scenario rules of the target application scenario may include predefined proportion rules for each reference emotion category, and each reference emotion category may specifically include one or more of neutral emotion, positive emotion, and negative emotion. For example, the interview scenario should be mostly neutral or positive emotions, and the scenario rule for the interview scenario may be that the proportion of negative emotions is less than 20%; the sales scenario should be mostly positive emotions, and the scenario rule for the sales scenario may be that the proportion of positive emotions is greater than 70%; the customers in the customer service scenario may have a certain proportion of negative emotions, and the customer service should all be neutral or positive emotions, and the scenario rule for the customer service scenario may be that the proportion of negative emotions of customers is greater than 20%, and the proportion of negative emotions of customer service is 0%; the meeting scene should be mostly neutral or positive emotions, and the scenario rule for the meeting scene may be that the sum of the proportion of neutral emotions and positive emotions is greater than 90%.

[0045] Among them, neutral emotion refers to an emotion between positive and negative, such as calmness and surprise. Positive emotion refers to a positive emotion, such as happiness, excitement, and joy. Negative emotion refers to a negative emotion, such as frustration, fear, and anger. The emotion category that can be recognized by the speech emotion recognition model can be classified into any reference emotion category, that is, it can be classified into one of neutral emotion, positive emotion, and negative emotion.

[0046] In a feasible manner, the various reference emotion categories may include one or more emotion categories that can be recognized by the speech emotion recognition model. For example, the scenario rule of the target application scenario may be that the proportion of "excitement" is greater than 10%, and the proportion of "surprise" is greater than 20%.

[0047] In one embodiment, the proportion of speech segments corresponding to different emotion categories in the emotion recognition result set can be obtained, including: according to the emotion category to which each speech segment in the emotion recognition result set belongs, the speech segments corresponding to the same emotion category are determined. Based on the number of speech segments corresponding to the same emotion category and the total number of speech segments in the emotion recognition result set, the proportion of speech segments corresponding to the same emotion category is calculated, that is, the proportion of speech segments corresponding to the same emotion category = (the number of speech segments corresponding to the same emotion category / the total number of speech segments in the emotion recognition result set) × 100%. In this way, the proportion of speech segments corresponding to different emotion categories in the emotion recognition result set can be obtained. For example, the emotion recognition result set A: {1: "neutral"; 2: "neutral"; 3: "neutral"; 4: "neutral"; 5: "excited"; 6: "neutral"; 7: "nervous"; 8: "neutral"; 9: "nervous"}, it can be seen that the proportion of speech segments corresponding to "neutral" is 67% = 6 / 9 × 100%, the proportion of speech segments corresponding to "excited" is 11% = 1 / 9 × 100%, and the proportion of speech segments corresponding to "nervous" is 22% = 2 / 9 × 100%.

[0048] Further determine whether the proportion of speech segments corresponding to different emotion categories in the emotion recognition result set meets the proportion rule of the corresponding reference emotion category. For example, if the target application scenario is an interview scenario, the proportion of negative emotions in the interview scenario must be less than 20%. If the proportion of sentences identified as "anger" (a negative emotion) in the emotion recognition result set A is greater than 20%, then the proportion of speech segments corresponding to "anger" does not meet the corresponding proportion rule of negative emotions. When the proportion of speech segments corresponding to an emotion category in the emotion recognition result set does not meet the proportion rule of the corresponding reference emotion category, it can be determined that the emotion recognition result set does not meet the scenario rule of the speech data to be processed.

[0049] When it is determined that the emotion recognition result set does not meet the scene rules of the speech data to be processed, it can be considered that the speech emotion recognition model fails to accurately perform emotion recognition on multiple speech segments of the speech data to be processed. At this time, the multiple speech segments of the speech data to be processed can be clustered. The clustering process refers to dividing the speech segments with high similarity between the corresponding speech features into a cluster cluster, and dividing the speech segments with large differences in the corresponding speech features into different cluster clusters, thereby obtaining multiple cluster clusters. Since the speech segments belonging to the same emotion category reflect the same emotion characteristics, the similarity between the corresponding speech features of the speech segments belonging to the same emotion category will be much higher than the speech features corresponding to the speech segments belonging to other emotion categories. For example, in the common speech emotion classification, it is because the similarity between the speech features of the speech belonging to the same category is much higher than the speech features of the speech belonging to other categories, so the speech of the same category can be classified into the same category. Based on this, it can be known that the purpose of the above clustering process is to divide the speech segments belonging to the same emotion category into a cluster cluster, so that the speech segments in the same cluster cluster correspond to an emotion category, so that multiple speech segments with inaccurate classification can be found at the same time.

[0050] Speech features can reflect the time domain characteristics and / or frequency domain characteristics of speech data, and can include frequency domain features and / or time domain features, for example, they can include time domain features such as timbre, tone, pitch, formant, spectrum, brightness, roughness, and frequency domain features such as spectral centroid, spectral plane, Mel-frequency cepstral coefficients (MFCC) features of different coefficients, Mel-frequency spectrogram, chromatogram, root mean square energy, etc. In addition, speech features can be features of different dimensions, that is, they can be one-dimensional features or two-dimensional features, and the present application does not limit the specific form of speech features.

[0051] S203: Determine, from the multiple clusters, a first cluster whose emotion category does not match the target application scenario, and obtain an emotion annotation label corresponding to the first cluster.

[0052] In one embodiment, based on the characteristic that speech segments in the same cluster correspond to an emotion category, the emotion category corresponding to any cluster among multiple clusters can be determined, that is, the emotion category corresponding to the speech segments in any cluster is used as the emotion category corresponding to any cluster.

[0053] If the emotion category corresponding to any cluster is not a positive emotion category that matches the target application scenario, it can be determined that the emotion category corresponding to the speech segment of any cluster does not conform to the objective law of the target application scenario, that is, it is an emotion category that is unlikely to appear in the target application scenario. It can be determined that any cluster is the first cluster whose emotion category does not match the target application scenario. At this time, manual intervention can be introduced to determine the emotion annotation label corresponding to the first cluster, and the emotion annotation label is used as the true emotion category of each speech segment in the first cluster. By adopting this embodiment, only a little manual intervention can be introduced to correct the erroneous recognition results of the speech emotion recognition model. At the same time, the true emotion category of each speech segment in the first cluster is determined by manual annotation, which is conducive to improving the accuracy of the emotion evaluation results of the speech data to be processed.

[0054] Among them, the positive emotion categories that match the target application scenario refer to the emotion categories that are likely to appear in the target application scenario. For example, the interview scenario should be mostly neutral or positive emotions, so the positive emotion categories that match the interview scenario are neutral emotions and positive emotions; the sales scenario should be mostly positive emotions, so the positive emotion category that matches the sales scenario is positive emotions; customers in the customer service scenario may have a certain proportion of negative emotions, and customer service should be all neutral or positive emotions, so the positive emotion categories that match the customer service scenario are negative emotions, neutral emotions, and positive emotions; the meeting scene should be mostly neutral or positive emotions, so the positive emotion categories that match the meeting scene are neutral emotions and positive emotions.

[0055] When the emotion category corresponding to any clustering cluster can be classified into the positive emotion category that matches the target application scenario, it can be considered that the emotion category corresponding to any clustering cluster is a positive emotion category that matches the target application scenario. Conversely, when the emotion category corresponding to any clustering cluster cannot be classified into the positive emotion category that matches the target application scenario, it can be considered that the emotion category corresponding to any clustering cluster is not a positive emotion category that matches the target application scenario.

[0056] S204: Determine an emotion evaluation result of the speech data to be processed based on the emotion annotation label and the emotion recognition result set corresponding to the first cluster.

[0057] In one embodiment, the emotion category to which each voice segment in the first cluster in the emotion recognition result set belongs can be modified to the emotion label tag corresponding to the first cluster, and the modified emotion recognition result set is obtained. Then, based on a series of scoring rules and the modified emotion recognition result set, the emotion evaluation result of the speech data to be processed is determined. For example, a score can be set for each emotion category, and each speech segment is scored according to the emotion category and the corresponding score to which each speech segment in the modified emotion recognition result set belongs, and the sum of the scores of all speech segments is used as the emotion score of the speech data to be processed, and the emotion score is the emotion evaluation result of the speech data to be processed. Among them, the score of positive emotions (such as excitement, happiness, excitement) can be relatively high, and the score of negative emotions (such as sadness, grievance, fear) can be relatively low. The larger the emotion score, the more the speech data to be processed tends to express positive emotions, and the lower the emotion score, the more the speech data to be processed tends to express negative emotions. This is just an example, and there are other scoring methods, which are not limited by this application. By using this scoring method to determine the emotional evaluation results of the voice data to be processed, we can know the emotional category that the voice data to be processed tends to reflect, which is helpful for analyzing the conversation atmosphere in the target application scenario. For example, when it tends to be positive emotion, the conversation atmosphere may be relaxed and pleasant, and when it tends to be negative emotion, the conversation atmosphere may be tense and serious.

[0058] By adopting the method of the present application, it is possible to determine whether the speech emotion recognition model can accurately perform speech emotion recognition by judging whether the proportion of speech segments corresponding to different emotion categories in the speech data to be processed meets the proportion rule of the corresponding scene. When the recognition result of the speech emotion recognition model is inaccurate, a little manual intervention is introduced to determine the true emotion category of each speech segment in the first cluster through manual labeling, thereby greatly improving the accuracy of speech emotion recognition.

[0059] See also Figure 3 , Figure 3 A flowchart of another speech processing method provided in an embodiment of the present application. The method can be applied to the speech processing device 10 in the above-mentioned speech processing system, and the method includes:

[0060] S301, obtaining speech data to be processed of a target application scenario, and determining an emotion recognition result set of the speech data to be processed based on a speech emotion recognition model, wherein the emotion recognition result set includes an emotion category to which each speech segment in a plurality of speech segments of the speech data to be processed belongs.

[0061] In one embodiment, the voice data to be processed can be pre-emphasized, and the pre-emphasis can emphasize the high-frequency part of the voice signal in the voice data to be processed and increase the high-frequency resolution. Then, the pre-emphasized voice data to be processed can be subjected to endpoint detection. Endpoint detection refers to detecting the start time and end time of the voice signal in a segment of voice data, which can be specifically achieved through an endpoint detection algorithm. The pre-emphasized voice data to be processed can be segmented (windowed and framed) based on the obtained endpoint detection results, that is, the start time and end time of the voice signal are used as segmentation points, and the voice data between the start time and the end time is used as a voice segment, thereby obtaining multiple voice segments of the voice data to be processed, and the multiple voice segments are valid voice data in the voice data to be processed, eliminating the noise of the silent segment of the voice data to be processed.

[0062] Further, the speech features of each speech segment in the multiple speech segments are determined, and the speech features may include one or more of frequency domain features and time domain features, which are not limited in the present application. By inputting the speech features of each speech segment into the speech emotion recognition model, the emotion category to which each speech segment belongs can be obtained. Finally, based on the emotion category to which each speech segment belongs, the emotion recognition result set of the speech data to be processed is determined. For example, after preprocessing a speech of long duration (hereinafter referred to as long speech A), the long speech A can be split into multiple sentences, including: sentence 1, sentence 2, sentence 3, sentence 4, sentence 5, sentence 6, sentence 7, sentence 8, ..., sentence m, and then each sentence is classified using the speech emotion recognition model to obtain the emotion category to which each sentence belongs, thereby obtaining the emotion recognition result set A of the speaker corresponding to the long speech A: {1: "neutral"; 2: "neutral"; 3: "neutral"; 4: "neutral"; 5: "excited"; 6: "neutral"; 7: "nervous"; 8: "neutral"; ...; m: "nervous"}. It can be seen that the emotion recognition result set A includes the emotion category to which each sentence belongs.

[0063] S302. If the emotion recognition result set does not meet the scenario rules of the target application scenario, multiple voice clips are clustered to obtain multiple clusters. The scenario rules include predefined proportion rules of each reference emotion category. Voice clips in the same cluster correspond to one emotion category.

[0064] In one embodiment, the proportion of speech segments corresponding to different emotion categories in the emotion recognition result set is obtained. If the proportion of speech segments corresponding to all emotion categories in the emotion recognition result set meets the proportion rule of the corresponding reference emotion category, it is determined that the emotion recognition result set meets the scene rule of the target application scenario. When it is determined that the emotion recognition result set meets the scene rule of the speech data to be processed, it can be considered that the speech emotion recognition model is accurately performing emotion recognition on multiple speech segments of the speech data to be processed. At this time, the emotion score of the speech data to be processed is directly determined by a series of scoring rules and the emotion recognition result set. For example, a score can be set for each emotion category, and each speech segment is scored according to the emotion category and corresponding score to which each speech segment belongs in the emotion recognition result set. The sum of the scores of all speech segments is used as the emotion score of the speech data to be processed, and the emotion score is the emotion evaluation result of the speech data to be processed. Among them, the score of positive emotions (such as excitement, happiness, excitement) can be relatively high, and the score of negative emotions (such as sadness, grievance, fear) can be relatively low. The larger the emotion score, the more the speech data to be processed tends to express positive emotions, and the lower the emotion score, the more the speech data to be processed tends to express negative emotions. This is just an example, there are other scoring methods, and this application does not limit it. By using this scoring method to determine the emotional evaluation result of the voice data to be processed, it is possible to know the emotional category that the voice data to be processed tends to reflect, which is conducive to analyzing the conversation atmosphere in the target application scenario.

[0065] In another embodiment, the proportion of speech segments corresponding to different emotion categories in the emotion recognition result set is obtained. If the proportion of speech segments corresponding to an emotion category in the emotion recognition result set does not meet the proportion rule of the corresponding reference emotion category, it is determined that the emotion recognition result set does not meet the scene rule of the target application scene. For example, the target application scene is a meeting scene, and the sum of the proportions of neutral emotions and positive emotions in the meeting scene must be greater than 90%. If in the emotion recognition result set A, the proportion of sentences identified as "neutral" (a neutral emotion) is 67%, the proportion of sentences identified as "nervous" (a negative emotion) is 22%, and the proportion of sentences identified as "excited" (a neutral emotion) is 11%, then the sum of the proportions of neutral emotions and positive emotions is 78%, which shows that the corresponding proportion rule is not met.

[0066] When it is determined that the emotion recognition result set does not meet the scenario rules of the speech data to be processed, the speech features of each of the multiple speech segments can be determined, and the multiple speech segments can be clustered using the speech features of each of the multiple speech segments and the target expected number to obtain at least one cluster cluster, and the number of the at least one cluster cluster is the target expected number, that is, the target expected number refers to the number of cluster clusters expected to be obtained through clustering processing.

[0067] Specifically, based on the clustering algorithm and the target expected number, the speech features of each speech segment are processed to obtain multiple clusters, and the speech segments whose speech features belong to the same cluster are divided into a cluster cluster to obtain at least one cluster cluster. The target expected number can be a first number, and the first number is N*2, where N refers to the type of emotion that can be recognized by the speech emotion recognition model. Taking the clustering algorithm as the k-means algorithm as an example: ① Select N*2 initial cluster centers; ② Calculate the distance between the speech features of each speech segment and the N*2 initial cluster centers, and assign the speech features to the cluster with the closest distance; ③ For each cluster, calculate the average speech features in the current cluster and use it as the new cluster center; ④ Repeat ②-③ until the termination condition is reached, for example, the termination condition can be that the preset upper limit of the number of iterations is reached or the speech features in each cluster do not change. When the termination condition is reached, N*2 clusters are obtained. The speech segments whose speech features belong to the same cluster are divided into one cluster, and N*2 clusters can be obtained, for example, B: {1: [1,2,4,5,10]; 2: [3,8,9,13,15,16,26]; 3: [5,6,7,11,12,14]; ...; N*2: [x,x,x]}. Figure 4 As shown in the figure, the clustering data is the effect diagram obtained by clustering the original data. Figure 4 A, B, C, and D in the figure are the effect diagrams of various clustering.

[0068] Furthermore, the total amount of speech segments of each cluster in at least one cluster is obtained, and the cluster whose corresponding total amount of speech segments is less than the preset total amount (for example, 5) is determined as the second cluster. If the number of the second cluster is less than the preset number, it can be determined that there are fewer clusters with too few speech segments, for example, only one or two clusters out of 20 clusters have less than 5 speech segments. At this time, based on the speech segment ratios corresponding to different emotion categories in each cluster in at least one cluster, the maximum speech segment ratio corresponding to each cluster is determined.

[0069] Specifically, according to the emotion category to which each speech segment in each cluster of at least one cluster belongs, the speech segments corresponding to the same emotion category are determined, and based on the number of speech segments corresponding to the same emotion category and the total number of speech segments in each cluster, the proportion of speech segments corresponding to the same emotion category is calculated, that is, the proportion of speech segments corresponding to the same emotion category = (the number of speech segments corresponding to the same emotion category / the total number of speech segments in each cluster) × 100%. In this way, the proportion of speech segments corresponding to different emotion categories in each cluster can be obtained. The maximum proportion of speech segments is determined from the proportion of speech segments corresponding to the different emotion categories, and is used as the maximum proportion of speech segments corresponding to each cluster, that is, the maximum proportion of speech segments corresponding to a cluster is the proportion of speech segments corresponding to the emotion category with the largest proportion in a cluster.

[0070] If the maximum speech segment ratio corresponding to each cluster is greater than the preset ratio (for example, 80%), it can be said that the speech segments in each cluster generally reflect the same emotional characteristics, and the clustering results of each cluster are accurate. Among them, an example of judging whether the result of the cluster is accurate is: for example, for cluster 1, the emotion recognition results of sentence 1, sentence 2, sentence 3, sentence 4, and sentence 5 are obtained from the emotion recognition result set A, and they are neutral, neutral, neutral, excited, and neutral respectively; among them, 83% of the sentences are neutral and 17% of the sentences are excited. If the preset ratio b% is 80%, since 83% is greater than 80%, it means that the result of cluster 1 is accurate. If the maximum speech segment ratio corresponding to each cluster in at least one cluster is greater than the preset ratio, it can be considered that the clustering results of each cluster are accurate, and the clusters other than the second cluster in at least one cluster are determined as multiple clusters.

[0071] If the number of the second clustering clusters is greater than or equal to the preset number, it can be determined that there are too many clusters with too few speech segments, for example, 15 of the 20 clustering clusters have less than 5 speech segments. At this time, the target expected number can be updated to the second number (less than the first number), and the steps of clustering the multiple speech segments based on the speech features of each speech segment and the target expected number to obtain at least one clustering cluster are performed to obtain multiple clustering clusters. The method can avoid the situation where there are too few speech segments in the clustering cluster due to noise data.

[0072] If there is a cluster in at least one cluster whose corresponding maximum speech segment ratio is less than or equal to the preset ratio, it means that the clustering result of the cluster is inaccurate, and the target expected number is updated to a third number (greater than the first number), and the steps of clustering the multiple speech segments based on the speech features of each speech segment and the target expected number are performed to obtain at least one cluster, so as to obtain multiple clusters. The clustering result of the cluster can be accurate by increasing the number of clusters and reducing the number of speech segments in each cluster.

[0073] S303, based on the speech segment proportions corresponding to different emotion categories in any cluster among the multiple clusters, determine the emotion category corresponding to the largest speech segment proportion, and determine the emotion category corresponding to the largest speech segment proportion as the emotion category corresponding to any cluster.

[0074] Since the emotion category corresponding to the maximum speech segment ratio of each cluster in multiple clusters is the emotion characteristic generally reflected by each cluster, in one embodiment, the maximum speech segment ratio corresponding to any cluster in multiple clusters can be determined according to the emotion category to which each speech segment in the cluster belongs. The maximum speech segment ratio corresponding to a cluster is the speech segment ratio corresponding to the emotion category with the largest ratio in the cluster, and then the emotion category corresponding to the maximum speech segment ratio can be obtained. For example, the emotion recognition result set A: {1: "neutral"; 2: "neutral"; 3: "neutral"; 4: "neutral"; 5: "excited"; 6: "neutral"; 7: "nervous"; 8: "neutral"; 9: "nervous"}, it can be known that the speech segment ratio corresponding to "neutral" is 67%, the speech segment ratio corresponding to "excited" is 11%, the speech segment ratio corresponding to "nervous" is 22%, the maximum speech segment ratio is 67%, and the emotion category corresponding to the maximum speech segment ratio is "neutral". Finally, the emotion category corresponding to the largest proportion of speech segments is taken as the emotion category corresponding to any cluster, that is, the emotion category corresponding to the speech segments in any cluster.

[0075] S304: Obtain a positive sentiment category that matches the target application scenario.

[0076] The positive emotion categories that match the target application scenario refer to the emotion categories that are likely to appear in the target application scenario. For example, the interview scene should be mostly neutral or positive emotions, so the positive emotion categories that match the interview scene are neutral emotions and positive emotions; the sales scene should be mostly positive emotions, so the positive emotion category that matches the sales scene is positive emotions; customers in the customer service scenario may have a certain proportion of negative emotions, and customer service should all have neutral or positive emotions, so the positive emotion categories that match the customer service scene are negative emotions, neutral emotions, and positive emotions; the meeting scene should be mostly neutral or positive emotions, so the positive emotion categories that match the meeting scene are neutral emotions and positive emotions.

[0077] S305: If the emotion category corresponding to any cluster is not a positive emotion category matching the target application scenario, determine that any cluster is a first cluster whose emotion category does not match the target application scenario.

[0078] When the emotion category corresponding to any cluster can be classified into the positive emotion category that matches the target application scenario, it can be considered that the emotion category corresponding to any cluster is a positive emotion category that matches the target application scenario. On the contrary, when the emotion category corresponding to any cluster cannot be classified into the positive emotion category that matches the target application scenario, it is considered that the emotion category corresponding to any cluster is not a positive emotion category that matches the target application scenario. At this time, any cluster is the first cluster whose emotion category does not match the target application scenario.

[0079] S306: Obtain the emotion annotation label corresponding to the first cluster, and determine the emotion evaluation result of the speech data to be processed based on the emotion annotation label corresponding to the first cluster and the emotion recognition result set.

[0080] Since the emotion category corresponding to the first cluster is an emotion category that is unlikely to appear in the target application scenario, the emotion characteristics reflected by the speech segments in the first cluster do not conform to the objective scenario rules (objective laws). It can be considered that the emotion categories to which the speech segments in the first cluster belong are not accurate, that is, the speech emotion recognition model fails to accurately perform emotion recognition on the speech segments in the first cluster. At this time, the speech segment corresponding to the cluster center of the first cluster is obtained. The speech segment corresponding to the cluster center can be a speech segment with a speech feature as the cluster center; or a speech segment with a speech feature closest to the cluster center. For example, Figure 5 As shown, the cluster center of each cluster is marked with a five-pointed star.

[0081] The speech segment corresponding to the cluster center of the first cluster is sent to the user terminal, so that the user can mark the speech segment corresponding to the cluster center and obtain the marked category of the speech segment (the emotional category marked by the user). The user terminal obtains the marked category marked by the user for the speech segment corresponding to the cluster center, and returns it to the voice processing device. The voice processing device can receive the marked category sent by the user terminal and use the marked category as the emotional annotation label corresponding to the first cluster.

[0082] Since the speech segments in the first cluster cluster correspond to the same emotion category, the emotion annotation label corresponding to the first cluster cluster can be used as the real emotion category of the speech segments in the first cluster cluster. In one embodiment, the emotion category to which each speech segment included in the first cluster cluster in the emotion recognition result set belongs can be modified to the emotion annotation label corresponding to the first cluster cluster, and a modified emotion recognition result set is obtained. The modified emotion recognition result set is processed according to the preset scoring rule to obtain the emotion score of the speech data to be processed, and the emotion score is used as the emotion evaluation result of the speech data to be processed. For example, a score can be set for each emotion category, and each speech segment is scored according to the emotion category and corresponding score to which each speech segment belongs in the modified emotion recognition result set, and the sum of the scores of all speech segments is used as the emotion score of the speech data to be processed. Among them, the score of positive emotions (such as excitement, happiness, excitement) can be relatively high, and the score of negative emotions (such as sadness, grievance, fear) can be relatively low. Then, the larger the emotion score, the more the speech data to be processed tends to express positive emotions, and the lower the emotion score, the more the speech data to be processed tends to express negative emotions. By using this scoring method to determine the emotional evaluation results of the speech data to be processed, we can know the emotional category that the speech data to be processed tends to reflect, which is conducive to analyzing the conversation atmosphere in the target application scenario.

[0083] In summary, see Figure 6 , Figure 6 A flow chart of another speech processing method provided in an embodiment of the present application, the method comprising:

[0084] S601: Acquire voice data to be processed.

[0085] S602: using a general model to identify the emotion category to which each speech segment in the speech data to be processed belongs, and generating an emotion recognition result set by recording the emotion category to which each speech segment belongs.

[0086] S603: Determine whether the recognition results in the emotion recognition result set are normal or conform to the scene rules.

[0087] S604: If the recognition results in the emotion recognition result set are abnormal or do not conform to the scene rules, clustering processing is performed based on the voice features, and multiple voice segments in the voice data to be processed are divided into multiple clusters according to the clustering results.

[0088] S605, traverse all clusters, and judge through the emotion recognition result set. If the recognition results exceeding a certain threshold in the same cluster are of the same emotion category, the clustering result of the cluster is considered correct. Otherwise, increase the number of clusters and continue to judge until each cluster has an emotion category exceeding the threshold.

[0089] S606: For the clusters determined to be abnormal, select the speech segments corresponding to the cluster centers for manual judgment, and use the manual judgment results as the emotion categories to which the speech segments in the clusters belong. The abnormal clusters are the first clusters mentioned above.

[0090] S607, end, return the emotion evaluation result according to the preset rules. Specifically determine the emotion evaluation result of the speech data to be processed according to the preset rules, the manual judgment results and the emotion recognition result set. In addition, if the recognition results in the emotion recognition result set are normal or meet the scene rules, then end, and return the emotion evaluation result according to the preset rules. Specifically process the emotion recognition result set according to the preset rules to obtain the emotion evaluation result of the speech data to be processed.

[0091] By adopting the method of the present application, it is possible to determine whether the speech emotion recognition model accurately performs speech emotion recognition by determining whether the proportion of speech segments corresponding to different emotion categories in the speech data to be processed meets the proportion rule of the corresponding scene, and introduce some manual intervention when the speech emotion recognition model does not accurately perform speech emotion recognition, thereby greatly improving the accuracy of speech emotion recognition. In addition, by dividing the multiple speech segments of the speech data to be processed into multiple clusters, the results of each cluster can be guaranteed to be accurate, that is, each speech segment in each cluster tends to reflect the same emotional characteristics, so that the emotion annotation label corresponding to the first cluster is accurate when it is the real emotion category of each speech segment in the first cluster.

[0092] It can be understood that in the specific implementation of the present application, related data such as voice data to be processed is involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0093] The above detailed description of the method of the embodiment of the present application, in order to facilitate better implementation of the above method of the embodiment of the present application, the following provides a device of the embodiment of the present application. Figure 7 , Figure 7is a structural diagram of a speech processing device provided in an embodiment of the present application. In one embodiment, the speech processing device 70 may include:

[0094] An acquisition unit 701 is used to acquire speech data to be processed of a target application scenario, and determine an emotion recognition result set of the speech data to be processed based on a speech emotion recognition model, wherein the emotion recognition result set includes an emotion category to which each speech segment in a plurality of speech segments of the speech data to be processed belongs;

[0095] A processing unit 702 is configured to cluster the multiple voice segments to obtain multiple clusters if the emotion recognition result set does not meet the scenario rule of the target application scenario, wherein the scenario rule includes a predefined proportion rule of each reference emotion category, and the voice segments in the same cluster correspond to one emotion category;

[0096] The processing unit 702 is further configured to determine, from the multiple clusters, a first cluster whose emotion category does not match the target application scenario, and obtain an emotion annotation label corresponding to the first cluster;

[0097] The processing unit 702 is further configured to determine an emotion evaluation result of the speech data to be processed based on the emotion annotation label corresponding to the first cluster and the emotion recognition result set.

[0098] In one embodiment, the acquisition unit 701 is specifically used to: acquire the proportion of speech segments corresponding to different emotion categories in the emotion recognition result set;

[0099] The processing unit 702 is specifically used to: if there is a speech segment ratio corresponding to an emotion category in the emotion recognition result set that does not meet the ratio rule of the corresponding reference emotion category, determine that the emotion recognition result set does not meet the scenario rule of the target application scenario.

[0100] In one embodiment, the processing unit 702 is specifically used to: determine the emotion category corresponding to the maximum speech segment ratio based on the speech segment ratios corresponding to different emotion categories in any of the multiple clusters, and determine the emotion category corresponding to the maximum speech segment ratio as the emotion category corresponding to any of the clusters;

[0101] The acquisition unit 701 is specifically used to: acquire a positive emotion category matching the target application scenario;

[0102] The processing unit 702 is specifically configured to: if the emotion category corresponding to any cluster is not the positive emotion category matching the target application scenario, determine that any cluster is a first cluster whose emotion category does not match the target application scenario.

[0103] In one embodiment, the processing unit 702 is specifically used to: determine the speech feature of each speech segment in the multiple speech segments, and cluster the multiple speech segments based on the speech feature of each speech segment and the target expected number to obtain at least one cluster cluster, and the number of the at least one cluster cluster is the target expected number;

[0104] The acquisition unit 701 is specifically used to: acquire the total amount of speech segments of each cluster in the at least one cluster, and determine the cluster whose total amount of corresponding speech segments is less than the preset total amount as the second cluster;

[0105] The processing unit 702 is specifically used for: if the number of the second clusters is less than the preset number, then based on the proportions of speech segments corresponding to different emotion categories in each cluster, determining the maximum speech segment proportion corresponding to each cluster, the maximum speech segment proportion corresponding to a cluster is the speech segment proportion corresponding to the emotion category with the largest proportion in the cluster; if the maximum speech segment proportion corresponding to each cluster in the at least one cluster is greater than the preset proportion, then the clusters other than the second cluster in the at least one cluster are determined as multiple clusters.

[0106] In one embodiment, the target expected number is a first number; the processing unit 702 is specifically used to: if the number of the second clustering clusters is greater than or equal to the preset number, then the target expected number is updated to the second number, and the step of clustering the multiple voice segments based on the voice features of each voice segment and the target expected number is executed to obtain at least one clustering cluster, and the second number is less than the first number; or, if there is a cluster in the at least one clustering cluster whose corresponding maximum voice segment proportion is less than or equal to the preset proportion, then the target expected number is updated to the third number, and the step of clustering the multiple voice segments based on the voice features of each voice segment and the target expected number is executed to obtain at least one clustering cluster, and the third number is greater than the first number.

[0107] In one embodiment, the processing unit 702 is specifically used to: determine the speech segment corresponding to the cluster center of the first clustering cluster; send the speech segment corresponding to the cluster center to the user terminal, so that the user terminal obtains the annotation category annotated by the user for the speech segment corresponding to the cluster center; receive the annotation category sent by the user terminal, and use the annotation category as the emotion annotation label corresponding to the first clustering cluster.

[0108] In one embodiment, the processing unit 702 is specifically used to: modify the emotion category to which each speech segment included in the first cluster in the emotion recognition result set belongs to into an emotion annotation label corresponding to the first cluster, to obtain a modified emotion recognition result set; process the modified emotion recognition result set according to a preset scoring rule, to obtain an emotion evaluation result of the speech data to be processed.

[0109] In one embodiment, the processing unit 702 is specifically used to: perform endpoint detection on the speech data to be processed, and segment the speech data to be processed based on the endpoint detection result to obtain multiple speech segments of the speech data to be processed; determine the speech features of each speech segment in the multiple speech segments; input the speech features of each speech segment into a speech emotion recognition model to obtain the emotion category to which each speech segment belongs; and determine a set of emotion recognition results for the speech data to be processed based on the emotion category to which each speech segment belongs.

[0110] It can be understood that the functions of each functional unit of the speech processing device described in the embodiment of the present application can be specifically implemented according to the method in the above method embodiment, and its specific implementation process can refer to the relevant description of the above method embodiment, which will not be repeated here.

[0111] Through the method of the present application, it is possible to determine whether it is necessary to introduce some manual intervention on the basis of speech emotion recognition model recognition based on whether the scene categories to which multiple speech segments of the speech data to be processed belong conform to objective scene rules, thereby greatly improving the accuracy of speech emotion recognition.

[0112] like Figure 8 As shown, Figure 8 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. The internal structure of the computer device 80 is as follows Figure 8 As shown, it includes: one or more processors 801, memory 802, and communication interface 803. The processor 801, memory 802, and communication interface 803 can be connected via a bus 804 or other methods. The embodiment of the present application takes the connection via the bus 804 as an example.

[0113] Among them, the processor 801 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device 80, which can parse various instructions in the computer device 80 and process various data of the computer device 80. For example, the CPU can be used to parse the power on and off instructions sent by the user to the computer device 80, and control the computer device 80 to perform power on and off operations; for another example, the CPU can transmit various interactive data between the internal structures of the computer device 80, and so on. The communication interface 803 can optionally include a standard wired interface, a wireless interface (such as Wi-Fi, a mobile communication interface, etc.), which is controlled by the processor 801 to send and receive data. The memory 802 (Memory) is a memory device in the computer device 80, which is used to store computer programs and data. It can be understood that the memory 802 here can include both the built-in memory of the computer device 80 and the extended memory supported by the computer device 80. The memory 802 provides a storage space, which stores the operating system of the computer device 80, which may include but is not limited to: Windows system, Linux system, Android system, iOS system, etc., and this application does not limit this. The processor 801 executes the following operations by running the computer program stored in the memory 802:

[0114] Acquire speech data to be processed of a target application scenario, and determine an emotion recognition result set of the speech data to be processed based on a speech emotion recognition model, wherein the emotion recognition result set includes an emotion category to which each speech segment in a plurality of speech segments of the speech data to be processed belongs;

[0115] If the emotion recognition result set does not meet the scenario rule of the target application scenario, clustering the multiple voice clips to obtain multiple clustering clusters, wherein the scenario rule includes a predefined proportion rule of each reference emotion category, and the voice clips in the same clustering cluster correspond to one emotion category;

[0116] Determine a first cluster whose emotion category does not match the target application scenario from the multiple clusters, and obtain an emotion annotation label corresponding to the first cluster;

[0117] Based on the emotion annotation label corresponding to the first cluster and the emotion recognition result set, an emotion evaluation result of the speech data to be processed is determined.

[0118] In one embodiment, the processor 801 is specifically used to: obtain the proportion of speech segments corresponding to different emotion categories in the emotion recognition result set; if there is a speech segment proportion corresponding to an emotion category in the emotion recognition result set that does not meet the proportion rule of the corresponding reference emotion category, then determine that the emotion recognition result set does not meet the scenario rule of the target application scenario.

[0119] In one embodiment, the processor 801 is specifically used to: determine the emotion category corresponding to the maximum speech segment ratio based on the speech segment ratios corresponding to different emotion categories in any of the multiple clusters, and determine the emotion category corresponding to the maximum speech segment ratio as the emotion category corresponding to any of the clusters; obtain a positive emotion category matching the target application scenario; if the emotion category corresponding to any of the clusters is not the positive emotion category matching the target application scenario, determine that any of the clusters is the first cluster whose emotion category does not match the target application scenario.

[0120] In one embodiment, the processor 801 is specifically used to: determine the speech features of each speech segment in the multiple speech segments, and cluster the multiple speech segments based on the speech features of each speech segment and the target expected number to obtain at least one cluster, and the number of the at least one cluster is the target expected number; obtain the total amount of speech segments of each cluster in the at least one cluster, and determine the cluster whose corresponding total amount of speech segments is less than the preset total amount as the second cluster; if the number of the second cluster is less than the preset number, then based on the proportion of speech segments corresponding to different emotion categories in each cluster, determine the maximum proportion of speech segments corresponding to each cluster, the maximum proportion of speech segments corresponding to a cluster is the proportion of speech segments corresponding to the emotion category with the largest proportion in the cluster; if the maximum proportion of speech segments corresponding to each cluster in the at least one cluster is greater than the preset proportion, then the clusters other than the second cluster in the at least one cluster are determined as multiple clusters.

[0121] In one embodiment, the target expected number is a first number; the processor 801 is specifically used to: if the number of the second clustering clusters is greater than or equal to the preset number, then the target expected number is updated to the second number, and the step of clustering the multiple voice segments based on the voice features of each voice segment and the target expected number is executed to obtain at least one clustering cluster, and the second number is less than the first number; or, if there is a cluster in the at least one clustering cluster whose corresponding maximum voice segment proportion is less than or equal to the preset proportion, then the target expected number is updated to the third number, and the step of clustering the multiple voice segments based on the voice features of each voice segment and the target expected number is executed to obtain at least one clustering cluster, and the third number is greater than the first number.

[0122] In one embodiment, the processor 801 is specifically used to: determine the speech segment corresponding to the cluster center of the first clustering cluster; send the speech segment corresponding to the cluster center to the user terminal, so that the user terminal obtains the annotation category annotated by the user for the speech segment corresponding to the cluster center; receive the annotation category sent by the user terminal, and use the annotation category as the emotion annotation label corresponding to the first clustering cluster.

[0123] In one embodiment, the processor 801 is specifically used to: modify the emotion category to which each speech segment included in the first cluster in the emotion recognition result set belongs to into an emotion annotation label corresponding to the first cluster, to obtain a modified emotion recognition result set; process the modified emotion recognition result set according to a preset scoring rule, to obtain an emotion evaluation result of the speech data to be processed.

[0124] In one embodiment, the processor 801 is specifically used to: perform endpoint detection on the voice data to be processed, and segment the voice data to be processed based on the endpoint detection result to obtain multiple voice segments of the voice data to be processed; determine the voice features of each voice segment in the multiple voice segments; input the voice features of each voice segment into a voice emotion recognition model to obtain the emotion category to which each voice segment belongs; and determine a set of emotion recognition results for the voice data to be processed based on the emotion category to which each voice segment belongs.

[0125] In a specific implementation, the processor 801, memory 802 and communication interface 803 described in the embodiment of the present application can execute the implementation method described in a speech processing method provided in the embodiment of the present application, and can also execute the implementation method described in a speech processing device provided in the embodiment of the present application, which will not be repeated here.

[0126] Through the method of the present application, it is possible to determine whether it is necessary to introduce some manual intervention on the basis of speech emotion recognition model recognition based on whether the scene categories to which multiple speech segments of the speech data to be processed belong conform to objective scene rules, thereby greatly improving the accuracy of speech emotion recognition.

[0127] The embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is run on a computer device, the computer device executes the speech processing method of any possible implementation method described above. The specific implementation method can be referred to the above description, which will not be repeated here.

[0128] The present application also provides a computer program product, which includes a computer program or computer instructions, and when the computer program or computer instructions are executed by a processor, the steps of the speech processing method provided in the present application are implemented. The specific implementation method can be referred to the above description, which will not be repeated here.

[0129] The embodiment of the present application also provides a computer program, the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium, the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the speech processing method provided in the embodiment of the present application. The specific implementation method can be referred to the above description, which will not be repeated here.

[0130] It should be noted that, for the above-mentioned various method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0131] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0132] The above disclosure is only part of the embodiments of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A speech processing method, characterized in that: The method comprises: Acquire speech data to be processed of a target application scenario, and determine an emotion recognition result set of the speech data to be processed based on a speech emotion recognition model, wherein the emotion recognition result set includes an emotion category to which each speech segment in a plurality of speech segments of the speech data to be processed belongs; If the emotion recognition result set does not meet the scenario rule of the target application scenario, clustering the multiple voice clips to obtain multiple clustering clusters, wherein the scenario rule includes a predefined proportion rule of each reference emotion category, and the voice clips in the same clustering cluster correspond to one emotion category; Determine a first cluster that does not match the target application scenario based on the emotion category of the largest speech segment proportion corresponding to any cluster among the multiple clusters, and obtain an emotion annotation label corresponding to the first cluster, wherein the emotion annotation label is used to indicate the true emotion category of each speech segment in the first cluster; Based on the emotion annotation label corresponding to the first cluster and the emotion recognition result set, an emotion evaluation result of the speech data to be processed is determined.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining the proportion of speech segments corresponding to different emotion categories in the emotion recognition result set; If there is a speech segment ratio corresponding to an emotion category in the emotion recognition result set that does not meet the ratio rule of the corresponding reference emotion category, it is determined that the emotion recognition result set does not meet the scenario rule of the target application scenario.

3. The method according to claim 1, characterized in that The determining of a first cluster that does not match the target application scenario based on the emotion category corresponding to the largest voice segment proportion of any cluster among the multiple clusters includes: Based on the speech segment proportions corresponding to different emotion categories in any cluster of the multiple clusters, determine the emotion category corresponding to the largest speech segment proportion, and determine the emotion category corresponding to the largest speech segment proportion as the emotion category corresponding to any cluster; Acquire a positive sentiment category matching the target application scenario; If the emotion category corresponding to any cluster is not the positive emotion category matching the target application scenario, then the any cluster is determined to be a first cluster whose emotion category does not match the target application scenario.

4. The method according to any one of claims 1 to 3, characterized in that The clustering process is performed on the multiple voice segments to obtain multiple clusters, including: Determine the speech feature of each speech segment in the multiple speech segments, and cluster the multiple speech segments based on the speech feature of each speech segment and the target expected number to obtain at least one cluster, and the number of the at least one cluster is the target expected number; Acquire the total amount of speech segments of each cluster in the at least one cluster, and determine the cluster whose total amount of corresponding speech segments is less than a preset total amount as a second cluster; If the number of the second clusters is less than the preset number, then based on the proportions of speech segments corresponding to different emotion categories in each cluster, determine the maximum proportion of speech segments corresponding to each cluster, the maximum proportion of speech segments corresponding to a cluster is the proportion of speech segments corresponding to the emotion category with the largest proportion in the cluster; If the maximum speech segment proportion corresponding to each cluster in the at least one cluster is greater than a preset proportion, the clusters in the at least one cluster except the second cluster are determined as a plurality of clusters.

5. The method according to claim 4, characterized in that The target expected quantity is a first quantity; the method further includes: If the number of the second clusters is greater than or equal to the preset number, the target expected number is updated to a second number, and the step of clustering the multiple speech segments based on the speech features of each speech segment and the target expected number is performed to obtain at least one cluster, and the second number is less than the first number; or, If there is a cluster in at least one of the clusters whose corresponding maximum speech segment ratio is less than or equal to the preset ratio, the target expected number is updated to a third number, and the step of clustering the multiple speech segments based on the speech features of each speech segment and the target expected number is performed to obtain at least one cluster, and the third number is greater than the first number.

6. The method according to any one of claims 1 to 3, characterized in that The obtaining of the sentiment label corresponding to the first cluster includes: Determining a speech segment corresponding to the cluster center of the first cluster; Sending the speech segment corresponding to the cluster center to a user terminal so that the user terminal obtains the annotated category annotated by the user for the speech segment corresponding to the cluster center; The annotation category sent by the user terminal is received, and the annotation category is used as the emotion annotation label corresponding to the first cluster.

7. The method according to any one of claims 1 to 3, characterized in that The step of determining the emotion evaluation result of the speech data to be processed based on the emotion annotation label corresponding to the first cluster and the emotion recognition result set includes: Modify the emotion category to which each speech segment included in the first cluster in the emotion recognition result set belongs to into the emotion annotation label corresponding to the first cluster, to obtain a modified emotion recognition result set; The modified emotion recognition result set is processed according to a preset scoring rule to obtain an emotion evaluation result of the speech data to be processed.

8. A speech processing device, characterized in that: The device comprises: An acquisition unit, used to acquire speech data to be processed of a target application scenario, and determine an emotion recognition result set of the speech data to be processed based on a speech emotion recognition model, wherein the emotion recognition result set includes an emotion category to which each speech segment in a plurality of speech segments of the speech data to be processed belongs; A processing unit, configured to cluster the multiple voice segments to obtain multiple clusters if the emotion recognition result set does not meet the scenario rule of the target application scenario, wherein the scenario rule includes a predefined proportion rule of each reference emotion category, and the voice segments in the same cluster cluster correspond to one emotion category; The processing unit is further used to determine a first cluster that does not match the target application scenario based on the emotion category of the largest speech segment proportion corresponding to any cluster among the multiple clusters, and obtain an emotion annotation label corresponding to the first cluster, wherein the emotion annotation label is used to indicate the true emotion category of each speech segment in the first cluster; The processing unit is further configured to determine an emotion evaluation result of the speech data to be processed based on the emotion annotation label corresponding to the first cluster and the emotion recognition result set.

9. A computer device, characterized in that: The computer device includes a memory, a communication interface and a processor, wherein the memory, the communication interface and the processor are interconnected; the memory stores a computer program, and the processor calls the computer program stored in the memory to implement the speech processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the speech processing method according to any one of claims 1 to 7 is implemented.

Images