Voice quality inspection method, computing device and computer readable storage medium

Abstract

The invention discloses a voice quality inspection method, a computing device and a computer readable storage medium, and the method comprises the steps: obtaining a target call voice of a first role, the target call voice being a corresponding speaking content when the first role communicates with a second role; performing emotion type recognition on the target call voice based on a trained voice emotion recognition model to obtain an emotion type recognition result of the first role; the speech emotion recognition model is established based on an attention mechanism and a residual network; and generating a voice quality inspection result according to the emotion type recognition result. Therefore, emotion type recognition is carried out on the call voice through the voice emotion recognition model established based on the attention mechanism and the residual network, the emotion of a dialogue person can be accurately recognized, the accuracy and comprehensiveness of a quality inspection result are improved, meanwhile, the quality inspection efficiency is improved, and the quality inspection cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of speech processing technology, and in particular to a speech quality inspection method, a computing device and a computer-readable storage medium. Background Art

[0002] In daily work and life, people often call customer service centers to solve their problems or handle certain business matters. The service quality of the staff affects the overall customer satisfaction with the company and also represents the company's image. Therefore, it is necessary to conduct quality inspections on the service process of the staff, and quality inspection of call voice is an important part of it.

[0003] Among them, manual quality inspection is a commonly used method, that is, professional quality inspectors check the call recordings one by one to identify whether there are irregular behaviors in the call based on the audio content of the call recordings and the attitude of the staff, and can detect customer satisfaction with the service. However, manual quality inspection methods have problems such as low efficiency and high cost.

[0004] However, although the existing speech quality inspection methods based on speech recognition technology have improved the quality inspection efficiency to a certain extent, they are unable to analyze and identify the emotions of the interlocutors, which in turn affects the accuracy of the quality inspection results. Summary of the invention

[0005] The purpose of this application is to provide a speech quality inspection method, computing equipment, engineering machinery and computer-readable storage medium, which can accurately identify the emotions of the interlocutors, improve the accuracy of the quality inspection results, and at the same time improve the quality inspection efficiency and reduce the quality inspection cost.

[0006] To achieve the above objectives:

[0007] In a first aspect, an embodiment of the present application provides a voice quality inspection method, comprising:

[0008] Obtaining a target call voice of the first character, where the target call voice is the corresponding speech content of the first character when talking to the second character;

[0009] Based on the trained speech emotion recognition model, the target call speech is subjected to emotion type recognition to obtain an emotion type recognition result of the first role; the speech emotion recognition model is established based on an attention mechanism and a residual network;

[0010] A speech quality inspection result is generated according to the emotion type recognition result.

[0011] Optionally, the performing emotion type recognition on the target call speech based on the trained speech emotion recognition model to obtain an emotion type recognition result for the first role includes:

[0012] Preprocessing the target call speech; the preprocessing includes pre-emphasis and frame windowing;

[0013] Performing feature extraction on the preprocessed target call speech to obtain Mel-spectrogram cepstral coefficient features;

[0014] The Mel-spectrogram cepstral coefficient feature is input into a trained speech emotion recognition model to obtain an emotion type recognition result of the first character output by the speech emotion recognition model.

[0015] Optionally, generating a speech quality inspection result according to the emotion type recognition result includes:

[0016] Recognize the target call voice to obtain the call text of the first role;

[0017] Using the trained emotion analysis model to perform emotion recognition on the call text to obtain the emotion type of the first role;

[0018] A speech quality inspection result is generated according to the emotion type and the emotion type recognition result.

[0019] Optionally, generating a speech quality inspection result according to the emotion type and the emotion type recognition result includes:

[0020] Detecting whether the call text contains any non-standard language and / or meets preset quality inspection rules, and obtaining a text detection result;

[0021] According to the emotion type, the emotion type recognition result and the text detection result, the target call voice is scored based on a preset scoring rule to obtain a voice quality inspection result.

[0022] Optionally, it also includes:

[0023] When it is determined according to the text detection result that non-standard language exists in the call text, a corresponding operation is performed based on the type of the first role.

[0024] Optionally, the performing a corresponding operation based on the type of the first role includes:

[0025] If the type of the first role is a preset type, a warning message is issued;

[0026] If the type of the first role is not a preset type, a comfort message is sent.

[0027] Optionally, it also includes:

[0028] When it is determined that the emotion type of the first character is a preset emotion type and / or the emotional type is a preset emotional type, a reminder message is output.

[0029] Optionally, before outputting the reminder message, the method further includes:

[0030] Get the interval time between the last output reminder message;

[0031] When the interval duration is greater than or equal to the preset duration, the step of outputting a reminder message is performed.

[0032] In a second aspect, an embodiment of the present application provides a computing device, comprising: a processor and a memory storing a computer program, and when the processor runs the computer program, the above-mentioned voice quality inspection method is implemented.

[0033] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the above-mentioned speech quality inspection method is implemented.

[0034] The speech quality inspection method, computing device and computer-readable storage medium provided in the embodiments of the present application include: obtaining the target call speech of the first character, the target call speech being the corresponding speech content of the first character when talking to the second character; performing emotion type recognition on the target call speech based on the trained speech emotion recognition model to obtain the emotion type recognition result of the first character; the speech emotion recognition model is established based on the attention mechanism and the residual network; and generating a speech quality inspection result according to the emotion type recognition result. In this way, by performing emotion type recognition on the call speech based on the speech emotion recognition model established based on the attention mechanism and the residual network, the emotions of the interlocutor can be accurately identified, which improves the accuracy and comprehensiveness of the quality inspection results, while improving the quality inspection efficiency and reducing the quality inspection cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flow chart of a speech quality inspection method provided by an embodiment of the present invention;

[0036] Figure 2 Schematic diagram of the process of voice quality inspection in an embodiment of the present invention;

[0037] Figure 3 Schematic diagram of the emotion recognition process of the speech emotion recognition model in an embodiment of the present invention;

[0038] Figure 4 A schematic diagram of the structure of a computing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0040] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0041] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of..." or "when..." or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising", "including" indicate that there are described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Thus, “A, B, or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A, B, and C.” An exception to this definition will occur only when a combination of elements, functions, steps, or operations are inherently mutually exclusive in some manner.

[0042] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0043] It should be noted that, in this article, step codes such as S101, S102, etc. are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the sequence. When implementing the step, those skilled in the art may execute S102 first and then S101, etc., but these should all be within the scope of protection of this application.

[0044] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0046] See also Figure 1 , is a voice quality inspection method provided in an embodiment of the present application. The voice quality inspection method can be executed by a voice quality inspection device provided in an embodiment of the present application. The voice quality inspection device can be implemented in software and / or hardware, such as a computing device such as a computer or a server. In this embodiment, the execution subject of the voice quality inspection method is a computing device as an example. The voice quality inspection method provided in this embodiment includes:

[0047] Step S101: Acquire a target call voice of the first character, where the target call voice is the corresponding speech content when the first character is talking with the second character.

[0048] Specifically, during a conversation between a first character and a second character, a target conversation voice of the first character is obtained in real time or periodically, and the target conversation voice is stored in a database.

[0049] In the case where the call voice of the first role cannot be collected separately, the call voices of the first role and the second role can be separated to obtain the target call voice of the first role. The first role and the second role are different, and the first role and the second role can be one of a staff member and a customer respectively, and the staff member can be specifically a seat staff, etc.

[0050] Step S102: Based on the trained speech emotion recognition model, the emotion type of the target call speech is recognized to obtain the emotion type recognition result of the first character; the speech emotion recognition model is established based on the attention mechanism and the residual network.

[0051] Among them, the speech emotion recognition model is established based on the attention mechanism and the residual network (ResNet), and the speech emotion recognition model can be trained by training samples including historical call voices and corresponding emotion types. The specific training process can refer to the existing technology. The residual network is a residual convolutional neural network. By introducing residual learning, complex mappings are decomposed into multiple simple mappings, thereby simplifying network training and improving model performance. The deep learning ability of ResNet can effectively extract and learn complex time-frequency domain features, thereby improving the accuracy and robustness of emotion recognition. In addition, by introducing the attention mechanism, the speech emotion recognition model can be helped to more accurately identify the emotional content in the speech signal, and then dynamically adjust the attention of the speech emotion recognition model to different features, making the speech emotion recognition model more sensitive and effective in processing emotional details.

[0052] In one embodiment, the emotion type of the target call speech is recognized based on the trained speech emotion recognition model to obtain the emotion type recognition result of the first role, including:

[0053] Preprocess the target call voice; the preprocessing includes pre-emphasis and frame windowing;

[0054] Perform feature extraction on the preprocessed target call speech to obtain Mel-spectrum cepstral coefficient features;

[0055] The Mel-spectrum cepstral coefficient features are input into the trained speech emotion recognition model to obtain the emotion type recognition result of the first character output by the speech emotion recognition model.

[0056] Among them, pre-emphasis is to compensate for the energy attenuation of the high-frequency part of the speech signal, so as to achieve the overall flattening of the speech signal spectrum, which is convenient for subsequent feature extraction. Frame windowing not only retains the transient characteristics of the speech signal, but also provides a short-term stable signal block for further analysis, so that it is convenient to extract the characteristics of the speech signal efficiently. By extracting the features of the pre-processed target call speech, the obtained Mel spectrum cepstral coefficient features will include the spectral features, short-term spectrum change features, cepstral features and acoustic characteristics of the speech signal, thereby improving the accuracy and comprehensiveness of the emotion type recognition based on the Mel spectrum cepstral coefficient features. Here, extracting the features of the pre-processed target call speech and obtaining the Mel spectrum cepstral coefficient features may include: performing a fast Fourier transform on the pre-processed target call speech to convert the time domain signal into a frequency domain signal; then, filtering the converted signal through a Mel filter to obtain a processed signal; then, performing logarithmic operations and discrete cosine transforms on the processed signal in sequence to obtain Mel spectrum cepstral coefficient features.

[0057] Step S103: Generate a speech quality inspection result according to the emotion type recognition result.

[0058] Among them, the emotion type recognition result includes the emotion type of the identified first character, which may be angry, sad, happy, natural, etc. It can be understood that since the emotion type of the first character will affect the views or attitudes of the second character, the corresponding call voice when the first character and the second character are talking can be quality-checked based on the emotion type recognition result to obtain the voice quality inspection result. It should be noted that the voice quality inspection result in this embodiment can be generated based on all the obtained emotion type recognition results after the emotion type recognition is performed on all the target call voices of the first character within a period of time. For example, the target call voice of the first character can be obtained in real time from the beginning of the call between the first character and the second character to the end of the call, and the emotion type of the target call voice can be recognized in real time, and the voice quality inspection result can be generated based on all the emotion type recognition results of the first character.

[0059] It should be noted that, since the conversation between the first character and the second character may last for a certain period of time, a voice quality inspection result can be generated based on all emotion type recognition results of the first character within this period of time.

[0060] In one embodiment, generating a speech quality inspection result according to the emotion type recognition result includes:

[0061] Recognize the target call voice and obtain the call text of the first role;

[0062] Use the trained sentiment analysis model to perform sentiment recognition on the call text to obtain the sentiment type of the first character;

[0063] Generate speech quality inspection results based on emotion type and sentiment type recognition results.

[0064] Among them, the target call recording can be recognized by voice recognition technology to obtain the call text of the first character. Here, the emotion type of the first character can be obtained as a positive or negative attitude by using the trained emotion analysis model to perform emotion recognition on the call text content. Among them, the emotion analysis model can be an existing natural language processing model, which will not be repeated here. Since the emotion type of the first character will also affect the call voice quality, the voice quality inspection result can be generated based on the emotion type of the first character and the recognition result of the emotion type of the first character. In this way, by recognizing the call audio data as text and combining the text and voice data for quality inspection, the quality inspection object is enriched, and the accuracy and comprehensiveness of the quality inspection results are further improved.

[0065] In one embodiment, a speech quality inspection result is generated according to the emotion type and the emotion type recognition result, including:

[0066] Detect whether the call text contains any non-standard language and / or meets the preset quality inspection rules, and obtain the text detection result;

[0067] According to the emotion type, sentiment type recognition results and text detection results, the target call voice is scored based on the preset scoring rules to obtain the voice quality inspection results.

[0068] Among them, in order to comprehensively check the quality of the call process, in addition to checking the emotion type of the first role and the emotional type of the first role, it is also possible to check whether the call text contains non-standard language and / or whether it meets the preset quality inspection rules. Non-standard language may include sensitive words and banned words, etc. Specifically, the call text can be matched with the preset sensitive word library and banned word library to detect whether the call text contains non-standard language. The preset quality inspection rules can be set according to actual needs, such as frequent appearance (such as more than 3 times) of words such as "well, ah, yeah, okay, right, right", etc., or failure to use standard spoken language; it can also be failure to use standard welcome words, including but not limited to failure to use welcome words and / or closing words, or use non-standard welcome words and closing words; it can also be polite words such as "please, excuse me, trouble you" that do not appear at least once during the call.

[0069] Among them, the preset scoring rules can be set according to the actual needs. For example, when negative emotions or emotional types such as anger or sadness are detected, 10 points will be deducted, while when positive emotions or emotional types such as happiness or naturalness are detected, no points will be deducted; each time sensitive words and banned words are triggered, 10 points will be deducted; frequent occurrence of words such as "yeah, ah, yeah, okay, right, right" will be deducted 5 points, etc. In this way, the target call voice can be scored accurately and comprehensively, further improving the accuracy and comprehensiveness of the quality inspection results.

[0070] In one embodiment, it further comprises:

[0071] When it is determined according to the text detection result that there is non-standard language in the call text, a corresponding operation is performed based on the type of the first role.

[0072] Among them, when there are non-standard languages ​​such as sensitive words and / or banned words in the call text, corresponding operations can be performed based on the type of the first character to minimize or prevent the appearance of non-standard languages ​​in subsequent calls, or to comfort characters who may be mentally harmed by non-standard languages, thereby improving user satisfaction.

[0073] In one implementation, performing corresponding operations based on the type of the first role includes:

[0074] If the type of the first character is a preset type, a warning message is issued;

[0075] If the type of the first character is not a preset type, a comfort message is sent.

[0076] The preset type may be a staff member such as an agent, etc., and the non-preset type may be a customer, a consulting user, etc. When the type of the first role is the preset type, a warning message may be issued to make the first role pay attention to language expression during subsequent calls. When the type of the first role is the non-preset type, a comforting message may be issued to enhance the experience or divert attention.

[0077] In one embodiment, it further comprises:

[0078] When it is determined that the emotion type of the first character is a preset emotion type and / or the emotion type is a preset emotion type, a reminder message is output.

[0079] Among them, the preset emotion type and the preset emotion type can be set according to the actual situation, for example, the preset emotion type can be set to anger or sadness, and the preset emotion type can be set to negativity, etc. It can be understood that when the emotion type of the first role is the preset emotion type and / or the emotion type is the preset emotion type, since the corresponding speech content of the first role when continuing the call with the second role may affect the experience of the second role, such as making the second role think that the service attitude is poor, etc., therefore, when it is determined that the emotion type of the first role is the preset emotion type and / or the emotion type is the preset emotion type, a reminder message can be output to promptly remind the first role to make corrections, etc., thereby improving the user experience. The reminder message can be set according to the actual situation, for example, it can be a text or voice message such as "Please pay attention to the expression of emotions or emotions".

[0080] In one embodiment, before outputting the reminder message, the method further includes:

[0081] Get the interval time between the last output reminder message;

[0082] When the interval duration is greater than or equal to the preset duration, a step of outputting a reminder message is executed.

[0083] Specifically, when it is necessary to output a reminder message, first obtain the interval time between the last output reminder message, and detect whether the interval time between the last output reminder message and the last output reminder message is greater than or equal to the preset time. If so, execute the step of outputting the reminder message. If not, prohibit outputting the reminder message, that is, do not execute the step of outputting the reminder message. In this way, by controlling the same reminder message to be issued within the preset time, repeated reminders can be effectively avoided and the number of reminders can be reduced.

[0084] In summary, in the speech quality inspection method provided in the above embodiment, the emotion type of the call speech is identified by using the speech emotion recognition model established based on the attention mechanism and the residual network, which can accurately identify the emotions of the interlocutors, improve the accuracy and comprehensiveness of the quality inspection results, and at the same time improve the quality inspection efficiency and reduce the quality inspection cost.

[0085] Based on the same inventive concept as the aforementioned embodiment, the speech quality inspection method provided in this embodiment is illustrated below by using a specific example.

[0086] See also Figure 2 The speech quality inspection method provided in this embodiment mainly includes the following processes:

[0087] 1. Acquire call voice data (hereinafter referred to as voice data) in real time.

[0088] 2. Convert voice data to text data.

[0089] Here, automatic speech recognition technology can be used to translate the call voice data into call text content, i.e., text data, in real time, and record the call text content.

[0090] 3. Match the call text content with the preset sensitive words and banned words to detect whether there is any non-standard language in the call voice. If so, first determine the sender of the voice. If the sender is the customer service, a warning needs to be issued to the customer service to remind them to use standard language and attitude; if the sender is the user, a reminder is issued to the customer service to appease the user's emotions. To avoid repetition or too many reminders, the same reminder message can be set not to be issued every 3 minutes. At the same time, during the detection process, scores are recorded according to the preset scoring rules and the results are recorded.

[0091] 4. Match the call text content with the preset quality inspection rule library to detect whether the call content triggers the quality inspection item content, score it according to the preset scoring rules, and record the corresponding results.

[0092] 5. Emotion recognition and sentiment analysis

[0093] (1) Preliminary emotion recognition: Use a pre-trained emotion analysis model to perform emotion recognition on the call text content and classify the target call as positive or negative. In this embodiment, the Transformers library of Hugging Face is used, which is a powerful natural language processing (NLP) tool set. These models are pre-trained on large-scale datasets and can be used for various NLP tasks, such as text classification, sentiment analysis, etc., and can be further fine-tuned to adapt to specific tasks. Pre-trained models usually have high performance characteristics, and direct use or fine-tuning saves a lot of computing resources and time.

[0094] (2) Emotional analysis: Speech signals have richer acoustic information and can more accurately reflect emotional characteristics. In this embodiment, an end-to-end speech emotion recognition model is used to efficiently learn the representation form related to emotional information from the original call audio data, thereby completing the emotion recognition task more accurately. Here, in this embodiment, a ResNet network speech emotion recognition model based on an attention mechanism is used to classify the emotions of the target call into four types: angry, sad, happy, and natural.

[0095] Among them, the speech emotion recognition model based on the attention mechanism and ResNet network proposed in this embodiment is as follows: Figure 3 As shown in the figure, the emotion recognition process of the speech emotion recognition model is as follows:

[0096] 1) Input voice data.

[0097] 2) Voice data preprocessing

[0098] It mainly includes pre-emphasis and frame windowing. Pre-emphasis is to compensate for the energy attenuation of the high-frequency part of the speech signal, so as to achieve the overall flattening of the signal spectrum, which is convenient for subsequent feature extraction. Frame windowing not only retains the transient characteristics of the speech signal, but also provides short-term stable signal blocks for further analysis, so as to efficiently extract the characteristics of the speech signal.

[0099] 3) Speech data feature extraction

[0100] The feature extraction of speech data mainly includes time domain features, frequency domain features and energy features. In the present embodiment, the Mel spectrum cepstral coefficients (MFCC) are extracted as the input of the network model. The specific steps are as follows: First, the pre-processed speech signal data is input into the Fast Fourier Transform (Fast Fourier Transform, FFT) module to convert the time domain signal into a frequency domain signal. Then, the converted signal is filtered through a Mel filter, and the Mel filter can simulate the perception characteristics of the human ear to different frequencies. Subsequently, a logarithmic operation is performed on the signal processed by the filter to compress the dynamic range of the signal. Finally, the MFCC feature coefficients are obtained by discrete cosine transform (Discrete Cosine Transform, DCT), and the first 13 MFCC coefficients are usually selected.

[0101] 4) Model training

[0102] ResNet (Residual Network) is a residual convolutional neural network. By introducing residual learning, complex mappings are decomposed into multiple simple mappings, thereby simplifying network training and improving model performance. ResNet's deep learning capabilities can effectively extract and learn complex time-frequency domain features, thereby improving the accuracy and robustness of emotion recognition.

[0103] Training data preparation: (1) Feature matrix, each audio clip is represented as an MFCC matrix with a shape of (time step, feature dimension), for example (100, 13); (2) Label, each sample corresponds to an emotion label, usually including angry, sad, happy and natural.

[0104] ResNet network design:

[0105] (1) Input layer: The input layer of ResNet is designed to adapt to the dimensions of the MFCC feature matrix. For example, if the shape of the MFCC feature matrix is ​​(time step, feature dimension), the input layer can be set to (time step, feature dimension, 1), where 1 represents a single channel (similar to a grayscale image).

[0106] (2) Residual block:

[0107] Convolutional layer: Use convolutional layer to extract features. Select the appropriate convolution kernel size according to the time step and feature dimension of MFCC features, such as 3x3 or 5x5. This article uses 3x3.

[0108] Residual connection: Introducing residual connection in the convolutional layer allows information to flow across layers in the network and alleviates the gradient vanishing problem.

[0109] (3) Pooling layer: A max pooling layer is used after the convolutional layer to reduce the feature dimension and computational complexity while retaining important information.

[0110] (4) Attention mechanism: assigning higher learning weight to the emotional information in speech.

[0111] (5) Fully connected layer: A fully connected layer is added after the convolution and pooling layers to map the extracted features to the final emotion categories.

[0112] (6) Classification layer: Use the Softmax activation function to perform sentiment classification and output category probabilities.

[0113] In this embodiment, the attention mechanism is specially introduced to help the model more accurately identify the emotional content in the speech signal. When the model processes data, the attention mechanism can assign a higher learning weight to the emotional information in the speech, ensuring that important emotional features are more fully learned. Specifically, the attention mechanism can dynamically adjust the model's attention to different features, making the model more sensitive and effective in processing emotional details.

[0114] Subsequently, the feature vector processed by the attention mechanism is input into a fully connected layer and nonlinearly transformed using the ReLU activation function to increase the model's expressiveness. In order to prevent the model from overfitting during training, a Dropout layer is added after the fully connected layer, and the dropout rate is set to 50% to randomly discard some neurons, thereby enhancing the generalization ability of the model.

[0115] In the classification stage, the processed feature vector is input into the Softmax classifier for sentiment classification. The Softmax classifier contains an output layer of size 4, corresponding to 4 different sentiment categories: angry, sad, happy, and natural. Through the Softmax function, the model can output the probability distribution of each sentiment category, and finally select the category with the highest probability as the prediction result of the model.

[0116] The training process of the entire model uses the cross entropy loss function, which can effectively measure the difference between the model's predicted value and the true value and guide the model to update parameters. At the same time, the stochastic gradient descent method (SGD) is used for optimization, and the model parameters are adjusted iteratively so that the loss function gradually converges to the minimum value.

[0117] Combining the sentiment analysis results of text and voice, when negative emotions are identified or classified as angry or sad, a reminder message will be sent to the customer service, and the score will be recorded according to the preset scoring rules. To avoid repetition or excessive reminders, set not to send the same reminder message every 3 minutes;

[0118] 6. Combining the test results of the above steps, the final intelligent quality inspection result, such as the quality inspection score, is obtained.

[0119] In summary, in the voice quality inspection method provided in the above embodiment, the call audio data is translated into text in real time through voice recognition technology, and the text and voice data are combined for quality inspection, which enriches the quality inspection object. At the same time, the converted call text can be stored and used for subsequent re-inspection, which is convenient for quick positioning and viewing. If the positioning is performed by playing the recording each time, the efficiency is low. This method not only matches and inspects sensitive words and banned words, as well as the quality inspection rule library preset by the enterprise, but also performs sentiment analysis on the call text content and voice data, uses a pre-trained NPL model to perform preliminary sentiment classification on the text content, and adopts an end-to-end method to learn the representation form related to the emotional information from the original call data to obtain the emotion recognition result. Finally, multiple scoring items are combined to obtain the final score and complete the intelligent quality inspection. In addition, this method is also real-time, and timely warns or reminds customer service when sensitive words or negative emotions are detected.

[0120] Based on the same inventive concept as the above embodiments, an embodiment of the present invention provides a computing device, such as Figure 4 As shown, the computing device includes: a processor 310 and a memory 311 storing a computer program; wherein, Figure 4 The processor 310 shown in the figure is not used to indicate that the number of the processor 310 is one, but is only used to indicate the positional relationship of the processor 310 relative to other devices. In actual applications, the number of the processor 310 may be one or more; similarly, Figure 4 The memory 311 shown in the figure has the same meaning, that is, it is only used to refer to the position relationship of the memory 311 relative to other devices. In practical applications, the number of memories 311 can be one or more. When the processor 310 runs the computer program, the above-mentioned voice quality inspection method is implemented.

[0121] The computing device may also include: at least one network interface 312. The various components in the computing device are coupled together via a bus system 313. It is understood that the bus system 313 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 313 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 313 is not used in the example above. Figure 4 Various buses are labeled as bus system 313.

[0122] The memory 311 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 311 described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable types of memories.

[0123] The memory 311 in the embodiment of the present invention is used to store various types of data to support the operation of the computing device. Examples of these data include: any computer program used to operate on the computing device, such as an operating system and an application; contact data; phone book data; messages; pictures; videos, etc. Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program may include various applications, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. Here, the program that implements the method of the embodiment of the present invention may be included in the application program.

[0124] Based on the same inventive concept as the above-mentioned embodiment, this embodiment further provides a computer storage medium, in which a computer program is stored. The computer storage medium may be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) or other memory; it may also be various devices including one or any combination of the above-mentioned memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc. When the computer program stored in the computer storage medium is run by the processor, the above-mentioned voice quality inspection method is implemented. For the specific steps implemented when the computer program is executed by the processor, please refer to Figure 1 The description of the illustrated embodiment will not be repeated here.

[0125] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0126] In this document, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion of elements other than those listed and may also include additional elements not expressly listed.

[0127] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A speech quality inspection method, characterized in that: The method comprises: Obtaining a target call voice of the first character, where the target call voice is the corresponding speech content of the first character when talking to the second character; Based on the trained speech emotion recognition model, the target call speech is subjected to emotion type recognition to obtain an emotion type recognition result of the first role; the speech emotion recognition model is established based on an attention mechanism and a residual network; A speech quality inspection result is generated according to the emotion type recognition result.

2. The method according to claim 1, characterized in that The performing emotion type recognition on the target call speech based on the trained speech emotion recognition model to obtain an emotion type recognition result for the first role includes: Preprocessing the target call speech; the preprocessing includes pre-emphasis and frame windowing; Performing feature extraction on the preprocessed target call speech to obtain Mel-spectrogram cepstral coefficient features; The Mel-spectrogram cepstral coefficient feature is input into a trained speech emotion recognition model to obtain an emotion type recognition result of the first character output by the speech emotion recognition model.

3. The method according to claim 1 or 2, characterized in that: Generating a speech quality inspection result according to the emotion type recognition result includes: Recognize the target call voice to obtain the call text of the first role; Using the trained emotion analysis model to perform emotion recognition on the call text to obtain the emotion type of the first role; A speech quality inspection result is generated according to the emotion type and the emotion type recognition result.

4. The method according to claim 3, characterized in that The generating of the speech quality inspection result according to the emotion type and the emotion type recognition result includes: Detecting whether the call text contains any non-standard language and / or meets preset quality inspection rules, and obtaining a text detection result; According to the emotion type, the emotion type recognition result and the text detection result, the target call voice is scored based on a preset scoring rule to obtain a voice quality inspection result.

5. The method according to claim 4, characterized in that Also includes: When it is determined according to the text detection result that non-standard language exists in the call text, a corresponding operation is performed based on the type of the first role.

6. The method according to claim 5, characterized in that The performing a corresponding operation based on the type of the first role includes: If the type of the first role is a preset type, issuing a warning message; If the type of the first role is not a preset type, a comfort message is sent.

7. The method according to claim 3, characterized in that Also includes: When it is determined that the emotion type of the first character is a preset emotion type and / or the emotional type is a preset emotional type, a reminder message is output.

8. The method according to claim 7, characterized in that Before outputting the reminder message, the method further includes: Get the interval time between the last output reminder message; When the interval duration is greater than or equal to the preset duration, the step of outputting a reminder message is performed.

9. A computing device, characterized in that include: A processor and a memory storing a computer program, wherein when the processor runs the computer program, the speech quality inspection method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the speech quality inspection method according to any one of claims 1 to 8 is implemented.

Images