A method, apparatus, device, and storage medium for voice emotion recognition.

Abstract

This application belongs to the field of digital medical technology and is applied in smart consultation and remote consultation scenarios. It relates to a voice emotion recognition method, device, equipment, and its storage medium, including real-time acquisition of patient voice information generated during patient service; feature extraction of the patient voice information using MATLAB to obtain the MFCC features corresponding to the patient voice information; inputting the MFCC features into a trained emotion detection model to obtain the emotion detection result; and feeding back the emotion detection result to the target customer service representative. Using a BP neural network as the emotion detection model architecture has good universality and self-learning ability. Furthermore, during model training and optimization, the improved bat optimization algorithm only requires training and optimization of the weights and biases, resulting in faster initial bat search speeds, reducing the likelihood of getting trapped in local optima, and gradually slowing down in later stages to approach the global optimum, thus improving iteration efficiency and thereby enhancing the training and optimization efficiency of the emotion detection model.

Description

Technical Field

[0001] This application relates to the field of digital medical technology and is applied to smart consultation and remote consultation scenarios. In particular, it relates to a voice emotion recognition method, device, equipment and its storage medium. Background Technology

[0002] Customer service is a stable and reliable service method in the service industry. With the development of digital healthcare, especially intelligent online consultation customer service, providing services through communication with patients allows them to experience the warmth of the service and makes it easier to gain their trust. When communicating with others, if you can obtain the patient's current emotional state and then tailor your communication accordingly, it is easier to shorten the distance between you and the patient and gain their trust. Usually, a person's emotional state is mainly expressed through facial expressions, voice, and body language, but it is most importantly expressed through voice.

[0003] Currently, numerous machine learning-based methods for speech emotion recognition exist, with convolutional neural networks and long short-term memory networks being the most widely used deep learning algorithms. However, due to the complexity of network model parameters and the uncontrollable training process, these deep learning networks often result in inaccurate emotional state classifications, making them difficult to apply to complex customer service environments. Therefore, existing technologies lack a universally applicable and highly self-learning speech emotion recognition method. Summary of the Invention

[0004] The purpose of this application is to propose a voice emotion recognition method, apparatus, device and storage medium, in order to solve the problem that the existing technology lacks a voice emotion recognition method with strong universality and self-learning ability.

[0005] To address the aforementioned technical problems, this application provides a voice emotion recognition method, employing the following technical solution:

[0006] A voice emotion recognition method includes the following steps:

[0007] Retrieve pre-set emotion category tags;

[0008] From the preset MFCC feature training set with labeled classification tags, obtain the MFCC features corresponding to different sentiment classification tags respectively, and add the MFCC features to different MFCC feature sets according to the different sentiment classification tags to obtain the set addition result;

[0009] All MFCC features in the MFCC feature training set are input into the pre-built sentiment detection model for model training. Based on the training results and the set addition results, the trained sentiment detection model is obtained. The pre-built sentiment detection model is a classification prediction model built with a BP neural network architecture.

[0010] Real-time acquisition of patient voice information generated during patient service;

[0011] MATLAB was used to extract features from the patient's speech information to obtain the MFCC features corresponding to the patient's speech information.

[0012] The MFCC features are input into the trained sentiment detection model to obtain the sentiment detection results;

[0013] The emotion detection results are then fed back to the target customer service representative.

[0014] Furthermore, the emotion classification labels include happiness, anger, sadness, fear, and surprise. The step of obtaining the MFCC features corresponding to different emotion classification labels, and adding the MFCC features to different MFCC feature sets according to the different emotion classification labels to obtain the set addition results specifically includes:

[0015] From the pre-defined MFCC feature training set with pre-labeled classification tags, obtain MFCC features representing happiness, anger, sadness, fear, and surprise, respectively.

[0016] And construct a distinction set using the categories of happiness, anger, sadness, fear, and surprise as the distinction names;

[0017] The MFCC features corresponding to happiness, anger, sadness, fear, and surprise are added to the corresponding distinction sets to obtain the set addition results.

[0018] Furthermore, before performing the step of inputting all MFCC features in the MFCC feature training set into the pre-built sentiment detection model for model training, and obtaining the trained sentiment detection model based on the training results and the set addition results, the method further includes:

[0019] Deploy pre-set initial weights and initial biases to the pre-built sentiment detection model;

[0020] The steps of inputting all MFCC features in the MFCC feature training set into the pre-built sentiment detection model for training, and obtaining the trained sentiment detection model based on the training results and the set addition results, specifically include:

[0021] Step A: The improved bat optimization algorithm is used to optimize the initial weights and initial biases of the pre-built sentiment detection model to obtain the optimal sentiment detection model after optimization.

[0022] Step B: Based on the optimal sentiment detection model, perform sentiment detection and classification on all MFCC features to obtain the classification results;

[0023] Step C: Calculate the loss of the optimal sentiment detection model using the classification results, the set addition results, and the loss function of the sentiment detection model.

[0024] Step D: Determine whether the loss exceeds a preset loss threshold;

[0025] If step E is exceeded, the weights and biases of the optimal sentiment detection model are optimized and adjusted using the backpropagation method, and steps B to D are repeated.

[0026] If step F does not exceed the limit, then the optimal sentiment detection model is taken as the completed sentiment detection model.

[0027] Furthermore, the step of using an improved bat optimization algorithm to optimize the initial weights and initial biases of the pre-built sentiment detection model to obtain the optimized optimal sentiment detection model specifically includes:

[0028] Step a: Obtain the pre-introduced dynamic inertia factor, and improve the bat optimization algorithm based on the dynamic inertia factor to obtain the improved bat optimization algorithm;

[0029] Step b: Use the initial weights and initial biases as the starting position information of the individual bats;

[0030] Step c: Based on the improved bat optimization algorithm and the starting position information of the individual bat, perform iterative updates to calculate the current position information of the individual bat;

[0031] Step d: After each iteration update, obtain the weight and bias values ​​corresponding to the current location information of the individual bat, and deploy them to the pre-built sentiment detection model for classification verification to obtain the classification verification results;

[0032] Step e: Based on the classification verification results, select the optimal weight and optimal bias value from previous iterations and reset them as the starting position information of the individual bats. Repeat steps c, d, and e.

[0033] Step f continues until the iteration termination condition is met. The optimal weights and optimal biases from previous iterations are selected and set to the pre-built sentiment detection model to obtain the optimal sentiment detection model. The iteration termination condition includes: the optimal weights and optimal biases no longer change, or the preset maximum number of iterations is reached.

[0034] Furthermore, the step of obtaining a pre-introduced dynamic inertia factor and improving the bat optimization algorithm based on the dynamic inertia factor to obtain an improved bat optimization algorithm specifically includes:

[0035] According to the preset first formula: V i (t+1)=σ t .V i (t)+(X i (t)-X′).f i And the pre-defined second formula: σ t =σ max -(σ max -σ min (1+t) -1 The improved bat optimization algorithm is obtained, where V i (t+1) and V i (t) represents the search change rate of individual bats at times t+1 and t, respectively; X i (t) represents the position of the i-th bat at time t; X′ represents the current global optimal position; f i The frequency of the transmitted pulse is a random number between (0,1); σ max and σ min σ represents the maximum and minimum values ​​of the dynamic inertia factor. t This represents the current dynamic inertia factor.

[0036] Furthermore, the step of obtaining the weights and biases corresponding to the current location information of the individual bat, and deploying them to the pre-built sentiment detection model for classification verification to obtain the classification verification result specifically includes:

[0037] Set the weights and biases to the latest weights and biases of the pre-built sentiment detection model;

[0038] All the MFCC features are input into the pre-built sentiment detection model to perform sentiment detection and classification, and the classification results are obtained.

[0039] Based on the classification results and the preset loss function, the backpropagation error of the BP neural network is calculated, and the error value is used as the classification verification result.

[0040] The step of selecting the optimal weights and biases from previous iterations based on the classification verification results and resetting them as the starting position information of individual bats specifically includes:

[0041] Obtain the error values ​​obtained after each previous iteration update, and compare all error values;

[0042] The weights and biases corresponding to the minimum error values ​​are selected as the optimal weights and biases in previous iterations.

[0043] Furthermore, the step of using MATLAB to extract features from the patient's speech information and obtain the MFCC features corresponding to the patient's speech information specifically includes:

[0044] The patient's voice information is preprocessed to obtain a preprocessing result, wherein the preprocessing steps include pre-emphasis, framing, and windowing in sequence;

[0045] The power spectrum of the preprocessed results is estimated using the periodogram method to obtain the target power spectrum.

[0046] The target power spectrum is filtered using the Mel filter bank, and the energy in each filter of the Mel filter bank is calculated.

[0047] Perform a logarithmic operation on the energy in each filter and obtain the result of each logarithmic operation;

[0048] Perform DCT transformation on each logarithmic operation result, and obtain the Delta coefficient corresponding to the preset frame value from the DCT transformation result as the MFCC feature.

[0049] To address the aforementioned technical problems, this application also provides a voice emotion recognition device, which employs the following technical solution:

[0050] A voice emotion recognition device, comprising:

[0051] The emotion category tag acquisition module is used to acquire pre-set emotion category tags;

[0052] The training set classification module is used to obtain MFCC features corresponding to different sentiment classification labels from the preset MFCC feature training set with labeled classification labels, and add the MFCC features to different MFCC feature sets according to the different sentiment classification labels to obtain the set addition result;

[0053] The sentiment detection model training module is used to input all MFCC features in the MFCC feature training set into the pre-built sentiment detection model for model training. Based on the training results and the set addition results, the trained sentiment detection model is obtained. The pre-built sentiment detection models are all classification prediction models built with a BP neural network architecture.

[0054] The patient voice information acquisition module is used to acquire patient voice information generated during the service process in real time.

[0055] The MFCC feature extraction module is used to extract features from the patient's speech information using MATLAB to obtain the MFCC features corresponding to the patient's speech information.

[0056] The emotion detection result acquisition module is used to input the MFCC features into the trained emotion detection model to obtain the emotion detection result;

[0057] The emotion detection result feedback module is used to feed back the emotion detection results to the target customer service representative.

[0058] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0059] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the speech emotion recognition method described above.

[0060] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0061] A computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the speech emotion recognition method described above.

[0062] Compared with the prior art, the embodiments of this application have the following main advantages:

[0063] The speech emotion recognition method described in this application involves: acquiring pre-set emotion classification labels; obtaining MFCC features corresponding to different emotion classification labels from a pre-set MFCC feature training set with labeled classification labels; adding the MFCC features to different MFCC feature sets according to the different emotion classification labels to obtain set addition results; inputting all MFCC features in the MFCC feature training set into a pre-constructed emotion detection model for model training to obtain a trained emotion detection model; acquiring patient speech information generated during patient service in real time; using MATLAB to extract features from the patient speech information to obtain the corresponding MFCC features; inputting the MFCC features into the trained emotion detection model to obtain emotion detection results; and feeding back the emotion detection results to the target customer service representative. Using a backpropagation (BP) neural network as the architecture for the sentiment detection model has good universality and self-learning ability. At the same time, when training and optimizing the model, the improved bat optimization algorithm only needs to train and optimize the weights and biases. This makes the search speed of the bat faster in the early stage and less likely to get stuck in local optima. In the later stage, the speed gradually slows down, allowing it to gradually approach the global optimum, thus improving the iteration efficiency and thereby improving the training and optimization efficiency of the sentiment detection model. Attached Figure Description

[0064] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0066] Figure 2 A flowchart of an embodiment of the speech emotion recognition method according to this application;

[0067] Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 202 shown;

[0068] Figure 4 yes Figure 2 A flowchart of a specific embodiment of step 203 shown;

[0069] Figure 5 yes Figure 4 A flowchart of a specific embodiment of step 401 shown;

[0070] Figure 6 yes Figure 5A flowchart of a specific embodiment of step 504 shown;

[0071] Figure 7 yes Figure 2 A flowchart of a specific embodiment of step 205 shown;

[0072] Figure 8 A schematic diagram of the structure of an embodiment of the voice emotion recognition device according to this application;

[0073] Figure 9 A schematic diagram of the structure of an embodiment of the computer device according to this application. Detailed Implementation

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0075] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0076] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0077] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0078] Patients can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various patient communication applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email patient clients, social media platform software, etc.

[0079] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0080] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0081] It should be noted that the voice emotion recognition method provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the voice emotion recognition device is generally set in the server / terminal device.

[0082] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0083] Continue to refer to Figure 2 A flowchart of an embodiment of the speech emotion recognition method according to this application is shown. The speech emotion recognition method includes the following steps:

[0084] Step 201: Obtain the pre-set emotion classification tags.

[0085] In this embodiment, the emotion classification labels include happiness, anger, sadness, fear, and surprise. Other labeling for emotion categories is not excluded, such as anger, happiness, or emotion / feeling level labels. Pre-setting emotion classification labels facilitates the classification and recognition of different emotional speech information.

[0086] Step 202: From the preset MFCC feature training set with labeled classification tags, obtain the MFCC features corresponding to different sentiment classification tags respectively, and add the MFCC features to different MFCC feature sets according to the different sentiment classification tags to obtain the set addition result.

[0087] Continue to refer to Figure 3 , Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 202 shown includes:

[0088] Step 301: Obtain MFCC features representing happiness, anger, sadness, fear, and surprise from the pre-defined MFCC feature training set with pre-labeled classification tags.

[0089] Step 302, and construct a distinction set using the categories of happiness, anger, sadness, fear, and surprise as the distinction names;

[0090] Step 303: Add the MFCC features corresponding to happiness, anger, sadness, fear and surprise to the corresponding distinction sets to obtain the set addition results.

[0091] By constructing sentiment discrimination sets using labels as distinguishing names, it is easier to compare the output results of the sentiment detection model in the form of a classification set when there are many elements in the MFCC feature training set, thereby improving the comparison efficiency and thus improving the training, optimization and detection efficiency of the model.

[0092] Step 203: Input all MFCC features in the MFCC feature training set into the pre-built sentiment detection model for model training. Based on the training results and the set addition results, obtain the trained sentiment detection model. The pre-built sentiment detection model is a classification prediction model built with a BP neural network architecture.

[0093] In this embodiment, before performing the steps of inputting all MFCC features in the MFCC feature training set into the pre-built sentiment detection model for model training, and obtaining the trained sentiment detection model based on the training results and the set addition results, the method further includes: deploying pre-set initial weights and initial biases to the pre-built sentiment detection model.

[0094] Backpropagation (BP) neural networks are multi-layer feedforward neural networks trained based on error backpropagation. Compared to existing convolutional neural networks and long short-term memory networks used for speech emotion recognition, they have a simpler structure, stronger non-linear mapping capabilities, and better universality and self-learning ability. BP neural networks primarily rely on the setting of weights and biases (thresholds) between network structures. Therefore, during model training and optimization, only the weights and biases need to be trained and optimized, thereby improving the detection accuracy and recognition success rate of the speech emotion detection model.

[0095] Continue to refer to Figure 4 , Figure 4 yes Figure 2 A flowchart of a specific embodiment of step 203 shown includes:

[0096] Step 401: Using the improved bat optimization algorithm, the initial weights and initial biases of the pre-built sentiment detection model are optimized to obtain the optimal sentiment detection model after optimization.

[0097] Continue to refer to Figure 5 , Figure 5 yes Figure 4 A flowchart of a specific embodiment of step 401 shown includes:

[0098] Step 501: Obtain the pre-introduced dynamic inertia factor, and improve the bat optimization algorithm according to the dynamic inertia factor to obtain the improved bat optimization algorithm;

[0099] In this embodiment, the step of obtaining a pre-introduced dynamic inertia factor and improving the bat optimization algorithm based on the dynamic inertia factor to obtain an improved bat optimization algorithm specifically includes: according to a preset first formula: V i (t+1)=σ t .V i (t)+(X i (t)-X′).f i And the pre-defined second formula: σ t =σ max -(σ max -σ min (1+t) -1 The improved bat optimization algorithm is obtained, where V i (t+1) and V i (t) represents the search change rate of individual bats at times t+1 and t, respectively; X i (t) represents the position of the i-th bat at time t; X′ represents the current global optimal position; f i The frequency of the transmitted pulse is a random number between (0,1); σ max and σmin σ represents the maximum and minimum values ​​of the dynamic inertia factor. t This represents the current dynamic inertia factor.

[0100] If the Bat Optimization Algorithm uses a fixed step size during the optimization process, its search speed will be relatively monotonous. Furthermore, if the speed is too large, it may deviate from the optimal value; if the speed is too small, it is prone to getting trapped in local optima. Therefore, to improve the Bat Optimization Algorithm's global search capability in the early stages and its local optimization capability in the later stages, a dynamic inertia factor σ is introduced. t And the maximum value σ of the dynamic inertia factor is limited. max and minimum value σ min This allows the bats to search faster in the early stages, making them less likely to get stuck in local optima, while the speed gradually slows down in the later stages, allowing them to gradually approach the global optimum, thus improving iteration efficiency and thereby improving the training and optimization efficiency of the sentiment detection model.

[0101] Step 502: Use the initial weights and initial biases as the starting position information of the individual bats;

[0102] Step 503: Based on the improved bat optimization algorithm and the starting position information of the individual bat, perform iterative updates to calculate the current position information of the individual bat;

[0103] Step 504: After each iteration update, obtain the weight and bias value corresponding to the current position information of the individual bat, and deploy it to the pre-built emotion detection model for classification verification to obtain the classification verification result;

[0104] Continue to refer to Figure 6 , Figure 6 yes Figure 5 A flowchart of a specific embodiment of step 504 shown includes:

[0105] Step 601: Set the weights and biases to the latest weights and biases of the pre-built sentiment detection model;

[0106] Step 602: Input all the MFCC features into the pre-built sentiment detection model, perform sentiment detection and classification, and obtain the classification results;

[0107] Step 603: Calculate the backpropagation error of the BP neural network based on the classification result and the preset loss function, and obtain the error value as the classification verification result.

[0108] Step 505: Based on the classification verification results, select the optimal weight and optimal bias value from previous iterations and reset them as the starting position information of the individual bat. Repeat steps 503 to 505.

[0109] In this embodiment, the step of selecting the optimal weights and biases from previous iterations based on the classification verification results and resetting them as the starting position information of the individual bats specifically includes: obtaining the error values ​​obtained after each previous iteration and comparing all error values; selecting the weights and biases corresponding to the smallest error value as the optimal weights and biases from previous iterations.

[0110] Step 506: Until the iteration termination condition is met, the optimal weight and optimal bias value from previous iteration updates are selected and set to the pre-built sentiment detection model to obtain the optimal sentiment detection model. The iteration termination condition includes: the optimal weight and optimal bias value no longer change, or the preset maximum number of iterations is reached.

[0111] By employing an improved bat optimization algorithm to optimize the initial weights and initial biases of the pre-built sentiment detection model, a better and more accurate sentiment detection model is obtained.

[0112] Step 402: Perform sentiment detection and classification on all MFCC features based on the optimal sentiment detection model to obtain the classification results;

[0113] Step 403: Calculate the loss degree of the optimal sentiment detection model using the classification results, the set addition results, and the loss function of the sentiment detection model;

[0114] Step 404: Determine whether the loss exceeds a preset loss threshold;

[0115] Step 405: If the error exceeds the limit, the weights and biases of the optimal emotion detection model are optimized and adjusted using the backpropagation method, and steps 402 to 404 are repeated.

[0116] Step 406: If the value does not exceed the limit, then the optimal sentiment detection model is taken as the completed sentiment detection model.

[0117] Step 204: Acquire patient voice information generated during the service process in real time.

[0118] In this embodiment, the real-time acquisition of patient voice information generated during the process of providing services to patients includes online smart consultation or remote consultation services. The patient voice information generated during the process of providing services to patients includes patient / symptom information received through online smart consultation or remote consultation services, which is exchanged between hospital customer service and patients in a question-and-answer manner.

[0119] By adopting online smart consultation or remote consultation services, combined with semantic recognition, it is more convenient to conduct remote diagnosis and treatment for patients. On the one hand, this alleviates the pressure of large offline patient traffic in hospitals, and on the other hand, it helps patients with transportation or mobility difficulties to save on consultation costs.

[0120] The online intelligent consultation or remote consultation service described in this application can be applied to consultations in various medical disciplines, such as consultations for minor surgical wounds, minor orthopedic sprains, dermatitis, and psychological medicine.

[0121] Step 205: Use MATLAB to extract features from the patient's speech information to obtain the MFCC features corresponding to the patient's speech information.

[0122] In this embodiment, MATLAB (Matrix Laboratory) is a general-purpose scientific computing software that can calculate the MFCC features contained in the patient's speech information.

[0123] Continue to refer to Figure 7 , Figure 7 yes Figure 2 A flowchart of a specific embodiment of step 205 shown includes:

[0124] Step 701: Preprocess the patient's voice information to obtain a preprocessing result, wherein the preprocessing steps include pre-emphasis, framing, and windowing in sequence;

[0125] In this embodiment, the purpose of the pre-emphasis is to ensure that the high frequency and low frequency are within the same audio fluctuation range, which facilitates frame segmentation and windowing processing.

[0126] Step 702: The power spectrum of the preprocessed result is estimated using the periodogram method to obtain the target power spectrum;

[0127] Step 703: Filter the target power spectrum according to the Mel filter bank and calculate the energy in each filter of the Mel filter bank;

[0128] In this embodiment, a typical Mel filter bank includes 16 filters.

[0129] Step 704: Perform a logarithmic operation on the energy in each filter and obtain the result of each logarithmic operation;

[0130] Step 705: Perform DCT transformation on each logarithmic operation result, and obtain the Delta coefficient corresponding to the preset frame value from the DCT transformation result as the MFCC feature.

[0131] In this embodiment, the preset frame value can be a specific target frame, such as the 5th frame or the 6th frame, or it can be a frame value range, such as all frames from the 2nd frame to the 8th frame. In addition, when acquiring the data, the Delta coefficient corresponding to each frame can be of first order or second order. If the Delta coefficient is of first order, the MFCC feature is a one-dimensional sequence. If the Delta coefficient is of second order, the MFCC feature is a two-dimensional feature matrix.

[0132] Step 206: Input the MFCC features into the trained sentiment detection model to obtain the sentiment detection result.

[0133] Step 207: Feedback the emotion detection results to the target customer service representative.

[0134] This application obtains pre-set emotion classification labels; from a pre-set MFCC feature training set with labeled classification labels, it obtains MFCC features corresponding to different emotion classification labels, adds the MFCC features to different MFCC feature sets according to the different emotion classification labels, and obtains set addition results; it inputs all MFCC features in the MFCC feature training set into a pre-built emotion detection model for model training, and obtains the trained emotion detection model; it acquires patient voice information generated during patient service in real time; it uses MATLAB to extract features from the patient voice information to obtain the MFCC features corresponding to the patient voice information; it inputs the MFCC features into the trained emotion detection model to obtain emotion detection results; and it feeds back the emotion detection results to the target customer service representative. Using a backpropagation (BP) neural network as the architecture for the sentiment detection model has good universality and self-learning ability. At the same time, when training and optimizing the model, the improved bat optimization algorithm only needs to train and optimize the weights and biases. This makes the search speed of the bat faster in the early stage and less likely to get stuck in local optima. In the later stage, the speed gradually slows down, allowing it to gradually approach the global optimum, thus improving the iteration efficiency and thereby improving the training and optimization efficiency of the sentiment detection model.

[0135] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0136] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0137] In this embodiment, the use of a BP neural network as the emotion detection model architecture has good universality and self-learning ability. At the same time, during model training and optimization, the improved bat optimization algorithm only needs to train and optimize the weights and biases, which makes the search speed of the bat faster in the early stage and less likely to get stuck in local optima. In the later stage, the speed gradually slows down, which can gradually approach the global optimum, thus improving the iteration efficiency and thereby improving the training and optimization efficiency of the emotion detection model.

[0138] Further reference Figure 8 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a voice emotion recognition device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0139] like Figure 8 As shown, the speech emotion recognition device 800 described in this embodiment includes: an emotion classification label acquisition module 801, a training set classification module 802, an emotion detection model training module 803, a patient speech information acquisition module 804, an MFCC feature extraction module 805, an emotion detection result acquisition module 806, and an emotion detection result feedback module 807. Wherein:

[0140] The emotion classification tag acquisition module 801 is used to acquire pre-set emotion classification tags;

[0141] The training set classification module 802 is used to obtain MFCC features corresponding to different sentiment classification labels from the preset MFCC feature training set with labeled classification labels, and add the MFCC features to different MFCC feature sets according to the different sentiment classification labels to obtain the set addition result;

[0142] The sentiment detection model training module 803 is used to input all MFCC features in the MFCC feature training set into the pre-built sentiment detection model for model training, and obtain the trained sentiment detection model based on the training results and the set addition results. The pre-built sentiment detection models are all classification prediction models built with BP neural network architecture.

[0143] The patient voice information acquisition module 804 is used to acquire patient voice information generated during the process of providing services to patients in real time.

[0144] The MFCC feature extraction module 805 is used to extract features from the patient's speech information using MATLAB to obtain the MFCC features corresponding to the patient's speech information.

[0145] The emotion detection result acquisition module 806 is used to input the MFCC features into the trained emotion detection model to obtain the emotion detection result;

[0146] The emotion detection result feedback module 807 is used to feed back the emotion detection result to the target customer service representative.

[0147] This application obtains pre-set emotion classification labels; from a pre-set MFCC feature training set with labeled classification labels, it obtains MFCC features corresponding to different emotion classification labels, adds the MFCC features to different MFCC feature sets according to the different emotion classification labels, and obtains set addition results; it inputs all MFCC features in the MFCC feature training set into a pre-built emotion detection model for model training, and obtains the trained emotion detection model; it acquires patient voice information generated during patient service in real time; it uses MATLAB to extract features from the patient voice information to obtain the MFCC features corresponding to the patient voice information; it inputs the MFCC features into the trained emotion detection model to obtain emotion detection results; and it feeds back the emotion detection results to the target customer service representative. Using a backpropagation (BP) neural network as the architecture for the sentiment detection model has good universality and self-learning ability. At the same time, when training and optimizing the model, the improved bat optimization algorithm only needs to train and optimize the weights and biases. This makes the search speed of the bat faster in the early stage and less likely to get stuck in local optima. In the later stage, the speed gradually slows down, allowing it to gradually approach the global optimum, thus improving the iteration efficiency and thereby improving the training and optimization efficiency of the sentiment detection model.

[0148] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0149] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0150] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 9 , Figure 9 This is a basic structural block diagram of the computer device in this embodiment.

[0151] The computer device 9 includes a memory 9a, a processor 9b, and a network interface 9c that are interconnected via a system bus. It should be noted that only the computer device 9 with components 9a-9c is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0152] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0153] The memory 9a includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 9a may be an internal storage unit of the computer device 9, such as the hard disk or memory of the computer device 9. In other embodiments, the memory 9a may also be an external storage device of the computer device 9, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 9. Of course, the memory 9a may include both the internal storage unit and its external storage device of the computer device 9. In this embodiment, the memory 9a is typically used to store the operating system and various application software installed on the computer device 9, such as computer-readable instructions for voice emotion recognition methods. In addition, the memory 9a can also be used to temporarily store various types of data that have been output or will be output.

[0154] In some embodiments, the processor 9b may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 9b is typically used to control the overall operation of the computer device 9. In this embodiment, the processor 9b is used to execute computer-readable instructions stored in the memory 9a or to process data, such as executing computer-readable instructions for the voice emotion recognition method.

[0155] The network interface 9c may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 9 and other electronic devices.

[0156] The computer device proposed in this embodiment belongs to the field of digital medical technology and is applied in intelligent consultation and remote diagnosis scenarios. This application obtains pre-set emotion classification labels; from a pre-set MFCC feature training set with labeled classification labels, it obtains MFCC features corresponding to different emotion classification labels, adds these MFCC features to different MFCC feature sets according to the different emotion classification labels, and obtains the set addition results; it inputs all MFCC features in the MFCC feature training set into a pre-constructed emotion detection model for model training, and obtains the trained emotion detection model; it acquires patient voice information generated during patient service in real time; it uses MATLAB to extract features from the patient voice information to obtain the MFCC features corresponding to the patient voice information; it inputs the MFCC features into the trained emotion detection model to obtain the emotion detection result; and it feeds back the emotion detection result to the target customer service representative. Using a backpropagation (BP) neural network as the architecture for the sentiment detection model has good universality and self-learning ability. At the same time, when training and optimizing the model, the improved bat optimization algorithm only needs to train and optimize the weights and biases. This makes the search speed of the bat faster in the early stage and less likely to get stuck in local optima. In the later stage, the speed gradually slows down, allowing it to gradually approach the global optimum, thus improving the iteration efficiency and thereby improving the training and optimization efficiency of the sentiment detection model.

[0157] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by a processor to cause the processor to perform the steps of the speech emotion recognition method described above.

[0158] The computer-readable storage medium proposed in this embodiment belongs to the field of digital medical technology and is applied in smart consultation and remote consultation scenarios. This application obtains pre-set emotion classification labels; from a pre-set MFCC feature training set with labeled classification labels, it obtains MFCC features corresponding to different emotion classification labels, adds the MFCC features to different MFCC feature sets according to the different emotion classification labels, and obtains the set addition results; it inputs all MFCC features in the MFCC feature training set into a pre-built emotion detection model for model training, and obtains the trained emotion detection model; it acquires patient voice information generated during patient service in real time; it uses MATLAB to extract features from the patient voice information to obtain the MFCC features corresponding to the patient voice information; it inputs the MFCC features into the trained emotion detection model to obtain the emotion detection result; and it feeds back the emotion detection result to the target customer service representative. Using a backpropagation (BP) neural network as the architecture for the sentiment detection model has good universality and self-learning ability. At the same time, when training and optimizing the model, the improved bat optimization algorithm only needs to train and optimize the weights and biases. This makes the search speed of the bat faster in the early stage and less likely to get stuck in local optima. In the later stage, the speed gradually slows down, allowing it to gradually approach the global optimum, thus improving the iteration efficiency and thereby improving the training and optimization efficiency of the sentiment detection model.

[0159] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0160] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A method for speech emotion recognition, characterized in that, The method comprises the following steps: obtaining a pre-set emotion classification label; from the pre-set labeled classification label MFCC feature training set, respectively obtaining the MFCC features corresponding to different emotion classification labels, and adding the MFCC features into different MFCC feature sets according to the different emotion classification labels to obtain a set addition result; inputting all MFCC features in the MFCC feature training set into a pre-constructed emotion detection model for model training, and obtaining a trained emotion detection model according to the training result and the set addition result, wherein the pre-constructed emotion detection model is a classification prediction model based on a BP neural network architecture, and specifically comprises: Step A, using an improved bat optimization algorithm to optimize the initial weight and initial bias of the pre-constructed emotion detection model to obtain an optimal emotion detection model after optimization; Step B, performing emotion detection classification on the all MFCC features based on the optimal emotion detection model to obtain a classification result; Step C, calculating the loss degree of the optimal emotion detection model through the classification result, the set addition result and the loss function of the emotion detection model; Step D, determining whether the loss degree exceeds a pre-set loss degree threshold; Step E, if it exceeds, using the error back propagation method to optimize and adjust the weight and bias of the optimal emotion detection model, and repeating steps B to D; Step F, if it does not exceed, taking the optimal emotion detection model as the trained emotion detection model; real-time obtaining patient voice information generated during service to a patient; using MATLAB to extract features of the patient voice information to obtain MFCC features corresponding to the patient voice information; inputting the MFCC features into the trained emotion detection model to obtain an emotion detection result; feeding back the emotion detection result to a target customer service.

2. The voice emotion recognition method of claim 1, wherein, The emotion classification label comprises happy, angry, sad, scared and surprised, and the step of respectively obtaining the MFCC features corresponding to different emotion classification labels and adding the MFCC features into different MFCC feature sets according to the different emotion classification labels to obtain a set addition result comprises: respectively obtaining MFCC features representing happy, angry, sad, scared and surprised from the pre-set labeled classification label MFCC feature training set; constructing a difference set with the happy, angry, sad, scared and surprised as the difference set names; adding the MFCC features corresponding to the happy, angry, sad, scared and surprised respectively into the corresponding difference set to obtain the set addition result. 3.The voice emotion recognition method of claim 1, wherein, Before the step of inputting all MFCC features in the MFCC feature training set into a pre-constructed emotion detection model for model training, and obtaining a trained emotion detection model according to the training result and the set addition result, the method further comprises: deploying pre-set initial weight and initial bias to the pre-constructed emotion detection model.

4. The voice emotion recognition method of claim 1, wherein, The step of optimizing the initial weight value and the initial bias value of the pre-constructed sentiment detection model by using the improved bat optimization algorithm to obtain an optimal sentiment detection model after optimization specifically comprises: Step a, obtaining a pre-introduced dynamic inertia factor, and improving the bat optimization algorithm according to the dynamic inertia factor to obtain an improved bat optimization algorithm; Step b, taking the initial weight value and the initial bias value as the starting position information of the bat individual; Step c, performing iterative updating according to the improved bat optimization algorithm and the starting position information of the bat individual to calculate the current position information of the bat individual; Step d, after each iterative updating, obtaining the weight value and the bias value corresponding to the current position information of the bat individual, and deploying the weight value and the bias value to the pre-constructed sentiment detection model for classification verification to obtain a classification verification result; Step e, according to the classification verification result, screening out the optimal weight value and the optimal bias value in the past iterative updating, and resetting the optimal weight value and the optimal bias value as the starting position information of the bat individual, and repeating steps c, d and e; Step f, until the iteration termination condition is met, the optimal weight value and the optimal bias value in the past iterative updating are screened out and set to the pre-constructed sentiment detection model to obtain the optimal sentiment detection model, wherein the iteration termination condition comprises that the optimal weight value and the optimal bias value no longer change or reach a preset maximum number of iterations.

5. The speech emotion recognition method of claim 4, wherein, The step of obtaining a pre-introduced dynamic inertia factor and improving the bat optimization algorithm according to the dynamic inertia factor to obtain an improved bat optimization algorithm specifically comprises: According to a preset first formula: V i (t+1) = σ t ·V i (t) + (X i (t) - X')·f i , and a preset second formula: σ t (t+1) = σ max - (σ max - σ min )·(1+t) -1 , an improved bat optimization algorithm is obtained, wherein V i (t+1) and V i (t) are search change speeds of a bat individual at t+1 and t moments respectively; X i (t) is a position of the i-th bat at t moment; X' is a current global optimal position; f i is a transmitted pulse frequency, which is a random number between 0 and 1; σ max and σ min are maximum and minimum values of a dynamic inertia factor, and σ t is a current dynamic inertia factor.

6. The voice emotion recognition method of claim 4, wherein, The step of obtaining the weight value and the bias value corresponding to the current position information of the bat individual and deploying the weight value and the bias value to the pre-constructed sentiment detection model for classification verification to obtain a classification verification result specifically comprises: The weight value and the bias value are set as the latest weight value and the latest bias value of the pre-constructed sentiment detection model; The step of inputting all the MFCC features into the pre-constructed sentiment detection model for sentiment detection classification to obtain a classification result specifically comprises: According to the classification result and a preset loss function, the back propagation error of the BP neural network is calculated, and the error value is obtained as the classification verification result; The step of screening out the optimal weight value and the optimal bias value in the past iterative updating according to the classification verification result and resetting the optimal weight value and the optimal bias value as the starting position information of the bat individual specifically comprises: Obtaining the error values obtained after the past iterative updating, and comparing all the error values; Screening out the weight value and the bias value corresponding to the minimum error value as the optimal weight value and the optimal bias value in the past iterative updating.

7. The speech emotion recognition method of any one of claims 1 to 6, characterized in that, The step of extracting the MFCC features corresponding to the patient voice information by using MATLAB specifically comprises: Pretreating the patient voice information to obtain a pretreatment result, wherein the pretreatment step comprises pre-emphasis, framing and windowing in sequence; Using a periodogram method to perform power spectrum estimation on the pretreatment result to obtain a target power spectrum; Filtering the target power spectrum according to a Mel filter bank, calculating energy in each filter of the Mel filter bank; Logarithmically operating the energy in each filter to obtain each logarithmically operated result; DCT converting the each logarithmically operated result, obtaining a Delta coefficient corresponding to a preset frame value from a DCT converted result as the MFCC feature.

8. A voice emotion recognition apparatus, characterized by, The voice emotion recognition device is used to implement the steps of the voice emotion recognition method in any one of claims 1 to 7, and the voice emotion recognition device comprises: An emotion classification label obtaining module is configured to obtain a pre-set emotion classification label; A training set classification module is configured to obtain MFCC features corresponding to different emotion classification labels from a pre-set MFCC feature training set with labeled classification labels, and add the MFCC features into different MFCC feature sets according to different emotion classification labels to obtain a set addition result; An emotion detection model training module is configured to input all MFCC features in the MFCC feature training set into a pre-built emotion detection model for model training, and obtain a trained emotion detection model according to a training result and the set addition result, wherein the pre-built emotion detection model is a classification prediction model formed by a BP neural network architecture; A patient voice information obtaining module is configured to obtain patient voice information generated in a service process for a patient in real time; An MFCC feature extraction module is configured to extract features of the patient voice information by using MATLAB to obtain MFCC features corresponding to the patient voice information; An emotion detection result obtaining module is configured to input the MFCC features into the trained emotion detection model to obtain an emotion detection result; An emotion detection result feedback module is configured to feed back the emotion detection result to a target customer service. 9.A computer device comprising a memory and a processor, wherein the memory stores computer readable instructions, and the processor implements the steps of the voice emotion recognition method in any one of claims 1 to 7 when executing the computer readable instructions.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer readable instructions, and the computer readable instructions are executed by the processor to implement the steps of the voice emotion recognition method in any one of claims 1 to 7.

Images