Voice emotion recognition method, device and equipment and storage medium

Abstract

Embodiments of the present application disclose a speech emotion recognition method, device and equipment and a storage medium to solve the problem of low recognition accuracy of speech emotion in the prior art. The method comprises: obtaining speech data to be recognized; extracting multi-modal emotion feature data corresponding to the speech data (including at least one of speech emotion feature data of the speech data and text emotion feature data of text data corresponding to the speech data); determining an optimized parameter of a support vector machine classifier used for emotion classification; the optimized parameter is a model parameter obtained by optimizing initial model parameters of the support vector machine classifier using a cat swarm algorithm based on chaotic disturbance and / or Cauchy variation disturbance; and inputting the multi-modal emotion feature data and the optimized parameter into the support vector machine classifier to obtain an emotion category corresponding to the speech data. The technical solution can more accurately recognize the emotion category corresponding to the speech data, thereby improving the accuracy of speech emotion recognition.

Description

Technical Field

[0001] This invention relates to the field of speech processing technology, and in particular to a speech emotion recognition method, apparatus, device, and storage medium. Background Technology

[0002] Research shows that 80% of information in human communication is related to emotions. Voice emotion recognition, especially in customer service scenarios, can quickly and accurately detect changes in customer emotions, particularly negative emotions (such as disappointment, complaints, and anger), which is crucial for improving service quality and user experience for telecom operators.

[0003] In traditional speech emotion recognition technology, due to limitations in data transmission and storage methods, speech emotion information needs to be obtained from text. Therefore, the object of speech emotion recognition is generally text data. However, text data cannot fully summarize the characteristics of speech data, and the softmax classifier for emotion classification of text data has low emotion classification accuracy, resulting in low accuracy of speech emotion recognition in existing technologies. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for voice emotion recognition, in order to solve the problem of low accuracy in voice emotion recognition in the prior art.

[0005] To solve the above-mentioned technical problems, the embodiments of the present invention are implemented as follows:

[0006] In a first aspect, embodiments of the present invention provide a voice emotion recognition method, comprising:

[0007] Acquire the speech data to be recognized;

[0008] Extract multimodal sentiment feature data corresponding to the speech data; the multimodal sentiment feature data includes at least one of the following: speech sentiment feature data of the speech data, and text sentiment feature data of the text data corresponding to the speech data;

[0009] Determine the optimization parameters for the support vector machine classifier used for sentiment classification; the optimization parameters are the model parameters obtained by optimizing the initial model parameters of the support vector machine classifier using a cat swarm algorithm based on chaotic perturbation and / or Cauchy mutation perturbation;

[0010] The multimodal emotion feature data and the optimization parameters are input into the support vector machine classifier so that the support vector machine classifier can perform emotion recognition on the speech data and obtain the emotion category corresponding to the speech data.

[0011] Secondly, embodiments of the present invention also provide a voice emotion recognition device, comprising:

[0012] The acquisition module is used to acquire the speech data to be recognized;

[0013] An extraction module is used to extract multimodal emotional feature data corresponding to the speech data; the multimodal emotional feature data includes at least one of the following: speech emotional feature data of the speech data, and text emotional feature data of the text data corresponding to the speech data;

[0014] The first determining module is used to determine the optimization parameters of the support vector machine classifier used for emotion classification; the optimization parameters are model parameters obtained by optimizing the initial model parameters of the support vector machine classifier using a cat swarm algorithm based on chaotic perturbation and / or Cauchy mutation perturbation;

[0015] The first execution module is used to input the multimodal emotion feature data and the optimization parameters into the support vector machine classifier, so that the support vector machine classifier can perform emotion recognition on the speech data and obtain the emotion category corresponding to the speech data.

[0016] Thirdly, embodiments of the present invention also provide a voice emotion recognition device, including:

[0017] Memory, which stores computer program instructions;

[0018] A processor that, when the computer program instructions are executed by the processor, implements the speech emotion recognition method as described in the first aspect above.

[0019] Fourthly, embodiments of the present invention also provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the speech emotion recognition method as described in the first aspect above.

[0020] In this embodiment of the invention, by acquiring the speech data to be recognized, extracting the multimodal emotion feature data corresponding to the speech data, and determining the optimized parameters for the support vector machine (SVM) classifier used for emotion classification, the multimodal emotion feature data and the optimized parameters are input into the SVM classifier so that the SVM classifier can perform emotion recognition on the speech data to obtain the emotion category corresponding to the speech data. Since the multimodal emotion feature data corresponding to the speech data can more accurately express the emotion of the speech data compared to single-modal speech feature data or text feature data, the accuracy of speech emotion recognition can be improved. Furthermore, by optimizing the initial model parameters of the SVM classifier using a cat swarm algorithm based on chaotic perturbation and / or Cauchy mutation perturbation, the optimized parameters can be obtained. This avoids the problem of the cat swarm algorithm getting trapped in local optima, improves the global optimization capability and convergence speed of the cat swarm algorithm, and obtains more accurate optimized parameters, thereby improving the emotion classification accuracy of the SVM classifier. Based on this SVM classifier, emotion recognition of the input multimodal emotion feature data can more accurately identify the emotion category corresponding to the speech data. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic flowchart of a speech emotion recognition method in one embodiment of the present invention.

[0023] Figure 2 This is a schematic block diagram of a speech emotion recognition architecture in one embodiment of the present invention.

[0024] Figure 3 This is a distribution diagram of four emotions in IEMOCAP in one embodiment of the present invention.

[0025] Figure 4 This is a distribution diagram of three emotions in the operator's customer service hotline database in one embodiment of the present invention.

[0026] Figure 5 This is a schematic flowchart illustrating the determination of optimization parameters in one embodiment of the present invention.

[0027] Figure 6 This is a schematic flowchart of a voice emotion recognition method in another embodiment of the present invention.

[0028] Figure 7 This is a schematic diagram of the structure of a voice emotion recognition device in one embodiment of the present invention.

[0029] Figure 8 This is a schematic diagram of the structure of a voice emotion recognition device in one embodiment of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Figure 1 This is a schematic flowchart of a speech emotion recognition method in one embodiment of the present invention. Figure 1 The methods may include:

[0032] S102, acquire the speech data to be recognized.

[0033] In one embodiment, the voice data to be identified may be operator hotline voice data, and S102 may be executed as follows: obtain the operator hotline voice data to be identified from the operator side.

[0034] S104, Extract multimodal emotional feature data corresponding to the speech data.

[0035] The multimodal emotion feature data includes at least one of the following: speech emotion feature data of speech data and text emotion feature data of the corresponding text data. In one embodiment, the speech data can be converted into text data using an ASR (Automatic Speech Recognition) engine to obtain the text data corresponding to the speech data.

[0036] S106, Determine the optimization parameters of the support vector machine classifier used for sentiment classification; the optimization parameters are the model parameters obtained by optimizing the initial model parameters of the support vector machine classifier using a cat swarm algorithm based on chaotic perturbation and / or Cauchy mutation perturbation.

[0037] Among them, the Support Vector Machine (SVM) classifier is superior to the softmax classifier in binary classification problems. When classifying emotions in speech data, the SVM classifier does not perform a multi-classification task (e.g., happy, excited, peaceful, angry, resentful, dissatisfied) all at once. Instead, it first performs a binary classification of positive or negative emotions. When a negative emotion is identified, it then performs a secondary emotion classification (e.g., angry, resentful, dissatisfied). Therefore, by utilizing the high accuracy of the SVM classifier in binary classification problems, replacing the original softmax classifier can improve the accuracy of emotion classification.

[0038] S108, input the multimodal emotion feature data and optimization parameters into the support vector machine classifier so that the support vector machine classifier can perform emotion recognition on the speech data and obtain the emotion category corresponding to the speech data.

[0039] In this embodiment of the invention, by acquiring the speech data to be recognized, extracting the multimodal emotion feature data corresponding to the speech data, and determining the optimized parameters for the support vector machine (SVM) classifier used for emotion classification, the multimodal emotion feature data and the optimized parameters are input into the SVM classifier so that the SVM classifier can perform emotion recognition on the speech data to obtain the emotion category corresponding to the speech data. Since the multimodal emotion feature data corresponding to the speech data can more accurately express the emotion of the speech data compared to single-modal speech feature data or text feature data, the accuracy of speech emotion recognition can be improved. Furthermore, by optimizing the initial model parameters of the SVM classifier using a cat swarm algorithm based on chaotic perturbation and / or Cauchy mutation perturbation, the optimized parameters can be obtained. This avoids the problem of the cat swarm algorithm getting trapped in local optima, improves the global optimization capability and convergence speed of the cat swarm algorithm, and obtains more accurate optimized parameters, thereby improving the emotion classification accuracy of the SVM classifier. Based on this SVM classifier, emotion recognition of the input multimodal emotion feature data can more accurately identify the emotion category corresponding to the speech data.

[0040] Figure 2 This is a schematic block diagram of a speech emotion recognition architecture according to one embodiment of the present invention. The architecture may include a basic data layer 210, a data preprocessing layer 220, an emotion and semantic understanding layer 230, a capability output layer 240, and an application layer 250. Wherein:

[0041] The basic data layer 210 is used to acquire the speech data to be recognized and to store the acquired speech data.

[0042] Data preprocessing layer 220 is used to extract multimodal emotional feature data corresponding to speech data.

[0043] The emotion and semantic understanding layer 230 is used to determine the optimized parameters of the support vector machine classifier for emotion classification. The optimized parameters are the model parameters obtained by optimizing the initial model parameters of the support vector machine classifier using a cat swarm algorithm based on chaotic perturbation and / or Cauchy mutation perturbation. The multimodal emotion feature data and the optimized parameters are input into the support vector machine classifier so that the support vector machine classifier can perform emotion recognition on the speech data and obtain the emotion category corresponding to the speech data.

[0044] The capability output layer 240 is used to uniformly open up speech emotion recognition capabilities. For example, it can be opened up by embedding a URL (Uniform Resource Locator) in the page, so that speech emotion recognition can be performed on the speech data by clicking the URL; it can also provide a speech emotion recognition capability interface, so that speech emotion recognition can be performed on the speech data through the speech emotion recognition capability interface.

[0045] Application layer 250 is used to connect with front-end business systems and enable scenarios.

[0046] It should be noted that the speech emotion recognition method provided in this embodiment of the invention is mainly implemented by the basic data layer 210, the data preprocessing layer 220, and the emotion and semantic understanding layer 230. The content implemented by the capability output layer 240 and the application layer 250 is not specifically elaborated in this embodiment of the invention. The speech emotion recognition method provided in this embodiment of the invention (i.e., S102-S108 above) is described in detail below.

[0047] In one embodiment, when performing S104, multimodal emotional feature data corresponding to the speech data can be extracted according to the following steps A1-A3:

[0048] Step A1: Obtain the speech emotion feature data corresponding to the speech data; and obtain the text emotion feature data corresponding to the text data.

[0049] Among them, speech emotion feature data can be considered as shallow feature data of speech, which can be obtained through the spectral features of speech. In one embodiment, a 39-dimensional MFCC (Mel-Frequency Cepstral Coefficients) feature vector can be extracted from the speech data by pre-emphasis, framing, windowing, fast Fourier transform, Mel filter filtering, discrete cosine transform, and first- and second-order difference extraction, which serves as the shallow feature data of speech.

[0050] The text sentiment feature data can be the word vector feature data of the text. In one embodiment, the text data can be encoded into 300-dimensional word vectors using pre-trained word vectors provided by Chinese Word Vectors (a set of pre-trained Chinese word vectors), which can then be used as the word vector feature data of the text.

[0051] Step A2: Based on the attention mechanism, semantically align the speech emotion feature data and the text emotion feature data.

[0052] Step A3 involves fusing the semantically aligned speech sentiment feature data and text sentiment feature data to obtain the multimodal sentiment feature data corresponding to the speech data.

[0053] When performing steps A2-A3 above, an encoder-decoder model combined with an attention mechanism can be used to semantically align the speech and text sentiment feature data. The semantically aligned speech and text sentiment feature data are then fused to obtain the multimodal sentiment feature data corresponding to the speech data. The encoder-decoder model, also known as the code-decoder model, is a model applied to the seq2seq (Sequence-to-sequence) problem. Seq2seq is a general framework where the input and output are both sequences. The encoder-decoder model can handle problems where both input and output are sequences. Both the encoder and decoder in the encoder-decoder model employ BiLSTM (Bi-directional Long Short-Term Memory) models. BiLSTM is a bidirectional LSTM (Long Short-Term Memory) model, composed of a forward LSTM and a backward LSTM, which can better capture bidirectional semantic dependencies.

[0054] The encoding results (i.e., the sequence of speech emotion feature vectors and the sequence of text emotion feature vectors) can be calculated according to the following formula (1). t The voice emotion feature data or text emotion feature data input at time t. Let be the forward latent vector at time t. Let be the backward latent vector at time t. Let be the forward latent vector at time t-1. h is the backward latent vector corresponding to time t-1. t Let t be the speech emotion feature vector or text emotion feature vector corresponding to time t.

[0055]

[0056] After obtaining the sequences of speech emotion feature vectors and text emotion feature vectors respectively through the encoder, the alignment information between the speech emotion feature vectors and text emotion feature vectors can be automatically learned through an attention mechanism. Specifically, for the speech emotion feature vector s i and the text sentiment feature vector h j According to formula (2), by s i and h j Calculate s using dot product i and h j Matching coefficient α j,i Then, according to formula (3), the matching coefficient α j,i Normalization is performed to obtain the normalized matching degree coefficient α. j,i The normalized matching coefficient α j,i The attention matrix formed represents the speech emotion feature vector s i and the text sentiment feature vector h j The alignment relationship is applied to the speech emotion feature vector as a weighting coefficient. According to formula (4), the weighted speech emotion feature vector can be obtained.

[0057]

[0058]

[0059]

[0060] Then, by weighting the speech emotion feature vector and the text sentiment feature vector h output by the encoder j After concatenation, the data is input into the decoder, and the decoding result c corresponding to the multimodal sentiment feature data can be calculated according to the following formula (5). j .

[0061]

[0062] In this embodiment, by acquiring the speech emotion feature data corresponding to the speech data and the text emotion feature data corresponding to the text data, and based on the attention mechanism, the speech emotion feature data and the text emotion feature data are semantically aligned, achieving the effect of aligning features of different modalities in the same semantic space. Then, the semantically aligned speech emotion feature data and text emotion feature data are fused to obtain the multimodal emotion feature data corresponding to the speech data. Compared with single-modal speech feature data or text feature data, multimodal emotion feature data can more accurately express the emotion of the speech data. Therefore, applying the obtained multimodal emotion feature data to subsequent embodiments to perform speech emotion recognition on speech data can improve the accuracy of speech emotion recognition.

[0063] In one embodiment, the initial model parameters are the penalty parameters of the radial basis function (RBF) of the support vector machine classifier. The penalty parameters of the RBF kernel function include C and λ.

[0064] Introducing the RBF kernel function into the SVM classifier allows low-dimensional vectors to be introduced into a high-dimensional space. Specifically, the SVM classifier uses a linear model, introducing non-linear input vectors into a multi-dimensional space to generate non-linear class boundaries, thereby establishing a maximum margin hyperplane in the multi-dimensional space for accurate classification. Given a training sample set (x) for classification... i ,y i ), where y i ∈{-1,1}x i ∈R d i = 1, 2, 3, ..., n. Nonlinear mapping function. The input sample space is mapped to a high-dimensional space, and then a linear classification function is constructed in the high-dimensional space. The discriminant function is as follows (6), where sgn(x) is the sign function, ω is the weight vector, and b is the offset vector:

[0065]

[0066] According to the principle of minimizing structural risk, the optimal problem to be solved is shown in (7). C is the error penalty factor, which controls the degree of penalty for incorrect sample classification, and achieves a trade-off between the proportion of misclassified samples and the complexity of the algorithm. i For slack variables:

[0067]

[0068] For SVM classifiers, the RBF kernel function, as a non-linear function, can reduce the computational complexity of the training process. Therefore, RBF is used as the kernel function for SVM classifiers. The expression formula of the RBF kernel function is shown in (8):

[0069] K(x i ,x j )=exp(-λx i -x j || 2 ),λ>0. (8)

[0070] Here, C and λ are the core parameters that need to be precisely adjusted. C is the error penalty coefficient, which controls the degree of penalty for erroneous samples, while the kernel function parameter λ mainly affects the complexity of the sample data distribution in the space. The parameters C and λ determine the learning and generalization ability of the support vector machine. The optimized objective function achieves marginal maximization, at which point the support vector machine achieves the highest accuracy in sentiment classification of multimodal sentiment feature data.

[0071] The following details how the optimized parameters (i.e., the optimized C and λ) in S106 are obtained, that is, how to use the cat swarm algorithm based on chaotic perturbation and / or Cauchy mutation perturbation to optimize the initial model parameters (i.e., penalty parameters C and λ) of the support vector machine classifier to obtain the model parameters (i.e., optimized parameters).

[0072] The cat swarm algorithm is a swarm intelligence biomimetic algorithm proposed by observing the behavior of cats in their daily lives. The cats represent feasible solutions to the optimization problem (i.e., penalty parameters C and λ). Cats typically focus on hunting and capturing moving prey, even during rest periods, continuously searching for potential hunting opportunities; this behavior is called the "search mode." Once prey is locked on, the cat rapidly increases its speed and uses a significant amount of energy to capture the moving prey; this behavior is called the "tracking mode." The search mode improves global search capabilities, while the tracking mode improves search efficiency and accuracy. The cat swarm assigns modes to individuals based on a mixture ratio (MR), determining which cats are in search mode and which are in tracking mode. MR represents the proportion of cats in the entire swarm that are in tracking mode.

[0073] Among them, a linear mixing ratio (MR) can be used. L The pattern allocation of the cat group is determined by the expression shown in (9). t is the current iteration number; IT max The preset maximum number of iterations:

[0074]

[0075] To solve an optimization problem using the cat swarm algorithm, it is first necessary to determine the number of individuals participating in the optimization calculation, that is, the number of cats. It is also necessary to determine the attributes of each cat (including its own position composed of M dimensions), the velocity of each dimension, the fitness value for the benchmark function, and the identification value indicating whether the cat is in the search mode or the tracking mode. After the cats have completed the search mode and the tracking mode, calculate their fitness values according to the fitness function and retain the best solution in the current cat swarm. Then, according to the chaotic perturbation algorithm and / or the Cauchy mutation perturbation algorithm, traverse the optimal solution to the global space to improve the global optimization ability, and perform iterative calculations in this way until the preset number of iterations is reached. Among them, to solve the optimization parameters, M dimensions can be 2 dimensions, that is, the two dimensions of C and λ.

[0076] In one embodiment, before executing S102 to obtain the voice data to be recognized, the optimization parameters can be obtained according to the following steps B1 - B4:

[0077] Step B1, determine the cat swarm data for the current iteration. Among them, each cat corresponds to a set of penalty parameters.

[0078] In this embodiment, the operating parameters of the cat swarm algorithm can be initialized first: population size (i.e., the number of cats), maximum number of iterations, PMO (mutation probability), CDC (size change count), SMP (search memory pool), SRD (change domain). Secondly, determine that the solution space dimension is D dimensions (1 < d < D), and in the embodiment of the present invention, D is 2. Randomly initialize the positions of the cat swarm in the d-dimensional space, and represent the position of the i-th cat in the d-dimensional space as x id , and the position vectors of each cat include two dimensions of C and λ. And randomly initialize the velocity vectors of each cat, v id represents the velocity of the i-th cat in the d-dimensional space. Then, according to MR L Randomly assign each cat in the cat swarm to the queues of the search mode and the tracking mode, so as to update the position of each cat in the d-dimensional space according to the position update method corresponding to the search mode or the tracking mode.

[0079] Among them, the position update method corresponding to the search mode is specifically:

[0080] (1) Generate K copies of the population individuals based on SMP to form the initial population x i .

[0081] (2) Select one cat from the K copies of the population, keep the current position unchanged, and the remaining copies (K - 1) are mutated under the influence of orthogonal mutation; x i represents the initial population, and the updated individual generated by mutation is x i m , and the expression is as shown in (10):

[0082] xi m =x i +σ*C(0,1). (10)

[0083] Where C(0,1) is a random number between 0 and 1, and the mutation value of the current dimension is set as the standard deviation σ = SRD*x i .

[0084] (3) Evaluate the fitness values ​​of all copies, including the original positions in the K copies and the variant positions of the copies (K-1). Based on the optimal fitness value, obtain the best individual among the K copies and take it as the optimal solution.

[0085] (4) Replace the cat's current position with the optimal solution. If the fitness value of the mutated individual is better, then reset the position of the mutated individual to the initial group position x. i .

[0086] In this scenario, for cats in tracking mode, the position vector of each cat at the next moment depends on its velocity and the global optimal position of the cat group. The position and velocity of the i-th cat in d-dimensional space are xi and xi, respectively. i,d =(x i,1 ,x i,2 ,x i,3 ...,x i,D ), v i,d =(v i,1 ,v i,2 ,v i,3 ...,v i,D The global optimal solution is x. g,d =(x g,1 ,x g,2 ,x g,3 ...,x g,D The speed of the i-th cat can be calculated using formula (11):

[0087] v i,d =w*v i,d +c*r*(x g,d -x i,d (11)

[0088] Where w is the inertia weight value, c is the acceleration constant term, and r is a random number between [0,1]. The new position of the i-th cat after each movement can be represented by (12):

[0089] x i,d n =x i,d +v i,d (12)

[0090] If the cat's position in any dimension is outside the search range, set the cat's velocity vector as the boundary value and assign a value of -1 to start a reverse search to update the cat's position.

[0091] Step B2: Calculate the fitness value for each cat.

[0092] In one embodiment, the fitness value is calculated by MSE (Mean-Square Error), which measures the degree of deviation between the variable and the mean, as shown in expressions (13)-(14):

[0093]

[0094]

[0095] Where, f(x) i Let be the fitness value of the i-th cat. It is the average fitness value of all cats currently, and n is the population size of the cat population.

[0096] Step B3: Based on the fitness value of each cat, perform chaotic perturbation and / or Cauchy mutation perturbation on the cat population data to obtain the target cat population data.

[0097] Step B4: Determine whether the target cat group data meets the iteration termination condition; if yes, stop the iteration and determine the penalty parameter corresponding to the optimal fitness value in the target cat group data as the optimization parameter; if no, proceed to the next iteration on the target cat group data until the iteration termination condition is met.

[0098] Specifically, when the current iteration number is the preset maximum iteration number, it can be determined that the target cat group data meets the iteration termination condition.

[0099] In this embodiment, the fitness value of each cat in the current iteration (each cat corresponds to a set of penalty parameters) is calculated. Based on the fitness values ​​of each cat, chaotic perturbation and / or Cauchy mutation perturbation are applied to the cat group data to obtain the target cat group data. When the target cat group data meets the iteration termination condition, the iteration stops, and the penalty parameter corresponding to the optimal fitness value in the target cat group data is determined as the optimization parameter. This avoids the problem of the cat swarm algorithm getting trapped in local optima, improves the global optimization capability and convergence speed of the cat swarm algorithm, and obtains more accurate optimization parameters.

[0100] In one embodiment, step B3 can be specifically executed as steps C1-C3, that is, the cat group data is subjected to chaotic perturbation based on the fitness value of each cat to obtain the first cat group data.

[0101] Chaotic perturbations, characterized by randomness, ergodicity, and regularity, are widely used to solve optimization problems in various fields. The fundamental principle of chaotic perturbations is to map the solution space to a chaotic ergodic trajectory, approaching the optimal solution with arbitrary precision, while simultaneously enabling the perturbation process to avoid getting trapped in local optima. Typical chaotic systems are derived from the Logistic mapping equation.

[0102] The formula for calculating the solution of a chaotic system is shown in the following (15):

[0103] z i+1 =μz i (1-z i ), i = 1, 2, 3, ..., n. (15)

[0104] Where, μ∈(3.57,4), z i ∈(0,1), μ=4.

[0105] Step C1: Determine the optimal fitness value from multiple fitness values, and determine the penalty parameter corresponding to the optimal fitness value as the first optimal solution of the current iteration.

[0106] Step C2: Perform a chaotic transformation on the first optimal solution to obtain the second optimal solution after the chaotic perturbation.

[0107] Assume the first optimal solution is x i By performing a chaotic transformation on the first optimal solution according to the following formula (16), the second optimal solution z after the chaotic perturbation can be obtained. i The formula is:

[0108]

[0109] Where, x i，max Let x be the maximum position of the i-th cat. i，min Let z be the minimum position of the i-th cat. i It can be transformed into the first optimal solution x using the contravariant formula (17). i .

[0110] x i =z i (x i,max -x i,min )+x i,min (17)

[0111] Step C3: Determine the worst fitness value among the fitness values ​​corresponding to each cat, and replace the penalty parameter corresponding to the worst fitness value with the second optimal solution to obtain the first cat group data after chaotic perturbation.

[0112] In one embodiment, the first cat group data obtained after chaotic perturbation can be directly used as the target cat group data, and the penalty parameter corresponding to the optimal fitness value in the target cat group data can be determined as the optimization parameter.

[0113] In this embodiment, by performing a chaotic transformation on the global optimal solution (i.e., the first optimal solution) obtained in each iteration of the cat swarm algorithm—that is, by performing a chaotic perturbation search—the solution space of the decision variables can be traversed. The globally optimal solution after chaotic perturbation replaces the solution with the worst fitness value in the original solution set. This effectively avoids the problem of the cat swarm algorithm getting trapped in local optima, improving the global convergence accuracy of the cat swarm algorithm. Simultaneously, the chaotic perturbation process improves the quality of the cat swarm data, thereby accelerating the convergence speed of the cat swarm algorithm.

[0114] In one embodiment, after performing step C3 above, replacing the penalty parameter corresponding to the worst fitness value with the second optimal solution, and obtaining the first cat group data after chaotic perturbation, the degree of cat group clustering can be judged according to the following steps D1-D5:

[0115] Step D1: Calculate the average fitness value of each cat in the first cat group data after chaotic perturbation.

[0116] The average fitness value of each cat can be calculated according to the above formula (14).

[0117] Step D2: Calculate the cat clustering degree corresponding to the first cat group data based on the average fitness value and the number of cats.

[0118] The degree of cat clustering σ corresponding to the first cat group data can be calculated according to the following formula (18). 2 .

[0119]

[0120] f = max(1, max|f) i -f avg (18)

[0121] Among them, f i Let f be the fitness value of the i-th cat. avg This is the average fitness value of all cats currently in existence. n is the population size of the cat population. σ 2 The smaller the value, the higher the degree of clustering of the cats, indicating that they are close to convergence.

[0122] Step D3: Determine whether the clustering degree of the cat group corresponding to the first cat group data is less than the preset threshold; if yes, proceed to step D4; if no, proceed to step D5.

[0123] Based on existing literature, the preset threshold for the degree of cat group clustering is set at 0.04.2 When the value is less than 0.04, it indicates that the solution of the cat swarm algorithm has fallen into a local optimum, and it is necessary to apply Cauchy mutation perturbation to improve the global optimization capability.

[0124] Step D4: Sort the cats according to their fitness values ​​from high to low, select the top-ranked cats (a preset percentage), and apply Cauchy mutation perturbation to the penalty parameters corresponding to the preset percentage of cats to obtain local cat population data; replace the corresponding data in the first cat population data with the local cat population data to obtain the second cat population data after Cauchy mutation perturbation.

[0125] The preset ratio can be any ratio, such as 30%, 40%, etc. The formula for the Cauchy variation perturbation is:

[0126] x i =x i ×(1+0.3Cauchy(0,1)),

[0127] Cauchy(0,1)=tan((rand-0.5)×π). (19)

[0128] Step D5: Determine the first cat group data after chaotic perturbation as the target cat group data.

[0129] In this embodiment, the degree of cat clustering is determined to assess whether the solution of the cat swarm algorithm is trapped in a local optimum. After confirming that the solution is trapped in a local optimum (i.e., the degree of cat clustering is less than a preset threshold), Cauchy mutation perturbation is applied to the penalty parameters corresponding to a preset proportion of cats to obtain local cat data. This local cat data is then used to replace the corresponding data in the first cat data, resulting in the second cat data after Cauchy mutation perturbation. Since the Cauchy mutation operator can generate distributions with longer ends compared to Gaussian mutation, it is well-suited to ensuring that the cat data escapes local optima. Therefore, applying Cauchy mutation perturbation to the cat data with relatively good fitness values ​​at this point is beneficial for escaping local optima.

[0130] In one embodiment, after performing step D4 above, replacing the corresponding data in the first cat group data with local cat group data to obtain the second cat group data after Cauchy mutation perturbation, the degree of cat group clustering can be further determined according to the following steps E1-E4:

[0131] Step E1: Calculate the degree of cat clustering corresponding to the second cat group data.

[0132] The degree of cat clustering corresponding to the second cat group data can be calculated using the above formula (18).

[0133] Step E2: Determine whether the clustering degree of the cat group corresponding to the second cat group data is less than the preset threshold; if yes, proceed to step E3; if no, proceed to step E4.

[0134] Step E3: Apply Cauchy mutation perturbation to the second cat group data again until the cat group clustering degree corresponding to the third cat group data obtained after Cauchy mutation perturbation is less than the preset threshold.

[0135] Step E4: Determine the second cat group data as the target cat group data.

[0136] In this embodiment, by replacing the corresponding data in the first cat group data with local cat group data to obtain the second cat group data after Cauchy mutation perturbation, the degree of cat group clustering corresponding to the second cat group data is calculated. When the degree of cat group clustering corresponding to the second cat group data is less than a preset threshold, Cauchy mutation perturbation is performed on the second cat group data again until the degree of cat group clustering corresponding to the third cat group data obtained after Cauchy mutation perturbation is less than the preset threshold, which can ensure that the cat group data escapes the local optimum.

[0137] In one embodiment, after performing step D4 above, replacing the corresponding data in the first cat group data with local cat group data to obtain the second cat group data after Cauchy mutation perturbation, if the penalty parameter corresponding to at least one cat in the second cat group data exceeds the preset penalty parameter boundary value, then the penalty parameter corresponding to at least one cat is replaced with the penalty parameter boundary value.

[0138] In this embodiment, if at least one cat in the second cat group data has a penalty parameter that exceeds the preset penalty parameter boundary value, the penalty parameter corresponding to at least one cat is replaced with the penalty parameter boundary value, which can ensure that the cat group data obtained after Cauchy mutation perturbation does not go out of bounds.

[0139] According to the above embodiments, the optimized parameters C and λ of the support vector machine classifier used for emotion classification can be obtained. Thus, the multimodal emotion feature data and the optimized parameters are input into the support vector machine classifier, so that the support vector machine classifier can more accurately identify the emotion category corresponding to the speech data and improve the accuracy of speech emotion recognition.

[0140] In addition, the classification accuracy of the support vector machine classifier after using the optimized parameters of the embodiments of the present invention can be verified through the following procedures 1-5.

[0141] Process 1: Data Preparation.

[0142] Research on speech emotion recognition cannot be separated from the support of speech emotion databases. In this experiment, two representative speech emotion databases were selected: IEMOCAP (Interactive Emotional Dyadic Motion Capture database) and the operator customer service hotline database for experimental verification.

[0143] IEMOCAP contains approximately 12 hours of recordings, with 10 people divided into five groups. Each group's dialogue includes audio, text, and video information. Before recording, staff inform the actors of the topic of the conversation and the emotional tone to be used. The actors then record their audio and video according to the emotional theme. This experiment used four emotions as options for evaluation: hap, neu, sad, and ang. Figure 3 The distribution of the four emotions in IEMOCAP is shown. Excitement accounts for 13%, peace for 37%, sadness for 24%, and anger for 26%.

[0144] The operator's customer service hotline database consists of telephone recordings from the operator's customer service center. After anonymization to remove user information, the recordings were transcribed into nearly 6,000 audio files and corresponding text files. These audio and text files were then cross-annotated (different staff members annotated the same file), resulting in 5,904 annotated sentiment data points. Since only negative emotions in agent conversations are relevant in customer service applications, three sentiment types were used: positive emo, nuetral, and negative emo. The data distribution of these three sentiment types in the operator's customer service hotline database is as follows: Figure 4 As shown in the figure, positive emotions accounted for 4%, neutral emotions accounted for 50%, and negative emotions accounted for 46%.

[0145] Step 2: Data preprocessing.

[0146] Before extracting speech emotion features from the audio files, the audio files are first converted to standard 16k WAV files. Simultaneously, due to the short-term invariance of speech emotion features, to reduce computational load, each audio file is segmented into 20-millisecond short-time audio frames, retaining a 10-millisecond overlap. Then, the MFCC feature vector is extracted as the speech emotion feature data. For text features, since this experiment involves two speech emotion databases (English and Chinese), two word vector models are selected to determine the text emotion feature data. The Chinese word vector model used is sgns.target.word-word.dynwin5.thr10.neg5.dim300.iter5, and the English word vector model used is glove.42B.300d.txt.

[0147] Step 3: Align the speech emotion feature data with the text emotion feature data to obtain the multimodal emotion feature data corresponding to the speech file.

[0148] In the experiment, the encoders for both speech and text emotion feature data used a BiLSTM with 100 hidden units. The attention layer used a 5-layer attention network, with each layer containing 40 weight units. The fully connected layers had a 200*n weight matrix, where n is related to the final classification result. For gradient optimization, the Adam optimization algorithm was chosen with a learning rate of 0.001.

[0149] Step 4: Perform speech emotion classification on the multimodal emotion feature data, and evaluate the classification results of speech emotion classification based on the results of the above cross-labeling.

[0150] In the field of speech emotion recognition based on discrete models, the common problem of class imbalance means that simply using accuracy as an evaluation metric can lead to an overemphasis on classes with larger sample sizes while selectively filtering out classes with smaller sample sizes. Therefore, to achieve a balanced assessment of classification accuracy for each class, Unweighted Accuracy (UA) is used as the evaluation metric, calculated as follows:

[0151]

[0152] At the same time, Weighted Accuracy (WA) is used to evaluate the classification accuracy of the overall sample. The formula for its calculation is as follows:

[0153]

[0154] Step 5: Evaluation Results and Analysis.

[0155] In this experiment, single-modal (e.g., voice) emotional feature data and multi-modal emotional feature data were used for testing in IEMOCAP and operator customer service hotline databases. The evaluation results are shown in Table 1 and Table 2.

[0156] Table 1. Evaluation results of the speech emotion classification corresponding to IEMOCAP.

[0157] IEMOCAP UA WA Voice emotion feature data 0.52783 0.60615 Multimodal sentiment feature data 0.70237 0.71231

[0158] Table 2: Evaluation Results of Voice Emotion Classification Corresponding to the Operator Customer Service Hotline Database

[0159] Operator customer service hotline database UA WA Voice emotion feature data 0.48655 0.83384 Voice + Text Sentiment Feature Data 0.65945 0.84756 Multimodal sentiment feature data 0.66121 0.84862

[0160] As can be seen from Tables 1 and 2, the voice emotion recognition method proposed in this embodiment of the invention significantly improves both UA and WA in both IEMOCAP and operator customer service hotline databases. It is evident that by acquiring the voice data to be recognized, extracting the corresponding multimodal emotion feature data, and determining the optimized parameters for the support vector machine classifier used for emotion classification, the multimodal emotion feature data and optimized parameters are input into the support vector machine classifier. This enables the support vector machine classifier to more accurately identify the emotion category corresponding to the voice data, thus improving the accuracy of voice emotion recognition.

[0161] Figure 5 This is a schematic flowchart illustrating the determination of optimization parameters in one embodiment of the present invention. The optimization parameters are model parameters obtained by optimizing the initial model parameters of the support vector machine classifier using a cat swarm algorithm based on chaotic perturbation and / or Cauchy mutation perturbation. The initial model parameters are the penalty parameters of the radial basis kernel function of the support vector machine classifier. Figure 5 The methods may include:

[0162] S501, determine the cat group data for the current iteration, with each cat corresponding to a set of penalty parameters.

[0163] S502, calculate the fitness value for each cat.

[0164] S503, determine the optimal fitness value from multiple fitness values, and determine the penalty parameter corresponding to the optimal fitness value as the first optimal solution of the current iteration.

[0165] S504. Perform a chaotic transformation on the first optimal solution to obtain the second optimal solution after the chaotic perturbation.

[0166] S505, determine the worst fitness value among the fitness values ​​corresponding to each cat, and replace the penalty parameter corresponding to the worst fitness value with the second optimal solution to obtain the first cat group data after chaotic perturbation.

[0167] S506, for the first cat group data after chaotic perturbation, calculate the average fitness value of each cat. Based on the average fitness value and the number of cats, calculate the degree of cat group clustering corresponding to the first cat group data.

[0168] S507, determine whether the degree of cat clustering corresponding to the first cat group data is less than the preset threshold; if not, proceed to S508; if yes, proceed to S509.

[0169] S508, determine the first cat group data after chaotic perturbation as the target cat group data. Then, jump to S513.

[0170] S509, sort the cats according to their fitness values ​​from high to low, select the top-ranked cats by a preset proportion, apply Cauchy mutation perturbation to the penalty parameters corresponding to the preset proportion of cats to obtain local cat group data; replace the corresponding data in the first cat group data with the local cat group data to obtain the second cat group data after Cauchy mutation perturbation.

[0171] If at least one cat in the second cat group data has a penalty parameter that exceeds the preset penalty parameter boundary value, then the penalty parameter corresponding to at least one cat is replaced with the penalty parameter boundary value.

[0172] S510, calculate the degree of cat clustering corresponding to the second cat group data; determine whether the degree of cat clustering corresponding to the second cat group data is less than a preset threshold; if yes, execute S511; if no, execute S512.

[0173] S511, the second cat group data is subjected to Cauchy mutation perturbation again until the cat group clustering degree corresponding to the third cat group data obtained after Cauchy mutation perturbation is less than the preset threshold. Then, jump to S512.

[0174] S512, determine the second cat group data as the target cat group data.

[0175] S513, determine whether the target cat group data meets the iteration termination condition; if yes, execute S514; if no, jump to S502, until the iteration termination condition is met.

[0176] S514, stop the iteration, and determine the penalty parameter corresponding to the optimal fitness value in the target cat group data as the optimization parameter.

[0177] The specific processes of S501-S514 described above have been explained in detail in the above embodiments and will not be repeated here.

[0178] In this embodiment of the invention, the fitness value of each cat in the current iteration (each cat corresponds to a set of penalty parameters) is calculated. Based on the fitness values ​​of each cat, chaotic perturbation and / or Cauchy mutation perturbation are applied to the cat group data to obtain the target cat group data. When the target cat group data meets the iteration termination condition, the iteration stops and the penalty parameter corresponding to the optimal fitness value in the target cat group data is determined as the optimization parameter. This avoids the problem of the solution of the cat swarm algorithm getting trapped in local optima, improves the global optimization capability and convergence speed of the cat swarm algorithm, and obtains more accurate optimization parameters.

[0179] Figure 6 This is a schematic flowchart of a voice emotion recognition method in another embodiment of the present invention. Figure 6 The methods may include:

[0180] S601, acquire the speech data to be recognized.

[0181] S602, acquire speech emotion feature data corresponding to the speech data; and acquire text emotion feature data corresponding to the text data.

[0182] S603, based on an attention mechanism, performs semantic alignment between speech emotion feature data and text emotion feature data.

[0183] S604 fuses the semantically aligned speech sentiment feature data and text sentiment feature data to obtain multimodal sentiment feature data corresponding to the speech data.

[0184] S605, determine the optimal parameters for the support vector machine classifier used for sentiment classification.

[0185] The optimized parameters are the model parameters obtained by optimizing the initial model parameters of the support vector machine classifier using a cat swarm algorithm based on chaotic perturbations and / or Cauchy mutation perturbations. The initial model parameters are the penalty parameters of the radial basis kernel function of the support vector machine classifier.

[0186] S606 inputs multimodal emotion feature data and optimization parameters into a support vector machine classifier so that the support vector machine classifier can perform emotion recognition on the speech data and obtain the emotion category corresponding to the speech data.

[0187] The specific processes of S601-S606 described above have been explained in detail in the above embodiments and will not be repeated here.

[0188] In this embodiment of the invention, by acquiring the speech data to be recognized, extracting the multimodal emotion feature data corresponding to the speech data, and determining the optimized parameters for the support vector machine (SVM) classifier used for emotion classification, the multimodal emotion feature data and the optimized parameters are input into the SVM classifier so that the SVM classifier can perform emotion recognition on the speech data to obtain the emotion category corresponding to the speech data. Since the multimodal emotion feature data corresponding to the speech data can more accurately express the emotion of the speech data compared to single-modal speech feature data or text feature data, the accuracy of speech emotion recognition can be improved. Furthermore, by optimizing the initial model parameters of the SVM classifier using a cat swarm algorithm based on chaotic perturbation and / or Cauchy mutation perturbation, the optimized parameters can be obtained. This avoids the problem of the cat swarm algorithm getting trapped in local optima, improves the global optimization capability and convergence speed of the cat swarm algorithm, and obtains more accurate optimized parameters, thereby improving the emotion classification accuracy of the SVM classifier. Based on this SVM classifier, emotion recognition of the input multimodal emotion feature data can more accurately identify the emotion category corresponding to the speech data.

[0189] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0190] Figure 7 This is a schematic diagram of a voice emotion recognition device according to one embodiment of the present invention. Please refer to it. Figure 7 The voice emotion recognition device may include:

[0191] The acquisition module 710 is used to acquire the speech data to be recognized;

[0192] The extraction module 720 is used to extract multimodal sentiment feature data corresponding to the speech data; the multimodal sentiment feature data includes at least one of the following: speech sentiment feature data of the speech data, and text sentiment feature data of the text data corresponding to the speech data;

[0193] The first determining module 730 is used to determine the optimized parameters of the support vector machine classifier used for emotion classification; the optimized parameters are the model parameters obtained after optimizing the initial model parameters of the support vector machine classifier using a cat swarm algorithm based on chaotic perturbation and / or Cauchy mutation perturbation;

[0194] The first execution module 740 is used to input multimodal emotion feature data and optimization parameters into a support vector machine classifier, so that the support vector machine classifier can perform emotion recognition on the speech data and obtain the emotion category corresponding to the speech data.

[0195] In one embodiment, the initial model parameters are the penalty parameters of the radial basis kernel function of the support vector machine classifier;

[0196] The voice emotion recognition device also includes:

[0197] The second determining module is used to determine the cat group data for the current iteration; where each cat corresponds to a set of penalty parameters.

[0198] The calculation module is used to calculate the fitness value of each cat;

[0199] The second execution module is used to perform chaotic perturbation and / or Cauchy mutation perturbation on the cat group data based on the fitness value of each cat to obtain the target cat group data.

[0200] The judgment module is used to determine whether the target cat group data meets the iteration termination condition; if yes, the iteration stops and the penalty parameter corresponding to the optimal fitness value in the target cat group data is determined as the optimization parameter; if no, the target cat group data is iterated again until the iteration termination condition is met.

[0201] In one embodiment, the second execution module includes:

[0202] The first determining unit is used to determine the optimal fitness value from multiple fitness values, and to determine the penalty parameter corresponding to the optimal fitness value as the first optimal solution of the current iteration;

[0203] The chaotic transformation unit is used to perform a chaotic transformation on the first optimal solution to obtain the second optimal solution after the chaotic disturbance.

[0204] The determination and replacement unit is used to determine the worst fitness value among the fitness values ​​corresponding to each cat, and replace the penalty parameter corresponding to the worst fitness value with the second optimal solution to obtain the first cat group data after chaotic perturbation.

[0205] In one embodiment, the second execution module further includes:

[0206] The first computing unit is used to calculate the average fitness value of each cat for the first cat group data after chaotic perturbation.

[0207] The second calculation unit is used to calculate the degree of cat clustering corresponding to the first cat group data based on the average fitness value and the number of cats.

[0208] The first judgment unit is used to determine whether the degree of cat group clustering corresponding to the first cat group data is less than a preset threshold.

[0209] The execution unit is used to sort the cats according to their fitness values ​​from high to low if the condition is met, select the top-ranked cats by a preset proportion, perform Cauchy mutation perturbation on the penalty parameters corresponding to the preset proportion of cats to obtain local cat group data, and replace the corresponding data in the first cat group data with the local cat group data to obtain the second cat group data after Cauchy mutation perturbation.

[0210] The second determining unit is used to determine the first cat group data after chaotic perturbation as the target cat group data if no.

[0211] In one embodiment, the second execution module further includes:

[0212] The third calculation unit is used to calculate the degree of cat clustering corresponding to the second cat group data.

[0213] The second judgment unit is used to determine whether the degree of cat group clustering corresponding to the second cat group data is less than a preset threshold.

[0214] The Cauchy mutation perturbation unit is used to perform Cauchy mutation perturbation on the second cat group data again if the condition is met, until the degree of cat group clustering corresponding to the third cat group data obtained after Cauchy mutation perturbation is less than a preset threshold.

[0215] The third determining unit is used to determine the second cat group data as the target cat group data if no.

[0216] In one embodiment, the second execution module further includes:

[0217] The replacement unit is used to replace the penalty parameter corresponding to at least one cat with the penalty parameter boundary value if the penalty parameter corresponding to at least one cat in the second cat group data exceeds the preset penalty parameter boundary value.

[0218] In one embodiment, the extraction module 720 includes:

[0219] The acquisition unit is used to acquire speech emotion feature data corresponding to speech data; and to acquire text emotion feature data corresponding to text data.

[0220] The semantic alignment unit is used to semantically align speech sentiment feature data and text sentiment feature data based on an attention mechanism.

[0221] The fusion unit is used to fuse semantically aligned speech sentiment feature data and text sentiment feature data to obtain multimodal sentiment feature data corresponding to the speech data.

[0222] The speech emotion recognition device provided in this embodiment of the invention can realize each process of the speech emotion recognition method in the above method embodiment. To avoid repetition, it will not be described again here.

[0223] In this embodiment of the invention, by acquiring the speech data to be recognized, extracting the multimodal emotion feature data corresponding to the speech data, and determining the optimized parameters for the support vector machine (SVM) classifier used for emotion classification, the multimodal emotion feature data and the optimized parameters are input into the SVM classifier so that the SVM classifier can perform emotion recognition on the speech data to obtain the emotion category corresponding to the speech data. Since the multimodal emotion feature data corresponding to the speech data can more accurately express the emotion of the speech data compared to single-modal speech feature data or text feature data, the accuracy of speech emotion recognition can be improved. Furthermore, by optimizing the initial model parameters of the SVM classifier using a cat swarm algorithm based on chaotic perturbation and / or Cauchy mutation perturbation, the optimized parameters can be obtained. This avoids the problem of the cat swarm algorithm getting trapped in local optima, improves the global optimization capability and convergence speed of the cat swarm algorithm, and obtains more accurate optimized parameters, thereby improving the emotion classification accuracy of the SVM classifier. Based on this SVM classifier, emotion recognition of the input multimodal emotion feature data can more accurately identify the emotion category corresponding to the speech data.

[0224] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of the voice emotion recognition device used in the embodiments of the present invention. It can realize the details of the voice emotion recognition method executed by the voice emotion recognition device in the above embodiments and achieve the same effect. Figure 8 As shown, the voice emotion recognition device 800 includes: a processor 801, a transceiver 802, a memory 803, a user interface 804, and a bus interface, wherein:

[0225] In this embodiment of the invention, the voice emotion recognition device 800 further includes: a computer program stored on a memory 803 and executable on a processor 801, wherein the computer program, when executed by the processor 801, performs the following steps:

[0226] Acquire the speech data to be recognized;

[0227] Extract multimodal sentiment feature data corresponding to the speech data; the multimodal sentiment feature data includes at least one of the following: speech sentiment feature data of the speech data, and text sentiment feature data of the text data corresponding to the speech data;

[0228] Determine the optimal parameters for the support vector machine classifier used for sentiment classification; the optimal parameters are the model parameters obtained by optimizing the initial model parameters of the support vector machine classifier using a cat swarm algorithm based on chaotic perturbation and / or Cauchy mutation perturbation.

[0229] Multimodal emotion feature data and optimization parameters are input into a support vector machine classifier so that the support vector machine classifier can perform emotion recognition on the speech data and obtain the emotion category corresponding to the speech data.

[0230] exist Figure 8 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 801 and memory represented by memory 803 together. The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. Transceiver 802 can be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium. For different user equipment, the user interface 804 can also be an interface capable of connecting external or internal devices, including but not limited to keypads, displays, speakers, microphones, joysticks, etc.

[0231] The processor 801 is responsible for managing the bus architecture and general processing, while the memory 803 can store the data used by the processor 801 when performing operations.

[0232] In this embodiment of the invention, by acquiring the speech data to be recognized, extracting the multimodal emotion feature data corresponding to the speech data, and determining the optimized parameters for the support vector machine (SVM) classifier used for emotion classification, the multimodal emotion feature data and the optimized parameters are input into the SVM classifier so that the SVM classifier can perform emotion recognition on the speech data to obtain the emotion category corresponding to the speech data. Since the multimodal emotion feature data corresponding to the speech data can more accurately express the emotion of the speech data compared to single-modal speech feature data or text feature data, the accuracy of speech emotion recognition can be improved. Furthermore, by optimizing the initial model parameters of the SVM classifier using a cat swarm algorithm based on chaotic perturbation and / or Cauchy mutation perturbation, the optimized parameters can be obtained. This avoids the problem of the cat swarm algorithm getting trapped in local optima, improves the global optimization capability and convergence speed of the cat swarm algorithm, and obtains more accurate optimized parameters, thereby improving the emotion classification accuracy of the SVM classifier. Based on this SVM classifier, emotion recognition of the input multimodal emotion feature data can more accurately identify the emotion category corresponding to the speech data.

[0233] Preferably, the present invention also provides a voice emotion recognition device, including a processor 801, a memory 803, and a computer program stored in the memory 803 and executable on the processor 801. When the computer program is executed by the processor 801, it implements the various processes of the above-described voice emotion recognition method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0234] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described speech emotion recognition method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0235] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0236] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0237] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A voice emotion recognition method, characterized in that, include: Acquire the speech data to be recognized; Extract multimodal sentiment feature data corresponding to the speech data; the multimodal sentiment feature data includes at least one of the following: speech sentiment feature data of the speech data, and text sentiment feature data of the text data corresponding to the speech data; Determine the optimization parameters for the support vector machine classifier used for sentiment classification; the optimization parameters are the model parameters obtained by optimizing the initial model parameters of the support vector machine classifier using a cat swarm algorithm based on chaotic perturbation and Cauchy mutation perturbation. The multimodal emotion feature data and the optimization parameters are input into the support vector machine classifier so that the support vector machine classifier can perform emotion recognition on the speech data and obtain the emotion category corresponding to the speech data. Before acquiring the speech data to be recognized, the following steps are also included: Determine the cat population data for the current iteration; where each cat corresponds to a set of penalty parameters; Calculate the fitness value for each of the cats; Based on the fitness value of each cat, the cat population data is subjected to the chaotic perturbation and the Cauchy mutation perturbation to obtain the target cat population data, including: The optimal fitness value is determined from the plurality of fitness values, and the penalty parameter corresponding to the optimal fitness value is determined as the first optimal solution of the current iteration; The first optimal solution is subjected to a chaotic transformation to obtain the second optimal solution after the chaotic perturbation. The worst fitness value among the fitness values ​​corresponding to each cat is determined, and the penalty parameter corresponding to the worst fitness value is replaced with the second optimal solution to obtain the first cat group data after chaotic perturbation; For the first cat group data after chaotic perturbation, calculate the average fitness value of each cat; Based on the average fitness value and the number of cats, calculate the degree of cat clustering corresponding to the first cat group data; Determine whether the clustering degree of the cat group corresponding to the first cat group data is less than a preset threshold; If so, the cats are sorted in descending order of fitness value, and a preset proportion of cats are selected from the top of the sorted cats. The penalty parameter corresponding to the preset proportion of cats is perturbed by Cauchy mutation to obtain local cat group data. The local cat group data is used to replace the corresponding data in the first cat group data to obtain the second cat group data after Cauchy mutation perturbation. Calculate the degree of cat clustering corresponding to the second cat group data; Determine whether the clustering degree of the cat group corresponding to the second cat group data is less than the preset threshold; If so, the second cat group data is subjected to Cauchy mutation perturbation again until the degree of cat group clustering corresponding to the third cat group data obtained after Cauchy mutation perturbation is less than the preset threshold. If not, then the second cat group data is determined to be the target cat group data; If the degree of cat clustering corresponding to the first cat group data is not less than a preset threshold, then the first cat group data after chaotic perturbation is determined to be the target cat group data.

2. The method according to claim 1, characterized in that, The initial model parameters are the penalty parameters of the radial basis kernel function of the support vector machine classifier; After obtaining the target cat group data, the process also includes: Determine whether the target cat group data meets the iteration termination condition; If yes, then stop the iteration and determine the penalty parameter corresponding to the optimal fitness value in the target cat group data as the optimization parameter; if no, then perform the next iteration on the target cat group data until the iteration termination condition is met.

3. The method according to claim 1, characterized in that, After replacing the corresponding data in the first cat group data with the local cat group data to obtain the second cat group data after Cauchy mutation perturbation, the method further includes: If at least one cat in the second cat group data has a penalty parameter that exceeds a preset penalty parameter boundary value, then the penalty parameter corresponding to the at least one cat is replaced with the penalty parameter boundary value.

4. The method according to claim 1, characterized in that, The extraction of multimodal emotion feature data corresponding to the speech data includes: Acquire the voice emotion feature data corresponding to the voice data; and acquire the text emotion feature data corresponding to the text data; Based on the attention mechanism, the speech emotion feature data and the text emotion feature data are semantically aligned; The semantically aligned speech emotion feature data and the text emotion feature data are fused to obtain the multimodal emotion feature data corresponding to the speech data.

5. A voice emotion recognition device, characterized in that, include: The acquisition module is used to acquire the speech data to be recognized; An extraction module is used to extract multimodal emotional feature data corresponding to the speech data; the multimodal emotional feature data includes at least one of the following: speech emotional feature data of the speech data, and text emotional feature data of the text data corresponding to the speech data; The first determining module is used to determine the optimization parameters of the support vector machine classifier used for sentiment classification; the optimization parameters are model parameters obtained by optimizing the initial model parameters of the support vector machine classifier using a cat swarm algorithm based on chaotic perturbation and Cauchy mutation perturbation. The first execution module is used to input the multimodal emotion feature data and the optimization parameters into the support vector machine classifier, so that the support vector machine classifier can perform emotion recognition on the speech data and obtain the emotion category corresponding to the speech data; The second determining module is used to determine the cat group data for the current iteration; where each cat corresponds to a set of penalty parameters. The calculation module is used to calculate the fitness value of each cat. The second execution module is used to perform the chaotic perturbation and the Cauchy mutation perturbation on the cat group data according to the fitness value of each cat to obtain the target cat group data. The second execution module includes: The first determining unit is configured to determine the optimal fitness value from a plurality of fitness values, and to determine the penalty parameter corresponding to the optimal fitness value as the first optimal solution of the current iteration; A chaotic transformation unit is used to perform a chaotic transformation on the first optimal solution to obtain a second optimal solution after chaotic perturbation. The determination and replacement unit is used to determine the worst fitness value among the fitness values ​​corresponding to each cat, and replace the penalty parameter corresponding to the worst fitness value with the second optimal solution to obtain the first cat group data after chaotic perturbation; The first computing unit is used to calculate the average fitness value of each cat for the first cat group data after chaotic perturbation. The second calculation unit is used to calculate the degree of cat clustering corresponding to the first cat group data based on the average fitness value and the number of cats. The first judgment unit is used to determine whether the degree of cat group clustering corresponding to the first cat group data is less than a preset threshold. An execution unit is configured to, if so, sort the cats according to the fitness value from high to low, select a preset proportion of cats that are ranked first, perform Cauchy mutation perturbation on the penalty parameter corresponding to the preset proportion of cats to obtain local cat group data; and replace the corresponding data in the first cat group data with the local cat group data to obtain the second cat group data after Cauchy mutation perturbation. The third calculation unit is used to calculate the degree of cat clustering corresponding to the second cat group data. The second judgment unit is used to determine whether the degree of cat group clustering corresponding to the second cat group data is less than the preset threshold. The Cauchy mutation perturbation unit is used to perform Cauchy mutation perturbation on the second cat group data again if the condition is met, until the degree of cat group clustering corresponding to the third cat group data obtained after Cauchy mutation perturbation is less than the preset threshold. The third determining unit is used to determine the second cat group data as the target cat group data if no; The second determining unit is used to determine the first cat group data after chaotic perturbation as the target cat group data if the degree of cat group clustering corresponding to the first cat group data is not less than a preset threshold.

6. A voice emotion recognition device, characterized in that, include: Memory, which stores computer program instructions; A processor that, when the computer program instructions are executed by the processor, implements the speech emotion recognition method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on a computer, cause the computer to perform the speech emotion recognition method as described in any one of claims 1 to 4.

Images