Method and apparatus for emotion recognition of speech information

Abstract

Embodiments of the present application provide a kind of sentiment recognition method and device of voice information.The method is to a few minority class voice feature set Random oversampling processing, and the minority class voice feature set after processing is merged with majority class voice feature set, obtains target voice feature set;According to inverse order selection algorithm, the voice feature in target voice feature set is selected, and the voice feature selected is constituted candidate feature set;The voice feature in candidate feature set is input into classifier, obtains the voice emotion category of voice information, and the embodiments of the present application avoid the decision rule that model learns from unbalanced data is too much inclined to majority class sample, still can select optimal candidate feature set by the correlation analysis between features, reduce the feature dimension of initial target voice feature set, improve the computing efficiency of model, and improve the detection performance of model.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and in particular relates to a method, apparatus, electronic device and readable storage medium for emotion recognition of voice information. Background Technology

[0002] Emotion recognition of voice information is an important research topic in human-computer interaction. For example, in the context of intelligent customer service, accurately recognizing the emotional expression of users' voices is an important factor in improving the service quality of intelligent customer service.

[0003] Currently, the main data source for speech emotion recognition research is standard format speech databases. These speech data enable models to learn the relationship between speech and emotion, thereby giving the models the ability to recognize speech emotions.

[0004] However, in practice, the above solutions often suffer from data imbalance and local optima, which can significantly interfere with the model's learning and affect the accuracy of emotion recognition. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and readable storage medium for emotion recognition of voice information, in order to solve the technical problem that data imbalance and local optima cause significant interference to model learning and affect the accuracy of emotion recognition in the prior art.

[0006] In a first aspect, the present invention provides a method for emotion recognition of voice information, the method comprising:

[0007] Acquire voice information and extract the voice features of the voice information;

[0008] All the aforementioned speech features are classified to obtain a minority class speech feature set and a majority class speech feature set;

[0009] Random oversampling is performed on the minority class speech feature set, and the processed minority class speech feature set is merged with the majority class speech feature set to obtain the target speech feature set;

[0010] According to the reverse order selection algorithm, the speech features in the target speech feature set are selected, and the selected speech features are used to form a candidate feature set.

[0011] The speech features in the candidate feature set are input into the classifier to obtain the speech emotion category of the speech information.

[0012] In a second aspect, the present invention provides an emotion recognition device for voice information, the device comprising:

[0013] The acquisition module is used to acquire voice information and extract the voice features of the voice information;

[0014] The classification module is used to classify all the speech features to obtain a minority class speech feature set and a majority class speech feature set;

[0015] The random sampling module is used to perform random oversampling on the minority class speech feature set and merge the processed minority class speech feature set with the majority class speech feature set to obtain the target speech feature set.

[0016] The reverse order selection module is used to select speech features in the target speech feature set according to the reverse order selection algorithm, and to form a candidate feature set by selecting the speech features.

[0017] The recognition module is used to input the speech features in the candidate feature set into the classifier to obtain the speech emotion category of the speech information.

[0018] Thirdly, the present invention provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the above-mentioned emotion recognition method for voice information when executing the program.

[0019] Fourthly, the present invention provides a readable storage medium that, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the aforementioned emotion recognition method for voice information.

[0020] In this embodiment of the invention, a random oversampling method is employed. By replicating minority class samples, the number of features in the minority class speech feature set is increased to the same level as that in the majority class speech feature set. This achieves data balance without information loss, preventing the decision rules learned by the model from imbalanced data from being overly biased towards the majority class samples. Furthermore, this embodiment of the invention uses an iterative approach to analyze the acoustic signal feature set. Through an inverse selection algorithm, features with the smallest contribution to the representational ability of the target speech feature set are deleted in each iteration. The remaining speech features then form a candidate feature set, which is input into the classifier for speech emotion category recognition. This approach, while ensuring the representational ability of the feature set, selects the optimal candidate feature set through correlation analysis between features, reducing the feature dimensionality of the initial target speech feature set, improving the computational efficiency of the model, and enhancing the model's detection performance. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the steps of an emotion recognition method for voice information provided in an embodiment of the present invention;

[0023] Figure 2 This is a flowchart illustrating the specific steps of an emotion recognition method for voice information provided in an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of a random oversampling process provided in an embodiment of the present invention;

[0025] Figure 4 This is a schematic diagram of the reverse order selection algorithm processing flow provided in an embodiment of the present invention;

[0026] Figure 5 This is a schematic diagram of the architecture of an extreme learning machine provided in an embodiment of the present invention;

[0027] Figure 6 This is a structural diagram of a voice information emotion recognition device provided in an embodiment of the present invention;

[0028] Figure 7 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Figure 1 This is a flowchart illustrating the steps of an emotion recognition method for voice information provided in an embodiment of the present invention, as follows: Figure 1 As shown, the method may include:

[0031] Step 101: Obtain voice information and extract the voice features of the voice information.

[0032] In this embodiment of the invention, the voice information can be an audio signal containing voice content collected by an electronic device. Furthermore, since the voice information has a high data dimension, it is difficult to process it directly. Therefore, this embodiment of the invention can extract the voice features of the voice information. The voice features can reduce the data dimension while retaining the acoustic meaning of the voice information, thereby reducing the difficulty of subsequent voice feature processing.

[0033] Among them, features are the characteristics or properties that distinguish one type of object from other types of objects, or a set of these characteristics and properties. Features are data that can be extracted through measurement or processing. The main purpose of feature extraction is dimensionality reduction, and its main idea is to project the original image samples into a low-dimensional feature space to obtain low-dimensional speech features that best reflect the essence of speech information or distinguish speech information.

[0034] Step 102: Classify all the speech features to obtain a minority class speech feature set and a majority class speech feature set.

[0035] In this embodiment of the invention, data imbalance during model learning is a serious factor affecting the accuracy of model recognition. Therefore, data imbalance between emotion categories can significantly impact the learning of speech emotion recognition models. Imbalanced emotional speech often exhibits characteristics such as multiple categories, small magnitude, and high degree of emotional confusion. Addressing these characteristics and resolving speech emotion data imbalance is a pressing issue for practical application of speech emotion recognition technology. Specifically, in real-world applications, the speech features extracted from speech information exhibit data imbalance. This imbalance arises because, after classifying speech features into minority and majority classes, the number of minority class features is significantly less than that of majority class features. Using imbalanced data for model training affects the algorithm's learning of decision rules during model fitting, potentially leading to minority class samples being overwhelmed by majority class samples. This is because machine learning algorithms typically learn potential rules reflected in sample data by continuously optimizing reward and cost functions during model training. The calculation results of these functions are strongly correlated with the number of training samples encountered during model training. This means that decision rules learned from imbalanced data are likely to favor majority class samples.

[0036] It is understandable that data imbalance in speech signals refers to a skewed class distribution in the speech signal dataset, meaning there is a significant difference in the sample size of different speech emotion categories. When the number of samples in one or more emotion categories differs significantly from the number of samples in other categories, the category with a relatively larger sample size is called the majority class, and the category with a relatively smaller sample size is called the minority class. Majority class speech features are also called negative samples (samples that do not belong to a particular category), and minority class speech features are also called positive samples (samples that belong to a particular category). This invention aims to solve this data imbalance problem. In this step, when distinguishing between minority and majority class speech features, the number of features can be used as a criterion. If the number of features in one category is less than the number of features in other categories, then that category of speech features can be considered a minority class feature; if the number of features in one category is greater than the number of features in other categories, then that category of speech features can be considered a majority class feature.

[0037] Step 103: Perform random oversampling on the minority class speech feature set, and merge the processed minority class speech feature set with the majority class speech feature set to obtain the target speech feature set.

[0038] In this embodiment of the invention, a random oversampling method can be used to address the data imbalance problem by randomly oversampling features from the minority class speech feature set. The principle of the random oversampling algorithm is to randomly sample the minority class speech feature set and merge the newly obtained features with the original minority class speech feature set. This increases the number of features in the merged minority class speech feature set to the same level as the majority class speech feature set. The merged minority class speech feature set and the majority class speech feature set can then jointly constitute the target speech feature set for subsequent processing. The random oversampling algorithm achieves data balance by replicating minority class samples, which does not result in any information loss.

[0039] Step 104: Select speech features from the target speech feature set using the reverse selection algorithm, and construct a candidate feature set from the selected speech features.

[0040] In this embodiment of the invention, after feature extraction and imbalance processing of the speech signal, the resulting target speech feature set may have problems such as excessively high data dimensionality or information redundancy among features. Using such a target speech feature set for model training can lead to the curse of dimensionality, i.e., the samples are sparse in a high-dimensional space, which increases the bias and variance of the model's prediction results, makes the model increasingly difficult to interpret, makes it prone to overfitting, and increases the computational cost of the model.

[0041] To address the aforementioned issues, embodiments of the present invention employ an iterative approach to analyze the acoustic signal feature set. Specifically, an inverse selection algorithm is used to remove features that contribute the least to the representational ability of the target speech feature set during each iteration, resulting in a candidate feature set composed of the remaining speech features. This approach ensures the representational ability of the feature set while selecting the optimal candidate feature set through correlation analysis between features, thereby reducing the feature dimension of the initial target speech feature set. This process fully considers the correlation between features, eliminates information redundancy, and thus improves the computational efficiency and detection performance of the model.

[0042] Step 105: Input the speech features in the candidate feature set into the classifier to obtain the speech emotion category of the speech information.

[0043] In this embodiment of the invention, the speech features in the candidate feature set are input into a classifier. The classifier can be an Extreme Learning Machine (ELM) model based on a single hidden layer feedforward neural network. Since the parameters input to its network model are randomly assigned, the ELM only needs one iteration to train the model. Therefore, it can achieve a faster learning speed than a backpropagation neural network and has better generalization ability, and is less prone to local optima. After the speech features in the candidate feature set of this embodiment of the invention are processed by the classifier, the speech emotion category of the speech information can be output. Since this embodiment of the invention overcomes the problem of local optima in related technologies, it improves the recognition accuracy of the speech emotion category of the model.

[0044] In summary, this invention employs a random oversampling method. By replicating minority class samples, the number of features in the minority class speech feature set is increased to the same level as the majority class speech feature set. This achieves data balance without information loss, preventing the model from learning decision rules that are overly biased towards the majority class samples from imbalanced data. Furthermore, this invention uses an iterative approach to analyze the acoustic signal feature set. Through an inverse selection algorithm, features with the lowest contribution to the representational ability of the target speech feature set are removed in each iteration. The remaining speech features then form a candidate feature set, which is input into the classifier for speech emotion category recognition. This approach, while ensuring the representational ability of the feature set, selects the optimal candidate feature set through correlation analysis between features, reducing the feature dimensionality of the initial target speech feature set, improving the computational efficiency of the model, and enhancing the model's detection performance.

[0045] Figure 2 This is a flowchart illustrating the specific steps of a voice information emotion recognition method provided in an embodiment of the present invention, as follows: Figure 2 As shown, the method may include:

[0046] Step 201: Preprocess the voice information to obtain a preprocessed signal.

[0047] In this embodiment of the invention, before extracting the speech features of speech information, the speech features can be preprocessed to obtain a preprocessed signal. The preprocessing aims to quantize the speech information into a standard signal, and on the basis of preserving the content of the speech information, eliminate impurities in the speech information and improve the convenience of subsequent processing of the speech information.

[0048] Optionally, step 201 may specifically include:

[0049] Sub-step 2011: Discretely sample the speech information according to a preset time interval to obtain discrete speech signals.

[0050] In this step, the purpose of discrete sampling is to discretize the continuous speech information s(t) through a preset time interval to obtain a discrete speech signal s(nT), where T is the preset time interval, and its value is generally greater than twice the highest frequency in the speech information. This discrete sampling method can ensure that the sampled signal information is relatively complete and has less distortion.

[0051] Sub-step 2012: Quantize the discrete speech signal to obtain a quantized speech signal.

[0052] In this step, the discrete speech signal obtained after sampling is only a discrete signal in time. In order to obtain a processable digital signal, this embodiment of the invention can perform quantization processing on the discrete speech signal to obtain a quantized speech signal in digital signal form. Quantization refers to the process in the field of digital signal processing of approximating the continuous values ​​(or a large number of possible discrete values) of a signal into a finite number (or a small number) of discrete values. Quantization is mainly used in the conversion from continuous signals to digital signals. A continuous signal becomes a discrete signal after sampling, and a discrete signal becomes a digital signal after quantization. The amplitude range divided by the quantization operation can be 8 bits or 16 bits.

[0053] Sub-step 2013: Input the quantized speech signal into a preset first-order high-pass filter to filter out low-frequency signals in the quantized speech signal and obtain a first filtered signal.

[0054] In this step, the quantized speech signal after quantization can be further pre-emphasized. This is because the high-frequency phase of the power spectrum affected by the operation of the motor will cause signal attenuation. Therefore, pre-emphasis is needed to enhance the attenuated power, thereby improving the signal fidelity.

[0055] Specifically, one feasible pre-emphasis method is to use a first-order high-pass filter to filter out the low-frequency components of the quantized speech signal, leaving only the high-frequency components. This is equivalent to high-frequency enhancement, which can be expressed by the following formula:

[0056] H(z) = 1 - μz -1

[0057] Where H represents a high-pass filter, z represents the complex frequency domain variable of the input quantized speech signal, and μ is the pre-emphasis coefficient, which typically ranges from 0.95 to 0.98.

[0058] Sub-step 2014: After performing frame segmentation processing on the first filtered signal, windowing processing is applied to the frame segmented signal to obtain the preprocessed signal.

[0059] In practical applications, while the sound signal changes continuously during human speech, the changes are relatively stable over a short period of time, which can be considered as minimal, a phenomenon known as local stationarity. Based on this concept, this invention can simulate the characteristics of human speech by processing the sound signal into continuous frames, each frame being stationary within its own frame. The length of a single frame obtained through framing can be 10-30 ms.

[0060] In this embodiment of the invention, windowing refers to performing short-time analysis on the framed signal after framing. Framing of the acoustic signal is achieved by moving a window of finite length, and the overlapping part of the previous frame and the next frame is called frame shift. Windowing can make the global signal more continuous and make the originally irregular acoustic signal exhibit characteristics similar to a periodic function.

[0061] In acoustic signal processing, one feasible windowing function is the Hamming window, whose expression is:

[0062]

[0063] Where φ(n) represents the Hamming window function and N is the frame length.

[0064] Step 202: Extract the speech features of the preprocessed signal.

[0065] By extracting language features from the preprocessed signal, the preprocessing process improves the ease of signal processing and filters out impurities in the signal, making feature extraction easier and resulting in more accurate representation of the extracted speech features.

[0066] Optionally, step 202 can be implemented by performing a fast Fourier transform on the preprocessed signal to obtain a transformed signal; inputting the transformed signal into a preset Mel filter bank to obtain a second filtered signal; and performing a discrete cosine transform on the second filtered signal to obtain the speech features.

[0067] In this embodiment of the invention, the first step in extracting the preprocessed signal is to perform a Fast Fourier Transform (FFT) on the preprocessed signal to obtain the transformed signal. The FFT allows the time-domain characteristics of the preprocessed signal to be converted into an energy distribution in the frequency domain for observation. The FFT generally requires discretizing the preprocessed signal; the Discrete Fourier Transform (DFT) can be calculated using the FFT, and its formula is:

[0068]

[0069] Where, x (n) , where N is the sampled sound sequence and X is the frame length. (k) A complex series of N points.

[0070] The transformed signal can then be input into a preset Mel filter bank to obtain a second filtered signal. The transfer function of the Mel filter bank is:

[0071]

[0072] Where f(m) is the center frequency and m is the number of Mel filters.

[0073] The energy spectrum is transmitted to the Mel filter, and after taking the logarithm, the following logarithmic spectrum is obtained. The logarithmic formula is:

[0074] Where S(m) is the logarithmic spectrum, H m (k) is the transfer function of the triangular filter.

[0075] In this embodiment of the invention, the function of the Mel filter bank is to convert the energy spectrum into Mel frequencies that are closer to human ear perception, thereby improving the expressibility of speech features.

[0076] Finally, the second filtered signal can be subjected to discrete cosine transform to obtain speech features. In this embodiment of the invention, the second filtered signal is transformed to the cepstral frequency domain, and the resulting Mel-frequency cepstral coefficient signal is as follows:

[0077]

[0078] Where c(n) represents the Mel-frequency cepstral coefficients.

[0079] Step 203: Classify all the speech features to obtain a minority class speech feature set and a majority class speech feature set.

[0080] For details, please refer to step 102 above; it will not be repeated here.

[0081] Step 204: Perform random oversampling on the minority class speech feature set, and merge the processed minority class speech feature set with the majority class speech feature set to obtain the target speech feature set.

[0082] For details, please refer to step 103 above; it will not be repeated here.

[0083] Optionally, the difference between the number of features in the minority class speech feature set after the random oversampling process and the number of features in the majority class speech feature set is less than or equal to a preset threshold.

[0084] In this embodiment of the invention, reference is made to Figure 3 The illustration shows a schematic diagram of a random oversampling processing flow according to an embodiment of the present invention. For a speech feature set (an imbalanced audio signal dataset) exhibiting data imbalance, it can be classified into a minority class speech feature set (minority samples) and a majority class speech feature set (majority samples). The majority class speech features are also called negative samples (samples that do not belong to a certain category), and the minority class speech features are also called positive samples (samples that belong to a certain category). If the number of speech features in one class is less than the number of speech features in other classes, then that class of speech features can be considered a minority class speech feature; if the number of speech features in one class is greater than the number of speech features in other classes, then that class of speech features can be considered a majority class speech feature.

[0085] Next, the minority class speech feature set (minority class samples) can be randomly oversampled to obtain new features. These new features are then merged with the original minority class speech feature set (minority class samples). The difference between the number of features in the new minority class speech feature set and the number of features in the majority class speech feature set is less than or equal to a preset threshold (this preset threshold can be set according to actual needs). This results in a new minority class speech feature set with almost the same number of features as the majority class speech feature set, thus solving the problem of imbalanced minority and majority class data. Finally, the new minority class speech feature set is merged with the majority class speech feature set to obtain the target speech feature set for subsequent processing (i.e., a balanced acoustic signal dataset).

[0086] Specifically, assuming the feature set before processing (imbalanced acoustic signal dataset) is S, and the new features obtained by randomly oversampling the minority class speech feature set (minority class samples) are E, then the final output target speech feature set S' is as follows:

[0087] |S′|=|S maj |+|S min |+|E|,

[0088] Among them, S maj and S min These represent the majority class samples and the minority class samples, respectively.

[0089] Step 205: In each iteration, the loss value of each speech feature in the target speech feature set is calculated using a preset evaluation function, and the speech feature with the smallest loss value is deleted from the target speech feature set; the loss value is used to characterize the magnitude of the loss to the target speech feature set after deleting the speech feature.

[0090] In this embodiment of the invention, after feature extraction and imbalanced data processing of the speech signal, the resulting target speech feature set may suffer from problems such as excessively high data dimensionality or information redundancy among features. To address these issues, this embodiment employs a reverse selection algorithm to select speech features from the target speech feature set, forming a candidate feature set. This approach, while maintaining the representational power of the feature set, selects the optimal candidate feature set through correlation analysis between features, reducing the feature dimensionality of the initial target speech feature set. This process fully considers the correlation between features, eliminates information redundancy, and thus improves model performance.

[0091] Specifically, embodiments of the present invention can employ an iterative approach to analyze and select features from the target speech feature set, fully considering the correlation between features to select the optimal feature subset. The reverse order selection algorithm is a greedy algorithm that sequentially removes certain features from the initial i-dimensional feature set, thereby achieving the target feature dimension in the processed feature set.

[0092] Reference Figure 4 The diagram illustrates a reverse selection algorithm processing flow according to an embodiment of the present invention. First, the reverse selection algorithm is a greedy algorithm, which removes certain features sequentially from the initial i-dimensional feature set, such that the feature dimension of the processed feature set reaches the target value j(j). <i)。

[0093] The reverse selection algorithm requires initializing the algorithm parameters Y0 = X, j = i, defining the expected minimization evaluation function F, and using the evaluation value calculated by this function as a benchmark to measure the difference in classification performance of the classifier on the feature set before and after the removal of a specific feature in each iteration. In this embodiment of the invention, the loss value of each feature can be calculated in each iteration based on the expected minimization evaluation function F, and the feature with the smallest loss in classification performance due to feature removal can be found in each iteration. This feature can also be called the feature with the smallest contribution to classification performance, and can be represented as follows:

[0094] y - ←arg max F(Y j -y k ),y k ∈Y j ,y k ∈X.

[0095] Then, in each iteration, the speech feature with the smallest loss value is removed from the target speech features. This process can be represented as:

[0096] Y j-1 ←Y j -y - j = j-1

[0097] Finally, after multiple iterations, it can be determined whether the dimension j of the remaining features in the target speech feature set reaches the target dimension N. If it does, the calculation process ends, and the remaining features in the target speech feature set are used to form a candidate feature set (the optimal feature subset). Otherwise, the iteration continues until the condition is met and a candidate feature set is obtained.

[0098] The idea behind feature selection methods based on the inverse order selection algorithm is to reduce the feature dimension of the initial target speech feature set by minimizing the model classification performance decay, thereby improving the model's computational efficiency and recognition performance.

[0099] Let X represent a target speech feature set with feature dimension i, and let the feature dimension of the feature subset during feature selection be j. Assume that the target dimension of the candidate feature set to be selected is N, and N ≤ j ≤ i. Let Y represent the selected subset of acoustic signal features. Then, feature sets X and Y can be represented as follows:

[0100] X = {x1, x2, ..., x} i},

[0101] Y j ={y k |k=1,2,…,j;y k ∈X},j=1,2,…,i.

[0102] Step 206: When the dimension of the target speech feature set reaches the target dimension, stop the iteration and select the remaining speech features in the target speech feature set to construct the candidate feature set.

[0103] In this embodiment of the invention, when the dimension of the target speech feature set reaches the target dimension, the iteration stops due to the termination condition. The remaining speech features in the target speech feature set are selected to construct a candidate feature set. By deleting the features that contribute the least to the representational ability of the target speech feature set in each iteration, the remaining speech features in the end constitute the candidate feature set. In this way, the optimal candidate feature set can be selected through correlation analysis between features while ensuring the representational ability of the feature set, thereby reducing the feature dimension of the initial target speech feature set.

[0104] Step 207: Input the speech features in the candidate feature set into the extreme learning machine classifier to obtain the speech emotion category of the speech information.

[0105] In this embodiment of the invention, reference is made to Figure 5 This illustration shows a schematic diagram of the architecture of an Extreme Learning Machine (ELM) according to an embodiment of the present invention. The ELM includes an input layer, hidden layers, and an output layer. The ELM is a single-hidden-layer feedforward neural network. Because its input parameters are randomly assigned, the ELM only requires one iteration to train the model. Therefore, it can achieve a faster learning speed than backpropagation neural networks and has better generalization ability, being less prone to local optima. Specifically, the goal of the ELM is to map a raw input space to an output space to handle classification or regression problems. The ELM proposes to randomly initialize network parameters, and the output weights are calculated using the least mean method based on the Moore-Penrose generalized inverse, thus reducing computational cost significantly compared to other classical algorithms.

[0106] Reference Figure 5 Unlike traditional single-hidden-layer feedforward neural networks, the Extreme Learning Machine (ELM) randomly assigns input weights and hidden layer biases, and does not require parameter adjustment as in backpropagation in neural networks. The output weights of the ELM network model are directly determined by solving a linear model; therefore, the training phase of the ELM is completed in a single iteration, resulting in extremely fast training speed. The connection between the input layer and hidden layers in the ELM is established through the input weights ω, and the connection between the hidden layers and the output layer is established through the output weights β.

[0107] Suppose we have a training dataset consisting of N arbitrarily distinct samples. Among individual samples include:

[0108] n features;

[0109] Label It includes m output categories. The output of a standard network containing L neurons can be represented as:

[0110]

[0111] in, b is the input weight of the i-th hidden layer neuron. i It is the bias of the i-th hidden layer neuron. It is the output weight of the i-th neuron, and the output value of the network is g() is the activation function. In Extreme Learning Machines, the sigmoid function is often used as the activation function.

[0112]

[0113] Standard loss function With network parameters ω, b, and β fully adjustable, the error can approach zero. In this case, the formula can be transformed into:

[0114]

[0115] Therefore, the above N formulas can be combined together to form a matrix Hβ=T.

[0116]

[0117]

[0118] Where: matrix H is the output of the hidden layer, and T is the true class label. The output weight β is calculated by solving a least squares problem.

[0119] The MP generalized inverse of the hidden layer output H, and the flow of the Extreme Learning Machine algorithm are as follows:

[0120] Input: Training dataset The activation function is g(), and the number of hidden neurons is L.

[0121] Output: Output weight β.

[0122] Step 1: Randomly generate input weights ω and hidden layer biases b with values ​​ranging from [-1, 1].

[0123] The second step is to calculate the hidden layer output matrix using the formula.

[0124] Step 3: Output weights using MP generalized inverse calculation.

[0125] Therefore, after the speech features in the candidate feature set of the present invention are processed by the extreme learning machine classifier, the speech emotion category of the speech information can be output. Since the present invention overcomes the problem of local optima in related technologies, it improves the recognition accuracy of the speech emotion category of the model.

[0126] In summary, this invention employs a random oversampling method. By replicating minority class samples, the number of features in the minority class speech feature set is increased to the same level as the majority class speech feature set. This achieves data balance without information loss, preventing the model from learning decision rules that are overly biased towards the majority class samples from imbalanced data. Furthermore, this invention uses an iterative approach to analyze the acoustic signal feature set. Through an inverse selection algorithm, features with the lowest contribution to the representational ability of the target speech feature set are removed in each iteration. The remaining speech features then form a candidate feature set, which is input into the classifier for speech emotion category recognition. This approach, while ensuring the representational ability of the feature set, selects the optimal candidate feature set through correlation analysis between features, reducing the feature dimensionality of the initial target speech feature set, improving the computational efficiency of the model, and enhancing the model's detection performance.

[0127] Figure 6 This is a structural diagram of a voice information emotion recognition device provided in an embodiment of the present invention. The device may include:

[0128] Acquisition module 301 is used to acquire voice information and extract the voice features of the voice information;

[0129] Classification module 302 is used to classify all the speech features to obtain a minority class speech feature set and a majority class speech feature set;

[0130] The random sampling module 303 is used to perform random oversampling on the minority class speech feature set and merge the processed minority class speech feature set with the majority class speech feature set to obtain the target speech feature set.

[0131] The reverse order selection module 304 is used to select speech features in the target speech feature set according to the reverse order selection algorithm, and to form a candidate feature set by selecting the speech features.

[0132] The recognition module 305 is used to input the speech features in the candidate feature set into the classifier to obtain the speech emotion category of the speech information.

[0133] Optionally, the difference between the number of features in the minority class speech feature set after the random oversampling process and the number of features in the majority class speech feature set is less than or equal to a preset threshold.

[0134] Optionally, the reverse order selection module 304 includes:

[0135] The iterative submodule is used to calculate the loss value of each speech feature in the target speech feature set in each iteration using a preset evaluation function, and to delete the speech feature with the smallest loss value from the target speech feature set; the loss value is used to characterize the magnitude of the loss to the target speech feature set after deleting the speech feature;

[0136] The termination submodule is used to stop the iteration when the dimension of the target speech feature set reaches the target dimension, and select the remaining speech features in the target speech feature set to construct the candidate feature set.

[0137] Optionally, the device further includes:

[0138] The preprocessing module is used to preprocess the voice information to obtain a preprocessed signal;

[0139] The acquisition module 301 includes:

[0140] An extraction submodule is used to extract the speech features of the preprocessed signal.

[0141] Optionally, the preprocessing module includes:

[0142] The discrete submodule is used to discretely sample the speech information according to a preset time interval to obtain discrete speech signals;

[0143] The quantization submodule is used to quantize the discrete speech signal to obtain a quantized speech signal;

[0144] The filtering submodule is used to input the quantized speech signal into a preset first-order high-pass filter to filter out low-frequency signals in the quantized speech signal and obtain a first filtered signal.

[0145] The frame segmentation and windowing module is used to perform frame segmentation processing on the first filtered signal, and then to perform windowing processing on the frame segmented signal to obtain the preprocessed signal.

[0146] Optionally, the acquisition module 301 includes:

[0147] The Fourier transform submodule is used to perform fast Fourier transform processing on the speech information to obtain the transformed signal;

[0148] The Mel filter submodule is used to input the transformed signal into a preset Mel filter bank to obtain a second filtered signal;

[0149] The cosine transform submodule is used to perform discrete cosine transform on the second filtered signal to obtain the speech features.

[0150] Optionally, the classifier is an extreme learning machine classifier.

[0151] In summary, this invention employs a random oversampling method. By replicating minority class samples, the number of features in the minority class speech feature set is increased to the same level as the majority class speech feature set. This achieves data balance without information loss, preventing the model from learning decision rules that are overly biased towards the majority class samples from imbalanced data. Furthermore, this invention uses an iterative approach to analyze the acoustic signal feature set. Through an inverse selection algorithm, features with the lowest contribution to the representational ability of the target speech feature set are removed in each iteration. The remaining speech features then form a candidate feature set, which is input into the classifier for speech emotion category recognition. This approach, while ensuring the representational ability of the feature set, selects the optimal candidate feature set through correlation analysis between features, reducing the feature dimensionality of the initial target speech feature set, improving the computational efficiency of the model, and enhancing the model's detection performance.

[0152] The present invention also provides an electronic device, see [link to relevant documentation]. Figure 7 The system includes: a processor 901, a memory 902, and a computer program 9021 stored in the memory and executable on the processor. When the processor executes the program, it implements the emotion recognition method for voice information in the foregoing embodiments.

[0153] The present invention also provides a readable storage medium that, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the emotion recognition method for voice information described in the foregoing embodiments.

[0154] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0155] It should be noted that all information and data obtained in the embodiments of the present invention were obtained with the authorization of the information / data holder.

[0156] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0157] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0158] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0159] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0160] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the sorting device according to the present invention. The present invention can also be implemented as a device or apparatus program for performing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0161] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0162] The user information (including but not limited to user device information, user personal information, etc.) and related data involved in this invention are all information authorized by the user or by the parties.

[0163] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0164] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0165] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for emotion recognition of voice information, characterized in that, The method includes: Acquire voice information and extract the voice features of the voice information; All the aforementioned speech features are classified to obtain a minority class speech feature set and a majority class speech feature set; Random oversampling is performed on the minority class speech feature set, and the processed minority class speech feature set is merged with the majority class speech feature set to obtain the target speech feature set; According to the reverse order selection algorithm, the speech features in the target speech feature set are selected, and the selected speech features are used to form a candidate feature set. The speech features in the candidate feature set are input into the classifier to obtain the speech emotion category of the speech information; The step of performing feature selection according to the reverse order selection algorithm on the speech features in the target speech feature set, and forming a candidate feature set from the selected speech features, includes: In each iteration, a loss value for each speech feature in the target speech feature set is calculated using a preset evaluation function, and the speech feature with the smallest loss value is removed from the target speech feature set; the loss value is used to characterize the magnitude of the loss to the target speech feature set after the speech feature is removed; When the dimension of the target speech feature set reaches the target dimension, the iteration stops, and the remaining speech features in the target speech feature set are selected to construct the candidate feature set. The reverse selection algorithm initializes the algorithm parameters. Define an expected minimization evaluation function F. The evaluation value calculated by this function F is used as a benchmark to measure the difference in classification performance of the classifier on the feature set before and after removing a specific feature in each iteration. Based on the expected minimization evaluation function F, calculate the loss value of each feature in each iteration, and find the feature that minimizes the classification performance loss due to feature removal in each iteration, as shown below: ； In each iteration, the speech feature with the smallest loss value is removed from the target speech feature set, denoted as: 。 2. The method according to claim 1, characterized in that, The difference between the number of features in the minority class speech feature set after the random oversampling process and the number of features in the majority class speech feature set is less than or equal to a preset threshold.

3. The method according to claim 1, characterized in that, Before extracting the speech features of the speech information, the method further includes: The voice information is preprocessed to obtain a preprocessed signal; The extraction of speech features from the speech information includes: Extract the speech features from the preprocessed signal.

4. The method according to claim 3, characterized in that, The preprocessing of the speech information to obtain a preprocessed signal includes: The speech information is discretely sampled according to a preset time interval to obtain discrete speech signals; The discrete speech signal is quantized to obtain a quantized speech signal; The quantized speech signal is input into a preset first-order high-pass filter to filter out low-frequency signals in the quantized speech signal, thereby obtaining a first filtered signal. After the first filtered signal is processed into frames, the framed signals are then windowed to obtain the preprocessed signal.

5. The method according to claim 1, characterized in that, The extraction of speech features from the speech information includes: The speech information is processed by Fast Fourier Transform to obtain the transformed signal; The transformed signal is input into a preset Mel filter bank to obtain a second filtered signal; The second filtered signal is subjected to discrete cosine transform to obtain the speech features.

6. The method according to any one of claims 1-5, characterized in that, The classifier is an Extreme Learning Machine classifier.

7. A voice information emotion recognition device, characterized in that, The device includes: The acquisition module is used to acquire voice information and extract the voice features of the voice information; The classification module is used to classify all the speech features to obtain a minority class speech feature set and a majority class speech feature set; The random sampling module is used to perform random oversampling on the minority class speech feature set and merge the processed minority class speech feature set with the majority class speech feature set to obtain the target speech feature set. The reverse order selection module is used to select speech features in the target speech feature set according to the reverse order selection algorithm, and to form a candidate feature set by selecting the speech features. The recognition module is used to input the speech features in the candidate feature set into the classifier to obtain the speech emotion category of the speech information; The reverse order selection module includes: The iterative submodule is used to calculate the loss value of each speech feature in the target speech feature set in each iteration using a preset evaluation function, and to delete the speech feature with the smallest loss value from the target speech feature set; the loss value is used to characterize the magnitude of the loss to the target speech feature set after deleting the speech feature; The termination submodule is used to stop the iteration when the dimension of the target speech feature set reaches the target dimension, and select the remaining speech features in the target speech feature set to construct the candidate feature set. The reverse selection algorithm initializes the algorithm parameters. Define an expected minimization evaluation function F. The evaluation value calculated by this function F is used as a benchmark to measure the difference in classification performance of the classifier on the feature set before and after removing a specific feature in each iteration. Based on the expected minimization evaluation function F, calculate the loss value of each feature in each iteration, and find the feature that minimizes the classification performance loss due to feature removal in each iteration, as shown below: ； In each iteration, the speech feature with the smallest loss value is removed from the target speech feature set, denoted as: 。 8. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as described in any one of claims 1-6.

9. A readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method described in any one of claims 1-6.

Images