Voice emotion recognition method, voice emotion recognition model training method and device

Abstract

This disclosure provides a speech emotion recognition method, a training method for a speech emotion recognition model, and an apparatus, which can be applied to the fields of speech recognition technology and finance. The speech emotion recognition method includes: extracting features from the speech data to be recognized to obtain a speech emotion feature vector sequence, wherein the speech emotion feature vector sequence includes at least one speech emotion feature vector, which is used to characterize speech intensity features and time increment features; and inputting the speech emotion feature vector sequence into a speech emotion recognition model to output a speech emotion recognition result, wherein the speech emotion recognition model includes an extreme learning machine neural network.

Description

Technical Field

[0001] This disclosure relates to the fields of speech recognition technology and finance, and more specifically, to a speech emotion recognition method, a training method and apparatus for a speech emotion recognition model, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] Speech is the external form of language, and language is the primary way people express themselves and the medium of communication in daily life. Therefore, the ability to correctly recognize speech and respond appropriately is fundamental to social interaction. Currently, the common approach is to convert speech into text and then use text analysis techniques to recognize the speech.

[0003] In realizing the concept disclosed herein, the inventors discovered at least the following problems in the related technologies: timely speech recognition cannot be guaranteed and recognition efficiency is low. Summary of the Invention

[0004] In view of this, the present disclosure provides a speech emotion recognition method, a training method and apparatus for a speech emotion recognition model, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to one aspect of this disclosure, a speech emotion recognition method is provided, comprising:

[0006] Feature extraction is performed on the speech data to be recognized to obtain a speech emotion feature vector sequence, wherein the speech emotion feature vector sequence includes at least one speech emotion feature vector, which is used to characterize speech intensity features and time increment features; and

[0007] The speech emotion feature vector sequence is input into the speech emotion recognition model, and the speech emotion recognition result is output. The speech emotion recognition model includes an extreme learning machine neural network.

[0008] According to embodiments of this disclosure, the speech intensity features include pitch periodic features and harmonic noise ratio features, and the time increment features include Mel frequency cepstral coefficient features.

[0009] Feature extraction is performed on the speech data to be recognized, resulting in a sequence of speech emotion feature vectors, including:

[0010] Feature extraction is performed on the speech data to be recognized to obtain a speech intensity feature sequence; and

[0011] Feature extraction is performed on the speech intensity feature sequence to obtain the speech emotion feature vector sequence.

[0012] According to embodiments of this disclosure, the above method further includes, before performing feature extraction on the speech data to be identified to obtain a sequence of emotion feature vectors for the speech data:

[0013] Acquire the speech data to be recognized;

[0014] The speech data to be recognized is segmented to obtain multiple raw speech segments corresponding to the speech data to be recognized; and

[0015] Each of the original speech segments in the multiple original speech segment data is filtered to obtain multiple filtered speech segment data.

[0016] According to embodiments of this disclosure, feature extraction of the speech data to be recognized to obtain a speech emotion feature vector sequence includes:

[0017] Multiple speech segments corresponding to the speech data to be recognized are processed to obtain speech emotion feature vectors for each speech segment; and

[0018] Based on the speech emotion feature vector and the position code, the speech emotion feature vector sequence is determined, where the position code is used to characterize the position information of the speech segment data in the speech data to be recognized;

[0019] The speech emotion feature vector sequence is input into the speech emotion recognition model, and the output speech emotion recognition results include:

[0020] The speech emotion feature vectors of multiple speech segments are processed to obtain the speech emotion information of each speech segment; and

[0021] The voice emotion recognition result is determined based on the voice emotion information of each of the multiple voice segments.

[0022] According to embodiments of this disclosure, the voice emotion information includes at least one voice emotion category and a probability value corresponding to each of the at least one voice emotion category;

[0023] Based on the individual emotional information of multiple speech segments, the speech emotion recognition results are determined as follows:

[0024] Based on the probability values, determine the target speech emotion category and target probability value for each speech segment in multiple speech segment data sets; and

[0025] The speech emotion recognition result is determined based on the target speech emotion category, target probability value, and location encoding of the speech segment data.

[0026] According to another aspect of this disclosure, a method for training a speech emotion recognition model is provided, comprising:

[0027] Feature extraction is performed on the sample speech data to obtain a sequence of sample speech emotion feature vectors. The sequence of sample speech emotion feature vectors includes at least one sample speech emotion feature vector, which is used to characterize the sample speech intensity feature and the sample time increment feature.

[0028] The sequence of sample speech emotion feature vectors is input into a speech emotion recognition model to be trained, and the output is the sample speech emotion recognition result. The speech emotion recognition model to be trained includes an Extreme Learning Machine neural network; and

[0029] A speech emotion recognition model is trained based on the results of sample speech emotion recognition and the results of real sample speech emotion recognition.

[0030] According to embodiments of this disclosure, the voice emotion recognition model includes an input layer, a hidden layer, and an output layer, wherein the hidden layer includes a preset number of nodes;

[0031] The sample speech emotion feature vector sequence is input into the speech emotion recognition model to be trained, and the output sample speech emotion recognition results include:

[0032] Based on a preset number, set a first weight value and a bias value for the hidden layer, wherein the first weight value is used to characterize the weight between the input layer and the hidden layer.

[0033] According to embodiments of this disclosure, training a speech emotion recognition model based on sample speech emotion recognition results and real sample speech emotion results includes:

[0034] The output matrix of the hidden layer is determined based on the sample speech emotion recognition results, the first weight value, and the bias value of the hidden layer.

[0035] Perform a generalized inverse operation on the output matrix of the hidden layer to obtain the output result of the hidden layer;

[0036] Based on the output of the hidden layer and the speech emotion results of real samples, a second weight value is determined, wherein the second weight value is used to represent the weight between the hidden layer and the output layer; and a speech emotion recognition model is trained according to a preset number, a first weight value, a bias value and a second weight value.

[0037] According to another aspect of this disclosure, a voice emotion recognition device is provided, comprising:

[0038] The first processing module is used to extract features from the speech data to be recognized, obtaining a speech emotion feature vector sequence, wherein the speech emotion feature vector sequence includes at least one speech emotion feature vector, which is used to characterize speech intensity features and time increment features; and

[0039] The first recognition module is used to input the speech emotion feature vector sequence into the speech emotion recognition model and output the speech emotion recognition result. The speech emotion recognition model includes an extreme learning machine neural network.

[0040] According to another aspect of this disclosure, a training apparatus for a speech emotion recognition model is provided, comprising:

[0041] The second processing module is used to extract features from the sample speech data to obtain a sequence of sample speech emotion feature vectors. The sequence of sample speech emotion feature vectors includes at least one sample speech emotion feature vector, which is used to characterize the sample speech intensity feature and the sample time increment feature.

[0042] The second recognition module is used to input the sequence of emotion feature vectors from sample speech into the speech emotion recognition model to be trained, and output the emotion recognition result of the sample speech. The speech emotion recognition model to be trained includes an extreme learning machine neural network.

[0043] The training module is used to train a speech emotion recognition model based on the speech emotion recognition results of sample speech and the speech emotion recognition results of real samples.

[0044] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0045] One or more processors;

[0046] Memory, used to store one or more instructions.

[0047] When one or more of the above instructions are executed by one or more processors, the one or more processors implement the method described above.

[0048] According to another aspect of this disclosure, a computer-readable storage medium is provided having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method described above.

[0049] According to another aspect of this disclosure, a computer program product is provided, which includes computer-executable instructions that, when executed, are used to implement the method described above.

[0050] According to embodiments of this disclosure, by inputting a sequence of speech emotion feature vectors obtained from feature extraction of the speech data to be recognized into a speech emotion recognition model including an extreme learning machine neural network, a speech emotion recognition result can be output based on the speech intensity features and time increment features of the speech data to be recognized. Through the above technical means, the technical problems of not being able to guarantee timely speech recognition and low recognition efficiency in related technologies are at least partially overcome. This achieves timely processing of the speech data to be recognized using a speech emotion recognition model, improving the efficiency and accuracy of speech emotion recognition. Attached Figure Description

[0051] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0052] Figure 1 The diagram illustrates a system architecture diagram of a speech emotion recognition method and a training method for a speech emotion recognition model that can be applied according to embodiments of the present disclosure.

[0053] Figure 2 A flowchart illustrating a speech emotion recognition method according to an embodiment of the present disclosure is shown schematically.

[0054] Figure 3 An example schematic diagram of a speech emotion recognition process according to an embodiment of the present disclosure is shown;

[0055] Figure 4 A flowchart illustrating a training method for a speech emotion recognition model according to an embodiment of the present disclosure is shown schematically.

[0056] Figure 5 The illustration shows an example diagram of the training process of a speech emotion recognition model according to an embodiment of the present disclosure;

[0057] Figure 6 A schematic block diagram of a voice emotion recognition device according to an embodiment of the present disclosure is shown.

[0058] Figure 7 A schematic block diagram of a training apparatus for a speech emotion recognition model according to an embodiment of the present disclosure is shown; and

[0059] Figure 8 A block diagram of an electronic device suitable for implementing a speech emotion recognition method and a training method for a speech emotion recognition model, according to embodiments of the present disclosure, is shown schematically. Detailed Implementation

[0060] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0061] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0062] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0063] When using expressions such as "at least one of A, B, and C," the expression should generally be interpreted in accordance with the meaning commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, systems having A alone, having B alone, having C alone, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.). Similarly, when using expressions such as "at least one of A, B, or C," the expression should generally be interpreted in accordance with the meaning commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, or C" should include, but is not limited to, systems having A alone, having B alone, having C alone, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0064] In the technical solution disclosed herein, the acquisition, storage, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0065] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0066] Speech is the external form of language, and language is the main way people express themselves and the medium of communication in daily life. Therefore, being able to correctly identify speech and respond appropriately is the foundation of social interaction.

[0067] With the development of artificial intelligence technology, it has been widely applied to simple scenarios such as facial recognition and optical character recognition, as well as integrated products such as smart homes, security and control, and autonomous driving. Current speech recognition methods typically use artificial intelligence technology to convert speech into text, and then use text analysis technology to recognize the speech.

[0068] However, besides the information obtained from semantic recognition of speech, emotional expression is also an important component of speech. Compared to semantic information, emotional expression has a certain degree of predictability and can be derived from a relatively small amount of speech.

[0069] In realizing the concept disclosed herein, the inventors discovered at least the following problems in the related technologies: timely speech recognition cannot be guaranteed and recognition efficiency is low.

[0070] To at least partially address the technical problems existing in related technologies, this disclosure provides a speech emotion recognition method, a training method and apparatus for a speech emotion recognition model, which can be applied to the fields of speech recognition technology and finance. The speech emotion recognition method includes: extracting features from the speech data to be recognized to obtain a speech emotion feature vector sequence, wherein the speech emotion feature vector sequence includes at least one speech emotion feature vector, which is used to characterize speech intensity features and time increment features; and inputting the speech emotion feature vector sequence into a speech emotion recognition model to output a speech emotion recognition result, wherein the speech emotion recognition model includes an extreme learning machine neural network.

[0071] It should be noted that the speech emotion recognition method, speech emotion recognition model training method, and apparatus provided in this disclosure can be used in the fields of speech recognition technology and finance, for example, they can be applied to bank branches to perform speech emotion recognition on users. The speech emotion recognition method, speech emotion recognition model training method, and apparatus provided in this disclosure can also be used in any field other than speech recognition technology and finance, for example, they can be applied to speech data processing. The application areas of the speech emotion recognition method, speech emotion recognition model training method, and apparatus provided in this disclosure are not limited.

[0072] Figure 1 The illustration schematically depicts a system architecture for applying speech emotion recognition methods and training methods for speech emotion recognition models according to embodiments of the present disclosure. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0073] like Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0074] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (for example only).

[0075] Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0076] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0077] It should be noted that the voice emotion recognition method and voice emotion recognition model training method provided in this embodiment can generally be executed by server 105. Correspondingly, the voice emotion recognition method and voice emotion recognition model training method provided in this embodiment can generally be set in server 105. The voice emotion recognition method and voice emotion recognition model training method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and can communicate with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the voice emotion recognition method and voice emotion recognition model training method provided in this embodiment can also be set in a server or server cluster that is different from server 105 and can communicate with terminal devices 101, 102, 103 and / or server 105. Alternatively, the voice emotion recognition method and voice emotion recognition model training method provided in this embodiment can also be executed by terminal devices 101, 102, or 103, or by other terminal devices different from terminal devices 101, 102, or 103. Accordingly, the voice emotion recognition method and the voice emotion recognition model training method provided in this embodiment can also be set in terminal device 101, 102, or 103, or in other terminal devices different from terminal device 101, 102, or 103.

[0078] For example, the speech data to be recognized and the sample speech data can be originally stored in any one of terminal devices 101, 102, or 103 (e.g., terminal device 101, but not limited thereto), or stored on an external storage device and can be imported into terminal device 101. Then, terminal device 101 can locally execute the speech emotion recognition method and speech emotion recognition model training method provided in the embodiments of this disclosure, or send the speech data to be recognized and the sample speech data to be recognized to other terminal devices, servers, or server clusters, and have the other terminal devices, servers, or server clusters that receive the speech data to be recognized and the sample speech data execute the speech emotion recognition method and speech emotion recognition model training method provided in the embodiments of this disclosure.

[0079] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0080] Figure 2 A flowchart illustrating a speech emotion recognition method according to an embodiment of the present disclosure is shown schematically.

[0081] like Figure 2 As shown, the voice emotion recognition method includes operations S201 to S202.

[0082] In operation S201, feature extraction is performed on the speech data to be recognized to obtain a speech emotion feature vector sequence. The speech emotion feature vector sequence includes at least one speech emotion feature vector, which is used to characterize speech intensity features and time increment features.

[0083] In operation S202, the sequence of speech emotion feature vectors is input into the speech emotion recognition model, and the speech emotion recognition result is output. The speech emotion recognition model includes an extreme learning machine neural network.

[0084] According to embodiments of this disclosure, the voice data to be recognized may include data recorded via voice and data transmitted via voice. The voice data to be recognized may be pre-recorded voice data or real-time input voice data.

[0085] According to embodiments of this disclosure, speech data to be recognized can be acquired, and features can be extracted from the speech data to obtain a speech emotion feature vector sequence corresponding to the speech data to be recognized. The speech emotion feature vector sequence may include at least one speech emotion feature vector. The speech emotion feature vector can be used to characterize at least one of the following: sound intensity feature, loudness feature, pitch periodicity feature, fundamental frequency periodicity feature, fundamental frequency feature, signal-to-noise ratio feature, harmonic noise ratio feature, Mel-Frequency Cepstral Coefficients (MFCC) feature, short-time energy feature, and short-time average amplitude feature.

[0086] According to embodiments of this disclosure, the speech emotion recognition model may include support vector machine (ELM), Gaussian mixture model (GMM), hidden markov model (HMM), bag-of-words model (BOW), convolutional neural network (CNN), decision tree model, etc.

[0087] According to embodiments of this disclosure, a speech emotion recognition model can be constructed based on an Extreme Learning Machine (ELM) neural network. The ELM is a single-hidden-layer neural network; the weights between the input and hidden layers, and the bias values ​​of the hidden layers, can be set according to actual needs and do not require adjustment after setting. The weights between the hidden and output layers do not require iterative adjustment and can be determined by solving a system of equations.

[0088] According to embodiments of this disclosure, by inputting a sequence of speech emotion feature vectors obtained from feature extraction of the speech data to be recognized into a speech emotion recognition model including an extreme learning machine neural network, a speech emotion recognition result can be output based on the speech intensity features and time increment features of the speech data to be recognized. Through the above technical means, the technical problems of not being able to guarantee timely speech recognition and low recognition efficiency in related technologies are at least partially overcome. This achieves timely processing of the speech data to be recognized using a speech emotion recognition model, improving the efficiency and accuracy of speech emotion recognition.

[0089] According to embodiments of this disclosure, prior to operation S201, the voice emotion recognition method may further include the following operations.

[0090] Acquire the speech data to be recognized. Segment the speech data to be recognized to obtain multiple raw speech segments corresponding to the speech data to be recognized. Filter each of the multiple raw speech segments to obtain multiple filtered speech segments.

[0091] According to embodiments of this disclosure, voice activity detection (VAD) can be performed on the acquired speech data to be recognized to remove the silent portions at the beginning and end, reducing interference to subsequent steps.

[0092] According to embodiments of this disclosure, the acquired speech data to be recognized can be preprocessed. Preprocessing may include at least one of the following: frame segmentation, windowing, and feature extraction. A moving window function can be used for frame segmentation, that is, dividing the sound into different segments, each segment being called a frame, with overlapping portions between frames. For example, setting the frame length to 25ms and the frame shift to 10ms will result in each frame being 25ms long, with a 15ms overlap between every two frames. Embodiments of this disclosure do not limit the specific values ​​of the frame length and frame shift; those skilled in the art can set them according to actual needs.

[0093] According to embodiments of this disclosure, each of the multiple original speech segment data can be filtered to remove useless parts such as sentence breaks and pauses, resulting in multiple filtered speech segment data.

[0094] According to embodiments of this disclosure, speech intensity features include pitch periodic features and harmonic noise ratio features, and time increment features include Mel frequency cepstral coefficient features.

[0095] According to embodiments of this disclosure, operation S201 may include the following operations.

[0096] Feature extraction is performed on the speech data to be recognized to obtain a speech intensity feature sequence. Further feature extraction is performed on the speech intensity feature sequence to obtain a speech emotion feature vector sequence.

[0097] According to embodiments of this disclosure, feature extraction can be performed on the speech data to be recognized to obtain a speech emotion feature vector sequence that characterizes pitch periodic features, harmonic noise ratio features, and Mel frequency cepstral coefficient features.

[0098] According to embodiments of this disclosure, speech emotion information can be obtained in a continuous window, and the speech emotion feature vector sequence X(m) can be represented by the following equation (1):

[0099] X(m)=[z(mk),...,z(k),...,z(m+k)] (1)

[0100] Where k represents the window size of the moving window, and the feature vector z(m) represents the Mel frequency cepstral coefficients extracted from m frames.

[0101] According to embodiments of this disclosure, the harmonic noise ratio characteristic (HNR) can be represented by the following equation (2):

[0102]

[0103] Where τ represents time, ACF(τ) represents the time autocorrelation function, and τ0(m) represents the pitch periodicity.

[0104] According to embodiments of this disclosure, by filtering multiple original speech segments corresponding to the speech data to be recognized, useless information in the speech data to be recognized can be filtered out, thereby improving the efficiency of speech emotion recognition. Furthermore, by extracting features from the speech data to be recognized, a sequence of speech emotion feature vectors including speech intensity features and time increment features is obtained, thereby improving the accuracy of speech emotion recognition.

[0105] According to embodiments of this disclosure, operation S201 may further include the following operations.

[0106] Multiple speech segments corresponding to the speech data to be recognized are processed to obtain speech emotion feature vectors for each segment. Based on these speech emotion feature vectors and positional encoding, a sequence of speech emotion feature vectors is determined. Positional encoding is used to represent the location information of the speech segment data within the speech data to be recognized.

[0107] According to embodiments of this disclosure, operation S202 may include the following operations.

[0108] The speech emotion feature vectors of multiple speech segments are processed to obtain the speech emotion information of each segment. Based on the speech emotion information of each segment, the speech emotion recognition result is determined.

[0109] According to embodiments of this disclosure, the voice emotion recognition result may include at least one of the following: happiness, surprise, anger, disgust, fear, sadness, and contempt.

[0110] According to embodiments of this disclosure, positional encoding can be used to characterize the positional information of speech segment data within speech data to be recognized. Positional encoding methods can be used to encode the positions of multiple speech segment data included in the speech data to be recognized, obtaining positional codes corresponding to each of the multiple speech segment data. Positional encoding methods may include sine / cosine positional encoding methods or learned position vector methods.

[0111] According to embodiments of this disclosure, after obtaining the speech emotion feature vectors and position codes of multiple speech segment data, the speech emotion feature vectors and position codes of each speech segment data can be fused separately to obtain a fusion result. Then, feature extraction is performed on the fusion result to obtain a speech emotion feature vector sequence.

[0112] According to embodiments of this disclosure, voice emotion information can be used to characterize the emotion at the segment level of voice segment data, and voice emotion recognition results can be used to characterize the emotion at the discourse level of the voice data to be recognized.

[0113] According to embodiments of this disclosure, voice emotion information includes at least one voice emotion category and a probability value corresponding to each of the at least one voice emotion category.

[0114] According to embodiments of this disclosure, determining the voice emotion recognition result based on the voice emotion information of each of multiple voice segment data may include the following operations.

[0115] Based on the probability values, the target speech emotion category and target probability value are determined for each speech segment in multiple speech segment data. The speech emotion recognition result is then determined based on the target speech emotion category, target probability value, and location encoding of the speech segment data.

[0116] According to embodiments of this disclosure, the voice emotion recognition result can be determined based on the probability values ​​of the emotional states represented by the voice emotion information of each of multiple voice segment data. For example, the emotional state corresponding to the highest probability value can be determined as the voice emotion recognition result.

[0117] According to embodiments of this disclosure, by processing the voice emotion feature vectors of each of multiple voice segment data, and based on the obtained voice emotion information of each of the multiple voice segment data, the target voice emotion category corresponding to the target probability value can be determined as the voice emotion recognition result, which can improve the efficiency and accuracy of voice emotion recognition.

[0118] The following is for reference. Figure 3 In conjunction with specific embodiments, Figure 2 The method shown will be further explained.

[0119] Figure 3 An example schematic diagram of a speech emotion recognition process according to an embodiment of the present disclosure is shown.

[0120] like Figure 3 As shown, speech data 301 to be recognized can be obtained. The speech data 301 is segmented to obtain multiple original speech segment data 302 corresponding to the speech data 301. Each original speech segment data 302 is filtered to obtain multiple filtered speech segment data 303. The multiple speech segment data 303 corresponding to the speech data 301 are processed to obtain individual speech emotion feature vectors 304. The individual speech emotion feature vectors 304 and position codes 305 of the multiple speech segment data 303 are input into the speech emotion recognition model 306 to obtain the speech emotion recognition result 307.

[0121] Figure 4 A flowchart illustrating a training method for a speech emotion recognition model according to an embodiment of the present disclosure is shown.

[0122] like Figure 4 As shown, the training method for this speech emotion recognition model includes operations S401 to S403.

[0123] In operation S401, feature extraction is performed on the sample speech data to obtain a sequence of sample speech emotion feature vectors. This sequence includes at least one sample speech emotion feature vector, which is used to characterize the sample speech intensity and time increment features.

[0124] In operation S402, the sequence of sample speech emotion feature vectors is input into the speech emotion recognition model to be trained, and the sample speech emotion recognition result is output. The speech emotion recognition model to be trained includes an extreme learning machine neural network.

[0125] In operation S403, a speech emotion recognition model is trained based on the sample speech emotion recognition results and the real sample speech emotion results.

[0126] According to embodiments of this disclosure, sample voice data may include pre-recorded voice data or real-time input voice data.

[0127] According to embodiments of this disclosure, a speech emotion recognition model can be trained to predict the probability value of each emotion state. By inputting the speech emotion feature vector and the corresponding positional code, setting the number of emotion states to k, and using the softmax function as the output layer, the output of the speech emotion recognition model can be a probability distribution t representing each emotion state, as shown in equation (3) below:

[0128] t=[P(E1),...,P(E K )] T (3)

[0129] Among them, P(E) k Let represent the probability of the k-th emotion, where k = 1, ..., K.

[0130] According to embodiments of this disclosure, the speech emotion recognition problem can be transformed into a sequence classification problem, that is, the speech emotion recognition result of the speech data to be recognized can be determined based on the speech emotion information of each of the multiple speech segment data, and can be calculated using the following formulas (4) to (7):

[0131]

[0132]

[0133]

[0134]

[0135] Where U represents a set of multiple speech segments corresponding to the speech data to be recognized, f1 k This represents the maximum probability value that the s-th speech segment belongs to the k-th emotion. This represents the minimum probability value that the s-th speech segment belongs to the k-th emotion. This represents the average probability value that the s-th speech segment belongs to the k-th emotion. θ represents the percentage of speech segments with the highest probability value belonging to the k-th emotion, and θ represents the threshold.

[0136] According to embodiments of this disclosure, after obtaining the sample speech emotion recognition results, the speech emotion recognition model can be trained using the sample speech emotion recognition results and the real sample speech emotion results to obtain the trained speech emotion recognition model.

[0137] According to embodiments of this disclosure, a speech emotion recognition model is trained using sample speech emotion recognition results and real sample speech emotion results as training samples. When the speech emotion recognition model is used to process the speech data to be recognized, it can accurately identify the speech emotion category of the speech data to be processed, thereby achieving the technical effects of improving the efficiency and accuracy of speech emotion category prediction and reducing the cost of manual recognition.

[0138] According to embodiments of this disclosure, the voice emotion recognition model includes an input layer, a hidden layer, and an output layer, wherein the hidden layer includes a preset number of nodes.

[0139] According to embodiments of this disclosure, operation S402 may include the following operations.

[0140] Based on a preset number, set the first weight value and the bias value of the hidden layer. The first weight value is used to characterize the weight between the input layer and the hidden layer.

[0141] According to embodiments of this disclosure, operation S403 may include the following operations.

[0142] Based on the sample speech emotion recognition results, the first weight value, and the bias value of the hidden layer, the output matrix of the hidden layer is determined. A generalized inverse operation is performed on the output matrix of the hidden layer to obtain the output result of the hidden layer. Based on the output result of the hidden layer and the actual sample speech emotion results, the second weight value is determined. The second weight value is used to represent the weight between the hidden layer and the output layer. The speech emotion recognition model is trained using a preset number of parameters, the first weight value, the bias value, and the second weight value.

[0143] According to embodiments of this disclosure, a sequence of sample speech emotion feature vectors can be used as training data for an input speech emotion recognition model. A preset number of hidden layer nodes is determined based on the speech data to be recognized. This disclosure does not limit the specific value of the preset number; it can be set according to actual needs.

[0144] According to embodiments of this disclosure, by setting a first weight value and a bias value for the extreme learning machine, and obtaining a second weight value based on the output of the hidden layer and the actual sample speech emotion results, the model parameters of the speech emotion recognition model can be determined.

[0145] The following is for reference. Figure 5 In conjunction with specific embodiments, Figure 4 The method shown will be further explained.

[0146] Figure 5 The illustration shows an example diagram of the training process of a speech emotion recognition model according to an embodiment of the present disclosure.

[0147] like Figure 5As shown, sample speech data 501 can be obtained. The sample speech data 501 is segmented to obtain multiple original sample speech segment data 502 corresponding to the sample speech data 501. Each original sample speech segment data 502 is filtered to obtain multiple filtered sample speech segment data 503. The multiple sample speech segment data 503 corresponding to the sample speech data 501 are processed to obtain the sample speech emotion feature vector 504 for each of the multiple sample speech segment data 503.

[0148] The sample speech emotion feature vectors 504 and sample position codes 505 of multiple sample speech segment data 503 are input into the input layer 5061 of the speech emotion recognition model 506. Based on the number of nodes in the hidden layer 5062, a first weight value w1 between the input layer 5061 and the hidden layer 5062, and a bias value b for the hidden layer 5062 are set. Based on the output result 507 of the hidden layer 5062 and the real sample speech emotion result 508, a second weight value w2 between the hidden layer 5062 and the output layer 5063 is determined. The speech emotion recognition model 506 is trained based on the number of nodes, the first weight value w1, the bias value b, and the second weight value w2.

[0149] Figure 6 A schematic block diagram of a voice emotion recognition device according to an embodiment of the present disclosure is shown.

[0150] like Figure 6 As shown, the voice emotion recognition device 600 includes a first processing module 601 and a first recognition module 602.

[0151] The first processing module 601 is used to extract features from the speech data to be recognized to obtain a speech emotion feature vector sequence, wherein the speech emotion feature vector sequence includes at least one speech emotion feature vector, which is used to characterize speech intensity features and time increment features.

[0152] The first recognition module 602 is used to input the speech emotion feature vector sequence into the speech emotion recognition model and output the speech emotion recognition result. The speech emotion recognition model includes an extreme learning machine neural network.

[0153] According to embodiments of this disclosure, speech intensity features include pitch periodic features and harmonic noise ratio features, and time increment features include Mel frequency cepstral coefficient features.

[0154] According to embodiments of this disclosure, the first processing module 601 includes a first extraction unit and a second extraction unit.

[0155] The first extraction unit is used to extract features from the speech data to be recognized, and obtain a speech intensity feature sequence.

[0156] The second extraction unit is used to extract features from the speech intensity feature sequence to obtain a speech emotion feature vector sequence.

[0157] According to embodiments of this disclosure, the voice emotion recognition device 600 further includes an acquisition module, a third processing module, and a fourth processing module.

[0158] The acquisition module is used to acquire the speech data to be recognized.

[0159] The third processing module is used to segment the speech data to be recognized to obtain multiple original speech segments corresponding to the speech data to be recognized.

[0160] The fourth processing module is used to filter each of the multiple raw speech segments to obtain multiple filtered speech segments.

[0161] According to embodiments of this disclosure, the first processing module 601 includes a first processing unit and a first determining unit.

[0162] The first processing unit is used to process multiple speech segment data corresponding to the speech data to be recognized, and obtain the speech emotion feature vectors of each of the multiple speech segment data.

[0163] The first determining unit is used to determine the speech emotion feature vector sequence based on the speech emotion feature vector and the position code, wherein the position code is used to characterize the position information of the speech segment data in the speech data to be recognized.

[0164] According to embodiments of this disclosure, the first identification module 602 includes a second processing unit and a second determining unit.

[0165] The second processing unit is used to process the voice emotion feature vectors of each of the multiple voice segment data to obtain the voice emotion information of each of the multiple voice segment data.

[0166] The second determining unit is used to determine the voice emotion recognition result based on the voice emotion information of each of the multiple voice segment data.

[0167] According to embodiments of this disclosure, voice emotion information includes at least one voice emotion category and a probability value corresponding to each of the at least one voice emotion category.

[0168] According to embodiments of this disclosure, the second determining unit includes a first determining subunit and a second determining subunit.

[0169] The first determining subunit is used to determine the target speech emotion category and target probability value for each speech segment data in multiple speech segment data based on the probability value.

[0170] The second determining subunit is used to determine the speech emotion recognition result based on the target speech emotion category, target probability value, and location encoding of the speech segment data.

[0171] Figure 7 A schematic block diagram of a training apparatus for a speech emotion recognition model according to an embodiment of the present disclosure is shown.

[0172] like Figure 7 As shown, the training device 700 for the voice emotion recognition model includes a second processing module 701, a second recognition module 702, and a training module 703.

[0173] The second processing module 701 is used to extract features from the sample speech data to obtain a sequence of sample speech emotion feature vectors. The sequence of sample speech emotion feature vectors includes at least one sample speech emotion feature vector, which is used to characterize the sample speech intensity feature and the sample time increment feature.

[0174] The second recognition module 702 is used to input the sequence of sample speech emotion feature vectors into the speech emotion recognition model to be trained and output the sample speech emotion recognition result. The speech emotion recognition model to be trained includes an extreme learning machine neural network.

[0175] Training module 703 is used to train a speech emotion recognition model based on the sample speech emotion recognition results and the real sample speech emotion results.

[0176] According to embodiments of this disclosure, the voice emotion recognition model includes an input layer, a hidden layer, and an output layer, wherein the hidden layer includes a preset number of nodes.

[0177] According to an embodiment of this disclosure, the second identification module 702 includes a setting unit.

[0178] The setting unit is used to set the first weight value and the bias value of the hidden layer according to a preset number, wherein the first weight value is used to characterize the weight between the input layer and the hidden layer.

[0179] According to embodiments of this disclosure, the training module 703 includes a third determining unit, a calculation unit, a fourth determining unit, and a training unit.

[0180] The third determining unit is used to determine the output matrix of the hidden layer based on the sample speech emotion recognition results, the first weight value, and the bias value of the hidden layer.

[0181] The operation unit is used to perform generalized inverse operations on the output matrix of the hidden layer to obtain the output result of the hidden layer.

[0182] The fourth determining unit is used to determine the second weight value based on the output of the hidden layer and the actual sample speech emotion result, wherein the second weight value is used to characterize the weight between the hidden layer and the output layer.

[0183] The training unit is used to train the speech emotion recognition model according to a preset number, a first weight value, a bias value, and a second weight value.

[0184] Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be implemented by dividing them into multiple modules. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as hardware circuitry, such as Field Programmable Gate Arrays (FPGAs), Programmable Logic Arrays (PLAs), Systems-on-Chip, Systems-on-Substrate, Systems-on-Package, Application-Specific Integrated Circuits (ASICs), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0185] For example, any plurality of the first processing module 601 and the first identification module 602, the second processing module 701, the second identification module 702, and the training module 703 can be combined into one module / unit / subunit, or any one of these modules / units / subunits can be split into multiple modules / units / subunits. Alternatively, at least part of the functionality of one or more of these modules / units / subunits can be combined with at least part of the functionality of other modules / units / subunits and implemented in one module / unit / subunit. According to embodiments of this disclosure, at least one of the first processing module 601 and the first identification module 602, the second processing module 701, the second identification module 702, and the training module 703 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the first processing module 601 and the first identification module 602, the second processing module 701, the second identification module 702, and the training module 703 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0186] It should be noted that the speech emotion recognition device part in the embodiments of this disclosure corresponds to the speech emotion recognition method part in the embodiments of this disclosure. For a detailed description of the speech emotion recognition device part, please refer to the speech emotion recognition method part, which will not be repeated here. Similarly, the training device part for the speech emotion recognition model in the embodiments of this disclosure corresponds to the training method part for the speech emotion recognition model in the embodiments of this disclosure. For a detailed description of the training device part for the speech emotion recognition model, please refer to the training method part for the speech emotion recognition model, which will not be repeated here.

[0187] Figure 8 A block diagram of an electronic device suitable for implementing a speech emotion recognition method and a training method for a speech emotion recognition model, according to embodiments of the present disclosure, is shown schematically. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0188] like Figure 8As shown, a computer electronic device 800 according to an embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0189] RAM 803 stores various programs and data required for the operation of electronic device 800. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0190] According to embodiments of this disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0191] According to embodiments of this disclosure, the method flow according to embodiments of this disclosure can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by processor 801, it performs the functions defined in the system of embodiments of this disclosure. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0192] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0193] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0194] For example, according to embodiments of this disclosure, a computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.

[0195] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of this disclosure. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the speech emotion recognition method and the speech emotion recognition model training method provided in the embodiments of this disclosure.

[0196] When the computer program is executed by the processor 801, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0197] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0198] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0199] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features recited in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not expressly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure may be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0200] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A voice emotion recognition method, comprising: Feature extraction is performed on the speech data to be recognized to obtain a speech emotion feature vector sequence, wherein the speech emotion feature vector sequence includes at least one speech emotion feature vector, which is used to characterize speech intensity features and time increment features; and The speech emotion feature vector sequence is input into the speech emotion recognition model, and the speech emotion recognition result is output. The speech emotion recognition model includes an extreme learning machine neural network, and the hidden layer of the extreme learning machine neural network includes a preset number of nodes, which is determined according to the speech data to be recognized. The step of inputting the speech emotion feature vector sequence into the speech emotion recognition model and outputting the speech emotion recognition result includes: The voice emotion feature vectors of each of the multiple voice segment data corresponding to the voice data to be identified are processed to obtain the voice emotion information of each of the multiple voice segment data, wherein the voice emotion information includes at least one voice emotion category and a probability value corresponding to each of the at least one voice emotion category; Based on the probability values, determine the target speech emotion category and target probability value for each speech segment in the plurality of speech segment data; and The speech emotion recognition result is determined based on the target speech emotion category, target probability value, and location code of the speech segment data, wherein the location code is used to characterize the location information of the speech segment data in the speech data to be recognized.

2. The method according to claim 1, wherein, The speech intensity features include pitch periodic features and harmonic noise ratio features, and the time increment features include Mel frequency cepstral coefficient features; The process of extracting features from the speech data to be recognized to obtain a sequence of speech emotion feature vectors includes: Feature extraction is performed on the speech data to be identified to obtain a speech intensity feature sequence; as well as Feature extraction is performed on the speech intensity feature sequence to obtain the speech emotion feature vector sequence.

3. The method according to claim 1 further includes, before performing feature extraction on the speech data to be recognized to obtain a speech emotion feature vector sequence: Acquire the speech data to be recognized; The speech data to be recognized is segmented to obtain multiple original speech segment data corresponding to the speech data to be recognized; and Each of the multiple original speech segment data is filtered to obtain the filtered multiple speech segment data.

4. The method according to claim 3, wherein, The process of extracting features from the speech data to be recognized to obtain a sequence of speech emotion feature vectors includes: The speech segment data corresponding to the speech data to be identified are processed to obtain the speech emotion feature vector of each of the speech segment data; and The speech emotion feature vector sequence is determined based on the speech emotion feature vector and the position encoding.

5. The method according to claim 1, wherein, The speech emotion recognition model was trained using the following method: Feature extraction is performed on the sample speech data to obtain a sequence of sample speech emotion feature vectors, wherein the sequence of sample speech emotion feature vectors includes at least one sample speech emotion feature vector, which is used to characterize the sample speech intensity feature and the sample time increment feature. The sample speech emotion feature vector sequence is input into a speech emotion recognition model to be trained, and the sample speech emotion recognition result is output. The speech emotion recognition model to be trained includes an Extreme Learning Machine (ELM) neural network, and the hidden layer of the ELM neural network includes a preset number of nodes, the preset number being determined based on the sample speech data. The speech emotion recognition model is trained based on the sample speech emotion recognition results and the real sample speech emotion results.

6. The method according to claim 5, wherein, The speech emotion recognition model includes an input layer, a hidden layer, and an output layer; The step of inputting the sample speech emotion feature vector sequence into the speech emotion recognition model to be trained and outputting the sample speech emotion recognition result includes: Based on the preset quantity, a first weight value and a bias value of the hidden layer are set, wherein the first weight value is used to characterize the weight between the input layer and the hidden layer.

7. The method according to claim 6, wherein, The step of training the speech emotion recognition model based on the sample speech emotion recognition results and the real sample speech emotion results includes: The output matrix of the hidden layer is determined based on the sample speech emotion recognition result, the first weight value, and the bias value of the hidden layer. Perform a generalized inverse operation on the output matrix of the hidden layer to obtain the output result of the hidden layer; Based on the output of the hidden layer and the actual sample speech emotion result, a second weight value is determined, wherein the second weight value is used to characterize the weight between the hidden layer and the output layer; and The speech emotion recognition model is trained based on the preset quantity, the first weight value, the bias value, and the second weight value.

8. A voice emotion recognition device, comprising: The first processing module is used to extract features from the speech data to be recognized, obtaining a speech emotion feature vector sequence, wherein the speech emotion feature vector sequence includes at least one speech emotion feature vector, which is used to characterize speech intensity features and time increment features; and The first recognition module is used to input the speech emotion feature vector sequence into the speech emotion recognition model and output the speech emotion recognition result. The speech emotion recognition model includes an extreme learning machine neural network, and the hidden layer of the extreme learning machine neural network includes a preset number of nodes, which is determined according to the speech data to be recognized. The first identification module includes: The second processing unit is used to process the voice emotion feature vectors of each of the multiple voice segment data corresponding to the voice data to be identified, to obtain the voice emotion information of each of the multiple voice segment data, wherein the voice emotion information includes at least one voice emotion category and a probability value corresponding to each of the at least one voice emotion category; The first determining subunit is configured to determine, based on the probability value, the target speech emotion category and target probability value for each speech segment in the plurality of speech segment data; and The second determining subunit is used to determine the voice emotion recognition result based on the target voice emotion category, target probability value and position code of the voice segment data, wherein the position code is used to characterize the position information of the voice segment data in the voice data to be recognized.

9. The apparatus according to claim 8, wherein, The speech emotion recognition model was trained using the following device: The second processing module is used to extract features from the sample speech data to obtain a sequence of sample speech emotion feature vectors, wherein the sequence of sample speech emotion feature vectors includes at least one sample speech emotion feature vector, and the sample speech emotion feature vector is used to characterize the sample speech intensity feature and the sample time increment feature. The second recognition module is used to input the sample speech emotion feature vector sequence into a speech emotion recognition model to be trained, and output the sample speech emotion recognition result. The speech emotion recognition model to be trained includes an Extreme Learning Machine (ELM) neural network, and the hidden layer of the ELM neural network includes a preset number of nodes, the preset number being determined based on the sample speech data. The training module is used to train the speech emotion recognition model based on the sample speech emotion recognition results and the real sample speech emotion results.

10. An electronic device, comprising: One or more processors; Memory, used to store one or more instructions. When the one or more instructions are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1 to 7.

11. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 7.

12. A computer program product comprising computer-executable instructions, which, when executed, are used to implement the method of any one of claims 1 to 7.

Images