Method and device for optimizing user voice emotion model of Internet of Vehicles, and electronic equipment

Abstract

The invention provides an Internet of Vehicles user voice emotion model optimization method and device and electronic equipment, and relates to the field of Internet of Vehicles user voice emotion recognition. Specifically, the method comprises the following steps: acquiring original voice text data for training a first voice emotion model, and first voice text data collected in an emotion prediction process by applying a second voice emotion model, the second voice emotion model is obtained by training the first voice emotion model through the original voice text data; screening out first target data from the first voice text data through the second voice emotion model; and optimizing the second speech emotion model by using the original speech text data and the first target data to obtain a target speech emotion model, the target speech emotion model being used for performing emotion prediction on the speech text data of the Internet of Vehicles user.

Description

Technical Field

[0001] The present invention relates to the field of voice emotion recognition for Internet of Vehicles users, and in particular to a method, device and electronic equipment for optimizing a voice emotion model for Internet of Vehicles users. Background Art

[0002] With the rapid development of Internet of Vehicles technology, vehicles are no longer simple means of transportation, but have become intelligent mobile spaces that integrate communication, entertainment, navigation and other functions. In this process, speech emotion recognition technology, as an important part of human-computer interaction, has received widespread attention. Speech emotion recognition technology aims to accurately identify the user's emotional state, such as happiness, sadness, anger, etc., by analyzing the user's voice signal, so as to provide more humane and intelligent services. Furthermore, in order to improve the effect of speech emotion recognition technology, speech emotion recognition technology is usually updated and optimized from time to time.

[0003] In related technologies, deep learning models are usually used to implement speech emotion recognition technology. Traditional models require a large amount of labeled data during the optimization process, and the generalization ability of the model is limited, resulting in low model accuracy and difficulty in adapting to the complex and changeable Internet of Vehicles environment. Summary of the invention

[0004] The present invention provides a method, device and electronic device for optimizing a voice emotion model of an Internet of Vehicles user, which are used to solve the problem of low accuracy in the related art of using traditional models to recognize voice emotions of Internet of Vehicles.

[0005] In a first aspect, the present invention provides a method for optimizing a voice emotion model of an Internet of Vehicles user, the method comprising: Acquire original speech text data used to train a first speech emotion model, and first speech text data collected during the process of applying a second speech emotion model to perform emotion prediction, wherein the second speech emotion model is obtained by training the first speech emotion model with the original speech text data; Filtering first target data from the first speech text data by using the second speech emotion model; The second speech emotion model is optimized using the original speech text data and the first target data to obtain a target speech emotion model, and the target speech emotion model is used to perform emotion prediction on the speech text data of the Internet of Vehicles user.

[0006] In a second aspect, the present invention further provides a device for optimizing a voice emotion model of a vehicle networking user, the device comprising: An acquisition module, used to acquire original speech text data used to train a first speech emotion model, and first speech text data collected in the process of applying a second speech emotion model to perform emotion prediction, wherein the second speech emotion model is obtained by training the first speech emotion model with the original speech text data; A screening module, used for screening out first target data from the first speech text data by using the second speech emotion model; An optimization module is used to optimize the second speech emotion model using the original speech text data and the first target data to obtain a target speech emotion model, and the target speech emotion model is used to perform emotion prediction on the speech text data of the Internet of Vehicles user.

[0007] In a third aspect, the present invention further provides an electronic device comprising a processor and a memory, wherein the memory stores a program or instruction running on the processor, and when the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented.

[0008] In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed, the steps of the method described in the first aspect are implemented.

[0009] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, wherein the computer program is executed by a processor to perform the steps of the method described in the first aspect.

[0010] The present invention provides a method for optimizing a speech emotion model of a user of an Internet of Vehicles. Specifically, the method comprises obtaining original speech text data for training a first speech emotion model and first speech text data collected in the process of applying a second speech emotion model for emotion prediction, wherein the second speech emotion model is obtained by training the first speech emotion model with the original speech text data; filtering out first target data from the first speech text data by the second speech emotion model; optimizing the second speech emotion model by using the original speech text data and the first target data to obtain a target speech emotion model, wherein the target speech emotion model is used for emotion prediction of speech text data of an Internet of Vehicles user. In this way, the second speech emotion model filters out the first target data from the first speech text data collected in the application process, and trains by using the original speech text data and the first target data. By iterative means, active learning of the second speech emotion model is realized, so that the decision boundary of the optimized target speech emotion model is more consistent with the distribution of the first target data, that is, closer to the user, thereby improving the accuracy of the target speech emotion model, and to a certain extent solving the problem of low accuracy in the related art of using traditional models to identify speech emotions of the Internet of Vehicles. At the same time, in the embodiment of the present invention, the first target data is screened out from the first voice text data, which reduces the demand for input data in the model optimization process and can improve the efficiency of the model optimization process. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0012] Figure 1 A flowchart of a method for optimizing a voice emotion model of an Internet of Vehicles user provided in an embodiment of the present invention; Figure 2 A schematic diagram of part of original speech text data in an embodiment of the present invention; Figure 3 An architecture diagram of a vehicle networking user voice emotion prediction system provided by an embodiment of the present invention; Figure 4 A conceptual diagram of the optimization process of the original speech emotion classification model provided by an embodiment of the present invention; Figure 5 A structural block diagram of a device for optimizing a user speech emotion model of an Internet of Vehicles provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0013] With the rapid development of Internet of Vehicles technology, vehicles are no longer simple means of transportation, but have become intelligent mobile spaces that integrate communication, entertainment, navigation and other functions. In this process, speech emotion recognition technology, as an important part of human-computer interaction, has received widespread attention. Speech emotion recognition technology aims to accurately identify the user's emotional state, such as happiness, sadness, anger, etc., by analyzing the user's voice signal, so as to provide more humane and intelligent services.

[0014] In related technologies, deep learning models are usually used to achieve voice emotion analysis and prediction of Internet of Vehicles users. For example, as an important model of deep learning, the Long Short-Term Memory (LSTM) network has a unique advantage in processing sequence data. In the field of voice emotion recognition of Internet of Vehicles users, the LSTM model can capture the temporal dependency and contextual information in the voice signal, thereby accurately predicting the user's emotional state. However, the traditional LSTM model requires a large amount of labeled data during the optimization process, and the generalization ability of the model is limited, making it difficult to adapt to the complex and changeable Internet of Vehicles environment. How to reduce the dependence on labeled data and improve the generalization ability and prediction accuracy of the model are key issues that need to be urgently solved in related technologies.

[0015] In this regard, the present invention provides a method for optimizing a speech emotion model of a user of an Internet of Vehicles, comprising: obtaining original speech text data for training a first speech emotion model, and first speech text data collected during the process of applying a second speech emotion model for emotion prediction, wherein the second speech emotion model is obtained by training the first speech emotion model with the original speech text data; filtering out first target data from the first speech text data by the second speech emotion model; optimizing the second speech emotion model by using the original speech text data and the first target data to obtain a target speech emotion model, wherein the target speech emotion model is used to perform emotion prediction on the speech text data of the user of the Internet of Vehicles. In this way, the second speech emotion model filters out the first target data from the first speech text data collected during the application process, and trains by using the original speech text data and the first target data, and realizes active learning of the second speech emotion model by iteration, so that the decision boundary of the optimized target speech emotion model is more consistent with the distribution of the first target data, that is, closer to the user, thereby improving the accuracy of the target speech emotion model, and to a certain extent solves the problem of low accuracy in the related art of using traditional models to identify speech emotions of the Internet of Vehicles. At the same time, in the embodiment of the present invention, the first target data is screened out from the first voice text data, which reduces the demand for input data in the model optimization process and can improve the efficiency of the model optimization process.

[0016] In addition, in order to solve the problem of insufficient labeled data, a speech emotion recognition method based on transfer learning is proposed in the related art. This method uses a model pre-trained on other tasks as a starting point and adapts to new speech emotion recognition tasks through fine-tuning. Although transfer learning can reduce the demand for labeled data to a certain extent, it still has great uncertainty in model selection, migration strategy, etc., and its prediction accuracy still needs to be improved for complex Internet of Vehicles environments. Due to the complexity and variability of the Internet of Vehicles environment, traditional speech emotion recognition methods are often difficult to adapt. However, the optimization method provided in the embodiment of the present invention can realize the active learning of the model. The model automatically adjusts the model parameters and structure through continuous learning and adaptation to adapt to the needs of speech emotion recognition in different scenarios. This adaptive capability makes the target speech emotion model provided in the embodiment of the present invention have stronger robustness and generalization ability in the field of speech emotion recognition of Internet of Vehicles users.

[0017] It should be understood that the method for optimizing the voice emotion model of a vehicle network user provided in an embodiment of the present invention can be executed by a target device. The target device can be one electronic device or multiple electronic devices that cooperate with each other to execute. The electronic device can be a server, such as an independent physical server, a server cluster composed of multiple servers, and a cloud server capable of cloud computing.

[0018] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0019] Figure 1 A flow chart of a method provided by an embodiment of the present invention. Figure 1 As shown, the method provided by the embodiment of the present invention includes the following steps 110 to 130.

[0020] Step 110, obtaining original speech text data used to train the first speech emotion model, and the first speech text data collected in the process of applying the second speech emotion model for emotion prediction, wherein the second speech emotion model is obtained by training the first speech emotion model with the original speech text data.

[0021] In an embodiment of the present invention, the first speech emotion model may be an untrained LSTM model. The original speech text data may include open source speech emotion data and emotion labels corresponding to each piece of speech emotion data, and the emotion labels may be labels that can accurately reflect the user's emotional state. In an embodiment of the present invention, the emotion labels corresponding to the speech text data are divided into 7 categories, namely: happy, sad, angry, calm, fearful, disgusted, and surprised. In order to better understand the original speech text data provided by the embodiment of the present invention, an example is given. Some of the original speech text data provided by the embodiment of the present invention can be as follows Figure 2 As shown, each piece of original speech text data has a corresponding emotion label (ie, the emotion label mentioned above). After obtaining the original speech text data, the first speech emotion model can be trained by the original speech text data to obtain the second speech emotion model.

[0022] In an embodiment of the present invention, the process of obtaining the second speech emotion model by training the first speech emotion model with the original speech text data may include the following steps: S1, obtain the original voice text data. You can collect part of the vehicle-side voice data and collect part of the open source voice emotion data, and match the emotion label with the user's real emotion in the voice text. For specific steps, please refer to the above description. After obtaining the original voice text data, you can execute step S2.

[0023] S2, speech text segmentation. In an embodiment of the present invention, word segmentation can be performed by a word segmentation tool (such as the jieba word segmentation tool). For example, in the process of using the jieba word segmentation tool for word segmentation, jieba will first generate a prefix dictionary, segment the words in the vocabulary according to the prefixes of the characters, and then convert the text to be segmented (i.e., the original speech text data) into a directed acyclic graph (DAG), in which each node represents a specific segment, and then use a dynamic programming algorithm (such as the Viterbi algorithm) to perform a path search on the word graph to find the most likely word combination. In order to identify unregistered words (i.e., words that do not exist in the dictionary), jieba also uses a hidden Markov model (HMM) for annotation. By training a large amount of Chinese corpus, it learns information such as the probability of occurrence and part of speech of words, thereby improving the accuracy and robustness of word segmentation.

[0024] In an embodiment of the present invention, the Paddle mode can be used in Jieba word segmentation, and the PaddlePaddle deep learning framework is used to train the sequence annotation (bidirectional GRU) network model to implement word segmentation. After a long piece of original speech text data is cut into individual words, step S3 can be executed.

[0025] S3, word encoding. Based on the word vector cut in step S1, the word vector is further encoded. In an embodiment of the present invention, the CBOW model in the Word2vec technology can be used to perform word encoding operations. During the word encoding process, the CBOW model will learn the context information of each word, predict the intermediate word through the context of one or more words, and encode it into the word vector. Word2Vec encoding maps words in the text to word vectors by training a neural network, captures the semantic relationship between words, and realizes text vectorization. Based on the idea of ​​distributed representation, the CBOW model can convert the original word vector with a dimension of V (vocabulary size) into a word vector in the form of data encoding with a dimension of N (N is much smaller than V), while retaining the correlation between word vectors.

[0026] In an embodiment of the present invention, the CBOW model may be a neural network having one hidden layer. In the process of word encoding through the CBOW model, a One-Hot encoded vocabulary vector is used as input. In the hidden layer, the weight from the input layer to the hidden layer is the word vector of each word, and finally a One-Hot encoded vocabulary vector is used as output.

[0027] In an embodiment of the present invention, in order to improve the training efficiency and the quality of word vectors, Word2Vec can adopt the optimization techniques of hierarchical softmax (constructing a Huffman tree to accelerate the calculation process of softmax) and negative sampling (approximating the training target by randomly extracting negative samples).

[0028] After step S3, the original speech text data can be expressed in the form of individual word vectors in a data encoding form, and step S4 can be executed to use the encoded data to train the first speech emotion model to obtain a second speech emotion model.

[0029] S4, training a first speech emotion model. Based on the encoding operation in S3 above, the encoded data can be used to train the first speech emotion model.

[0030] In an embodiment of the present invention, the data after step S3 can be divided into a training set and a test set. The first speech emotion model is trained by the data of the training set, and the prediction effect of the first speech emotion model is verified by the data of the test set, so as to obtain the second speech emotion model.

[0031] In an embodiment of the present invention, the first speech emotion model may be an LSTM model. Three gating mechanisms are introduced into LSTM, namely forget gate, input gate and output gate, which allow the network to dynamically determine the retention and forgetting of information. Through the gating mechanism, LSTM can effectively solve the problem of long-term dependency. LSTM shares the same weights at each time step of the sequence, which allows the model to process sequences of arbitrary length. The operation of the LSTM unit at each time step (t) can be described in Table 1 below: Table 1 in, is the input of the current time step, is the hidden state at the previous time step, is the cell state at the previous time step, W and h are the weight matrix and bias vector respectively, σ is the sigmoid function, and * represents element-by-element multiplication.

[0032] In the embodiment of the present invention, after the second speech emotion model is obtained in step S4, the second speech emotion model can be applied online and the first speech text data can be collected. After the first speech text data is obtained, step 120 can be executed.

[0033] Step 120: Filter out first target data from the first speech text data using the second speech emotion model.

[0034] In an embodiment of the present invention, the emotion prediction of the first voice text data can be performed by the second voice emotion model, and the text information in the first voice text data that cannot be accurately predicted by the second voice emotion model can be used as the first target data. For example, for a certain voice text in the first voice text data, the prediction result of the second voice emotion model is wrong, then the voice text data can be used as the first target data. Alternatively, in the prediction process of the second voice emotion model, the multiple emotions predicted by the second voice emotion model for a certain voice text and the probabilities corresponding to the multiple emotions are highly close. For example, the second voice emotion model determines that one of the emotion labels of a certain voice text is happy with a probability of 51%, and the other emotion label is sad with a probability of 48%, then the voice text data can also be used as the first target data.

[0035] After the first target data is obtained by filtering the first speech text data through the second speech emotion model, the amount of data in the process of optimizing the second speech emotion model has been greatly reduced, and step 130 can be executed.

[0036] Step 130, using the original voice text data and the first target data, optimize the second voice emotion model to obtain a target voice emotion model, and the target voice emotion model is used to predict the emotion of the voice text data of the Internet of Vehicles user.

[0037] In an embodiment of the present invention, after obtaining the first target data, the second speech emotion model can be optimized, i.e., trained again, using the original speech text data and the first target data. The optimization process can refer to the method steps S1 to S4 described above for training the first speech emotion model into the second speech emotion model to obtain the target speech emotion model.

[0038] In an embodiment of the present invention, after obtaining the target speech emotion model, the target speech emotion model can be applied online to perform emotion prediction on the speech text data of Internet of Vehicles users. At the same time, the user's speech text data can be collected for subsequent optimization.

[0039] In this way, the second voice emotion model selects the first target data from the first voice text data collected during the application process, and uses the original voice text data and the first target data for training. Through iteration, the active learning of the second voice emotion model is realized, so that the decision boundary of the optimized target voice emotion model is more consistent with the distribution of the first target data, that is, closer to the user, thereby improving the accuracy of the target voice emotion model, and to a certain extent solving the problem of low accuracy in the related technology of using traditional models to identify voice emotions in the Internet of Vehicles. At the same time, in an embodiment of the present invention, the first target data is selected from the first voice text data, which reduces the demand for input data in the model optimization process and can improve the efficiency of the model optimization process.

[0040] In an embodiment of the present invention, the first voice text data includes N voice texts. A specific implementation method of step 120 may be: according to the prediction results obtained by sequentially inputting the N voice texts into the second voice emotion model, N entropy values ​​are determined, and the N entropy values ​​correspond one to one to the N voice texts; based on the N entropy values, the first target data is screened out from the N voice texts. The entropy value can characterize the uncertainty of the voice text input into the second voice emotion model, that is, the larger the entropy value, the greater the uncertainty of the voice text, and the greater the probability that the second voice emotion model predicts that the voice text is inaccurate, thereby characterizing that the voice text has a higher value for optimizing the second voice emotion model.

[0041] In an embodiment of the present invention, the second speech emotion model can be used to perform emotion prediction on the speech text to obtain a prediction result corresponding to the speech text, wherein the prediction result includes the emotion category into which the speech text is divided by the second speech emotion model, and the probability that the second speech emotion model divides the speech text into the emotion category.

[0042] In an embodiment of the present invention, the speech text is preprocessed by the second speech emotion model to obtain word encoding data corresponding to the speech text; and the word encoding data is subjected to emotion prediction by the second speech emotion model to obtain a prediction result corresponding to the word encoding data.

[0043] In an embodiment of the present invention, the preprocessing may be a process of converting the speech text data information into a word code in digital form, and the word code data may be a word code in digital form. For example, the preprocessing may refer to steps S2 to S3 described above, and the word code data may be data processed by step S3 described above.

[0044] In an embodiment of the present invention, the process of the second speech emotion model performing emotion prediction on the word encoding data can refer to step S4, and through the processing of the forget gate, input gate and output gate of the second speech emotion model, the speech text in the first target data is predicted to obtain the corresponding prediction result of the word encoding data.

[0045] In an embodiment of the present invention, the emotion categories included in the prediction result may be preset emotion categories before optimizing the second speech emotion model, and these emotion categories are represented respectively in the form of numerical labels. For example, the seven emotion categories described in step 110 can be represented by the number 1 for happiness, the number 2 for sadness, and so on. The prediction result may be based on the preset emotion categories, the emotion categories predicted by the second speech emotion model, and the corresponding probabilities. For example, the prediction result of a certain speech text may be: "1,50%; 2,40%; 3,5%; 4,5%".

[0046] In the embodiment of the present invention, after the prediction result is obtained, the entropy value of the speech text can be determined by the following formula: in, represents a voice text; i represents the emotion category that the second voice emotion model classifies the voice text into, for example, the number 1 represents happiness as described above. When the second voice emotion model predicts that the voice text represents happiness, i is 1; represents the probability that the second speech emotion model classifies the speech text as the i-th, such as the "1,50%" mentioned above, then at this time 50%; is the parameter space of the second speech emotion model; Represents the entropy value of the speech text corresponding to x.

[0047] In an embodiment of the present invention, after obtaining N entropy values, if the entropy value corresponding to the speech text is not less than the first threshold value, the speech text can be directly used as the first target data, and the speech text can also be manually annotated to obtain the speech text and the corresponding correct emotion category through manual annotation, and the speech text and the manual annotation result can be used together as the first target data.

[0048] In an embodiment of the present invention, after obtaining N entropy values, the first target data can be filtered out from the N voice texts according to other entropy value query strategies, for example, the N entropy values ​​can be sorted and the first target data can be filtered out according to a preset percentage, or the first target data can be filtered out according to a preset ratio. The embodiment of the present invention does not impose specific restrictions on this.

[0049] In an embodiment of the present invention, after the first target data is obtained by screening, step 130 can be executed. A specific implementation method of step 130 can be: based on the original speech text data and the first target data, a first training set and a first test set are obtained; the second speech emotion model is trained by the first training set to obtain an intermediate speech emotion model; the intermediate speech emotion model is verified by the first test set to obtain the accuracy of the intermediate speech emotion model; if the accuracy of the intermediate speech emotion model is greater than the second threshold, the intermediate speech emotion model is used as the target speech emotion model.

[0050] In an embodiment of the present invention, the first training set may include a portion of the original speech text data and a portion of the first target data randomly selected, and the first test set may include the original speech text data and the first target data other than the first training set. The process of training the second speech emotion model through the first training set may refer to steps S2 to S4 described above. After processing through step S4, an intermediate speech emotion model is obtained, and the accuracy of the intermediate speech emotion can be verified through the first test set. When the accuracy of the intermediate speech model is greater than the second threshold, the intermediate speech emotion model can be used as the target speech emotion model.

[0051] In an embodiment of the present invention, if the accuracy of the intermediate speech emotion model is not greater than the second threshold, the intermediate speech emotion model can be put online and put into use to collect the second speech text data of the user. Then, through the intermediate speech emotion model, the second target data is screened out from the second speech text data collected in the process of applying the intermediate speech emotion model for emotion prediction; the intermediate speech emotion model is optimized using the first target data and the second target data until the accuracy of the intermediate speech emotion model is greater than the second threshold. By continuously collecting user data and predicting and screening, active learning of the intermediate speech emotion model is achieved, and the accuracy of the optimized target speech emotion model is further improved.

[0052] In an embodiment of the present invention, after obtaining the target voice emotion model through the above-mentioned vehicle network user voice emotion model optimization method, the target voice emotion model can be used to perform emotion prediction on the target voice text data. An embodiment of the present invention also provides a method for predicting the voice emotion of a vehicle network user, comprising: obtaining target voice text data, the target voice text data being voice text data generated when the vehicle network user uses the target voice emotion model, the target voice emotion model being a model obtained through the above-mentioned vehicle network user voice emotion model optimization method; predicting the target emotion category to which the target voice text data belongs based on the preset emotion category through the target voice emotion model, the target emotion category coming from the preset emotion category.

[0053] In an embodiment of the present invention, the target voice text data may be any voice text data of a vehicle networking user, such as voice text data for navigation instructions, etc., and the present application embodiment does not impose specific restrictions on this. The preset emotion category may be the seven emotion categories in the above step S1. The process of predicting the target voice text data by the target voice emotion model may refer to the above steps S2 to S4, and the target voice text data may be analyzed through the gating mechanism of the optimized target voice emotion model to obtain the target emotion category.

[0054] In addition, in an embodiment of the present invention, a vehicle networking user voice emotion prediction system based on active learning is also provided. Figure 3 The architecture diagram of a vehicle networking user voice emotion prediction system provided by an embodiment of the present invention is as follows: Figure 3As shown, the vehicle networking user speech emotion prediction system provided by the embodiment of the present invention may include the following: a vehicle networking speech text acquisition unit, a speech text segmentation unit, a Word2vec word encoding unit, a speech emotion original classification model training unit, a speech emotion model optimization unit based on active learning, and a speech emotion model emotion classification unit. Among them, the second speech emotion model can be first obtained through the vehicle networking speech text acquisition unit, the speech text segmentation unit, the Word2vec word encoding unit, and the speech emotion original classification model training unit, and then the target speech emotion model is obtained by implementing the above-mentioned vehicle networking user speech emotion model optimization method through the active learning-based speech emotion model optimization unit, and the target speech emotion model is put online for application. After the target speech emotion model is put online, the target speech text data can be obtained through the vehicle networking speech text acquisition unit, and the target speech text data can be emotionally predicted through the speech text segmentation unit, the Word2vec word encoding unit, and the speech emotion model emotion classification unit.

[0055] 1. IoV voice text acquisition unit. In IoV voice conversations, users collect and integrate some of the vehicle-side voice data and some open-source voice emotion data. Some data can be displayed as follows: Figure 2 As shown in the figure, the relevant data includes the user's voice text and the emotion label corresponding to the relevant voice text. There are 4371 pieces of user emotion data in total, and the labels are divided into 7 categories: happy, sad, angry, calm, fearful, disgusted, and surprised. The emotion label corresponds to the user's real emotion in the voice text one by one, and can accurately reflect the user's emotional state label.

[0056] 2. Speech-text segmentation unit. In the speech-text segmentation, the embodiment of the present invention mainly uses Jieba to segment Chinese text. Jieba segmentation is a Chinese segmentation tool based on statistics and rules. It uses the following main technical means to achieve efficient and accurate Chinese segmentation. The specific theoretical process is shown in Table 2 below: Table 2 Paddle mode can be used in jieba word segmentation. The PaddlePaddle deep learning framework is used to train the sequence annotation (bidirectional GRU) network model to achieve word segmentation.

[0057] 3. Word2vec word encoding unit. Based on the previously cut word vectors, the word vectors are further encoded. The embodiment of the present invention adopts the CBOW model in Word2vec to perform word encoding operations (that is, during the training process, the model will learn the context information of each word, predict the intermediate word through the context of one or more words, and encode it into the word vector). Word2Vec encoding maps the words in the text to word vectors by training a neural network, captures the semantic relationship between words, and realizes text vectorization. Based on the idea of ​​distributed representation, the model converts the original word vector with a dimension of V (vocabulary size) into a word vector with a dimension of N (N is much smaller than V), while retaining the correlation between word vectors. Its training model is essentially a neural network with only one hidden layer. The input, output, and hidden layer description of the model are shown in Table 3 below: Table 3 In order to improve the training efficiency and the quality of word vectors, Word2Vec adopts the optimization techniques of hierarchical softmax (constructing a Huffman tree to speed up the softmax calculation process) and negative sampling (approximating the training target by randomly extracting negative samples).

[0058] 4. Speech emotion original classification model training unit. Based on the above encoding operation, the encoded data is used to train the speech emotion original classification model. The embodiment of the present invention adopts a long short-term memory network (LSTM) for speech emotion classification prediction. LSTM is a special recurrent neural network. Its original design is to solve the problem of gradient vanishing or gradient explosion when traditional RNN processes long sequence data, so that the network can learn long-term dependencies. LSTM introduces three gating mechanisms, namely forget gate, input gate and output gate. These gating mechanisms allow the network to dynamically determine the retention and forgetting of information. Through the gating mechanism, LSTM can effectively solve the problem of long-term dependency. LSTM shares the same weights at each time step of the sequence, which enables the model to process sequences of any length. The operation of the LSTM unit at each time step (t) can be described by the following formula, as shown in Table 1 described above.

[0059] To this end, the original data is divided into a training set and a test set. The data of the training set is used to train the LSTM model, and the data of the test set is used to verify the effect of the original classification prediction model, thereby obtaining the original classification model of speech emotion after preliminary training.

[0060] 5. Speech emotion model optimization unit based on active learning. The previously trained speech emotion original classification model (LSTM) can be further optimized by the vehicle networking user speech emotion model optimization method provided above. The specific model optimization process can be as follows: Figure 4 shown.

[0061] In an embodiment of the present invention, the user's Internet of Vehicles voice and text data will be continuously added. How to use the newly added valuable data to conduct a one-step training to strengthen the voice and text classification prediction model is of great significance to improving the accuracy of the model in predicting the user's voice emotion. The newly added voice and text data is screened out with extremely valuable new user voice and text data using active learning methods, and this part of the valuable newly added voice and text data is manually labeled using manual marking methods. These manually labeled new voice and text data are put into the classification model together with the original data to retrain the model, which will greatly improve the accuracy of the model in voice and text emotion prediction. Based on the above analysis, the specific active learning optimization model algorithm process is shown in Table 4 below: Table 4 Among them, the uncertainty query strategy is the core step of active learning. The embodiment of the present invention selects the entropy query strategy, which uses the entropy value to measure the uncertainty of the sample. The larger the entropy value, the greater the uncertainty of the sample, and the greater the probability of error in the prediction of the sample by the classification model, which further indicates that the value of manually marking the sample is very high. The entropy value measurement formula can be expressed as: In this formula Represents the new speech text emotion sample data, Represents the categories divided by the classification model, Indicates that the original LSTM model divides users into The probability of is the parameter space of the classification model, It is the entropy value corresponding to the solved behavior id.

[0062] In this regard, the embodiment of the present invention screened and manually marked a total of 173 new high-value speech text emotion data through the method of active learning entropy query strategy. The data is divided into 7 major emotion annotation sections, which correspond to the user's speech text emotion data in the Internet of Vehicles. The high-value speech text emotion data screened by active learning is used to train the LSTM classification model together with the original data, thereby obtaining a new LSTM classification model with higher accuracy.

[0063] Using the LSTM speech-text emotion classification model optimized by active learning to classify and predict the user's speech emotion can achieve better results. The following are some specific examples of predicting the user's speech emotion through the final optimized model in the embodiment of the present invention, as shown in Table 5: Table 5 In the embodiment of the present invention, the speech emotion text data in the Internet of Vehicles is segmented and encoded, and the processed data is trained for prediction and classification using the LSTM classification model. For the trained classification model, there will be continuous input of user speech text data in the later stage. The active learning method is used to screen out the extremely valuable part of the user speech text data, and the emotional category of this part of the valuable speech text data is annotated by the manual labeling method. These manually annotated data and the original data are put into the classification model together for re-model training, thereby improving the accuracy and robustness of the model classification prediction. It is intended to provide a more accurate speech emotion analysis prediction model for Internet of Vehicles users, improve the accuracy of related speech emotion prediction classification, and enhance user experience.

[0064] Figure 5 This is a structural block diagram of a device for optimizing a user speech emotion model of an Internet of Vehicles provided by an embodiment of the present invention. Figure 5 As shown, the Internet of Vehicles user voice emotion model optimization device 500 provided in an embodiment of the present invention includes: an acquisition module 510, a screening module 520 and an optimization module 530.

[0065] An acquisition module 510 is used to acquire original speech text data used to train a first speech emotion model, and first speech text data collected in the process of applying a second speech emotion model to perform emotion prediction, wherein the second speech emotion model is obtained by training the first speech emotion model with the original speech text data; A screening module 520, configured to screen out first target data from the first speech text data by using the second speech emotion model; The optimization module 530 is used to optimize the second speech emotion model using the original speech text data and the first target data to obtain a target speech emotion model, and the target speech emotion model is used to perform emotion prediction on the speech text data of the Internet of Vehicles user.

[0066] It should be noted that the embodiment of the vehicle network user voice emotion model optimization device in this specification and the embodiment of the vehicle network user voice emotion model optimization method in this specification are based on the same inventive concept. Therefore, the specific implementation method of this embodiment can refer to the corresponding embodiment of the vehicle network user voice emotion model optimization method in the previous text, and the repeated parts will not be repeated.

[0067] In an exemplary embodiment, an electronic device including a processor and a memory is also provided. The electronic device can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0068] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the instructions can be executed by a processor of the device to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the electronic device, enables the electronic device to perform Figure 1 The method for optimizing the voice emotion model of a vehicle network user and the method for predicting the voice emotion of a vehicle network user provided in an embodiment of the present invention are shown.

[0069] The present invention also provides a computer program product, comprising a computer program, which when executed by a processor Figure 1 The method for optimizing the voice emotion model of a vehicle network user and the method for predicting the voice emotion of a vehicle network user provided in an embodiment of the present invention are shown.

[0070] In the above description, the technical details such as the patterning of each layer are not described in detail. However, those skilled in the art should understand that various technical means can be used to form layers, regions, etc. of desired shapes. In addition, in order to form the same structure, those skilled in the art can also design methods that are not completely the same as the methods described above. In addition, although the various embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage.

[0071] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0072] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for optimizing a voice emotion model of an Internet of Vehicles user, characterized in that: The method comprises: Acquire original speech text data used to train a first speech emotion model, and first speech text data collected during the process of applying a second speech emotion model to perform emotion prediction, wherein the second speech emotion model is obtained by training the first speech emotion model with the original speech text data; Filtering first target data from the first speech text data by using the second speech emotion model; The second speech emotion model is optimized using the original speech text data and the first target data to obtain a target speech emotion model, and the target speech emotion model is used to perform emotion prediction on the speech text data of the Internet of Vehicles user.

2. The method according to claim 1, characterized in that The first speech text data includes N speech texts; and the filtering out first target data from the first speech text data by using the second speech emotion model includes: According to the prediction results obtained by sequentially inputting the second voice emotion model into the N voice texts, N entropy values ​​are determined, and the N entropy values ​​correspond to the N voice texts one by one; Based on the N entropy values, first target data is filtered out from the N voice texts.

3. The method according to claim 2, characterized in that Determining N entropy values ​​according to the prediction results obtained by sequentially inputting the second voice emotion model into the N voice texts includes: For each voice text, perform emotion prediction on the voice text by using the second voice emotion model to obtain a prediction result corresponding to the voice text, wherein the prediction result includes the emotion category into which the voice text is divided by the second voice emotion model, and the probability that the voice text is divided into the emotion category by the second voice emotion model; Based on the N prediction results corresponding to the N voice texts, N entropy values ​​are determined.

4. The method according to claim 2, characterized in that: The step of screening out first target data from the N voice texts based on the N entropy values ​​includes: If the entropy value corresponding to the speech text is not less than the first threshold, the speech text is used as the first target data.

5. The method according to claim 3, characterized in that: The performing emotion prediction on the speech text by using the second speech emotion model to obtain a prediction result corresponding to the speech text includes: Preprocessing the speech text by using the second speech emotion model to obtain word encoding data corresponding to the speech text; The emotion prediction of the word encoding data is performed using the second speech emotion model to obtain a prediction result corresponding to the word encoding data.

6. The method according to claim 1, characterized in that The step of optimizing the second speech emotion model by using the original speech text data and the first target data to obtain an optimized target speech emotion model includes: Based on the original speech text data and the first target data, obtaining a first training set and a first test set; Training the second speech emotion model through the first training set to obtain an intermediate speech emotion model; Verifying the intermediate speech emotion model through the first test set to obtain the accuracy of the intermediate speech emotion model; If the accuracy of the intermediate speech emotion model is greater than the second threshold, the intermediate speech emotion model is used as the target speech emotion model.

7. The method according to claim 6, characterized in that After verifying the intermediate speech emotion model through the first test set and obtaining the accuracy of the intermediate speech emotion model, the method further includes: If the accuracy of the intermediate speech emotion model is not greater than the second threshold, second target data is screened out from the second speech text data collected in the process of applying the intermediate speech emotion model for emotion prediction by using the intermediate speech emotion model; The intermediate speech emotion model is optimized using the first target data and the second target data until the accuracy of the intermediate speech emotion model is greater than a second threshold.

8. A method for predicting speech emotion of a vehicle network user, characterized in that: include: Acquire target speech text data, wherein the target speech text data is speech text data generated when a vehicle networking user uses a target speech emotion model, and the target speech emotion model is a model optimized by the method described in any one of claims 1 to 7; The target speech emotion model is used to predict the target emotion category to which the target speech text data belongs based on the preset emotion category, and the target emotion category comes from the preset emotion category.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction running on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 7 and the steps of the method according to claim 8 are implemented.

10. A computer-readable storage medium, characterized in that: The medium stores a program or an instruction, and when the program or the instruction is executed, the steps of the method according to any one of claims 1 to 7 and the steps of the method according to claim 8 are implemented.

Images