Voice-based psychological state recognition method, device and equipment

Abstract

The invention provides a voice-based psychological state recognition method, device and equipment. The method comprises the following steps: acquiring voice data of a user; preprocessing the voice data to obtain target voice data; extracting voice features of the target voice data; inputting the voice features into a pre-trained voice psychological state model; based on the voice psychological state model, the psychological state of the user corresponding to the voice feature is determined, clinical evaluation of professional doctors is not depended on any more, consumed time is short, efficiency is high, and popularization and promotion are facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of user psychological state recognition, and in particular to a method, device and equipment for recognizing a psychological state based on speech. Background Art

[0002] As people's pace of life accelerates, people's mental and psychological problems are increasing year by year, bringing huge life burdens and psychological torture to patients and their families. The assessment of psychological symptoms is an important part of the diagnosis and treatment of psychological problems. However, the current assessment of mental and psychological problems mainly relies on the clinical assessment of professional doctors.

[0003] The method of having professional doctors assess people's psychological symptoms is not only time-consuming and inefficient, but also has high requirements on the professional quality of doctors, and is difficult to popularize and promote. Summary of the invention

[0004] The present invention provides a method, device and equipment for recognizing psychological state based on speech, so as to solve the technical problem that people's psychological state cannot be quickly evaluated in the prior art.

[0005] In one aspect, the present invention provides a method for recognizing a psychological state based on speech, comprising: Get the user's voice data; Preprocessing the voice data to obtain target voice data; Extracting speech features of the target speech data; Inputting the speech features into a pre-trained speech psychological state model; Based on the speech psychological state model, the psychological state of the user corresponding to the speech feature is determined.

[0006] According to a speech-based psychological state recognition method provided by the present invention, the speech psychological state model is trained in the following manner: Obtaining voice sample data of sample users; Preprocessing the speech sample data to obtain sample processing data; Annotating the psychological symptoms of the sample processing data to obtain sample annotated data; Extracting speech sample features of the sample annotation data; Based on the psychological symptoms, classifying the speech sample features to obtain speech classification features; Inputting the speech classification features into the speech psychological state model to train the speech psychological state model; Evaluate the trained speech psychological state model.

[0007] According to a speech-based psychological state recognition method provided by the present invention, the speech sample features include at least one of the following: Basic features, pitch features, spectrum features and nonlinear features; Wherein, the basic characteristics include zero crossing rate, energy, and zero crossing rate; The fundamental pitch characteristics include fundamental pitch period and fundamental pitch frequency; The spectral features include Mel frequency cepstral coefficients, perceptual linear prediction coefficients, and linear prediction coding; The nonlinear characteristics include: HNR, Jitter, and Shimmer.

[0008] According to a speech-based psychological state recognition method provided by the present invention, the psychological symptoms of the sample processing data are labeled to obtain sample labeled data, including: identifying and annotating emotions in the sample processing data; Converting the sample processed data into text data; Performing sentiment classification on the text data to obtain various types of sentiment text data; Enhance and verify the various types of emotional text data to obtain sample annotation data.

[0009] According to a speech-based psychological state recognition method provided by the present invention, the speech psychological state model is established in the following manner: Establishing the speech psychological state model based on convolutional neural network and recurrent neural network; Wherein, the convolutional neural network and the recurrent neural network are in a series connection state; The convolutional neural network is used to mine local spatial features of audio; The recurrent neural network is used to mine the time series features of audio.

[0010] According to a speech-based psychological state recognition method provided by the present invention, the speech psychological state model after evaluation training includes: Determine the true positive examples, true negative examples, false positive examples and false negative examples obtained by processing the speech classification features by the speech psychological state model; Adding the true example and the true counterexample to obtain a first addition result; Adding the true positive example, the true negative example, the false positive example, and the false negative example to obtain a second addition result; determining a ratio of the first addition result to the second addition result; The trained speech psychological state model is evaluated based on the ratio; wherein, The true positive example represents a speech classification feature whose mental state is actually positive and is predicted to be positive; The true negative example indicates that the mental state is actually negative and the predicted speech classification feature is negative; The false positive example represents a speech classification feature whose mental state is actually negative but is predicted to be positive; The false negative examples represent speech classification features whose mental states are actually positive but predicted to be negative.

[0011] According to a speech-based psychological state recognition method provided by the present invention, the speech psychological state model after evaluation training further includes: Determining the accuracy of the speech psychological state model based on the true positive examples and the false positive examples; Determining a recall rate of the speech psychological state model based on the true positive examples and the false negative examples; The trained speech psychological state model is evaluated based on the precision and the recall.

[0012] According to a speech-based psychological state recognition method provided by the present invention, the accuracy of the speech psychological state model is determined based on the true positive examples and the false positive examples, including the following formula: Precision = True Positives / (True Positives + False Positives); The recall rate of the speech psychological state model is determined based on the true examples and the false negative examples, including the following formula: Recall rate = true positive examples / (true positive examples + false negative examples); The trained speech psychological state model is evaluated based on the precision and the recall rate, and includes the following formula: Evaluation result = 2×(precision*recall) / (precision+recall).

[0013] On the other hand, the present invention also provides a speech-based psychological state recognition device, comprising: An acquisition module, used to acquire the user's voice data; A preprocessing module, used for preprocessing the voice data to obtain target voice data; An extraction module, used for extracting speech features of the target speech data; An input module, used for inputting the speech features into a pre-trained speech psychological state model; An evaluation module is used to determine the user's psychological state corresponding to the speech feature based on the speech psychological state model.

[0014] On the other hand, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements any of the above-mentioned speech-based psychological state recognition methods.

[0015] The method, device and equipment for recognizing the psychological state based on speech provided by the present invention obtain the speech data of the user; pre-process the speech data to obtain the target speech data; extract the speech features of the target speech data; input the speech features into a pre-trained speech psychological state model; and determine the psychological state of the user corresponding to the speech features based on the speech psychological state model. The method, device and equipment no longer rely on the clinical evaluation of professional doctors, and is time-saving, efficient and easy to popularize and promote. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 is a flow chart of a method for recognizing a psychological state based on speech provided by an embodiment of the present invention; Figure 2 is a schematic diagram of the structure of a speech-based psychological state recognition device provided by an embodiment of the present invention; Figure 3 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Figure 1 The flowchart of the method for recognizing a psychological state based on speech provided by an embodiment of the present invention is shown in FIG. The method may be executed by a computer, a mobile phone, a smart wearable device, etc.

[0020] See also Figure 1 , the method for recognizing psychological state based on speech may include the following steps 101 to 105.

[0021] 101. Obtain the user's voice data.

[0022] In this step, the user's voice data can be collected through recording equipment and online platforms. When obtaining the user's voice data, the user can also be placed in a preset range to directly obtain the voice data emitted by the user. In addition, considering that people of different ages, occupations, and genders may have different psychological states, while collecting the user's voice data, the user's identity information can also be collected. For example, the identity information may include age, gender, occupation, etc. These data can be stored locally or on a server, and a connection is established with the server through the network to obtain the above data.

[0023] 102. Preprocess the voice data to obtain target voice data.

[0024] In this step, the preprocessing may mainly include at least one of noise reduction, frame segmentation, feature extraction, and standardization.

[0025] Noise reduction: Noise reduction refers to reducing or eliminating background noise in speech data, such as wind noise, traffic noise, air conditioning noise, etc. It can be achieved through various algorithms, such as spectral subtraction, Wiener filtering, etc. The purpose of noise reduction is to make the speech signal clearer and reduce the impact of noise on speech recognition and analysis.

[0026] Framing: Speech signals are continuous waveforms. For analysis, they need to be divided into shorter time periods. This process is called framing. Usually, each frame contains tens of milliseconds of speech data. The purpose of framing is to allow local characteristics of speech signals (such as phonemes) to be analyzed separately.

[0027] Feature extraction: Feature extraction is to extract information useful for speech recognition from the framed speech signal, such as Mel-frequency cepstral coefficients (MFCC), Mel-frequency band energy (MFB), etc. These features can represent certain characteristics of the speech signal, such as pitch, timbre, etc., and are insensitive to changes in the speaker's voice.

[0028] Standardization: Standardization refers to converting speech data collected from different sources or under different conditions into a unified format or scale. The purpose of this is to eliminate inconsistencies between data and make the features of different data sets comparable. For example, the sampling rate of all speech data can be unified, or the feature values ​​can be normalized.

[0029] 103. Extract speech features of the target speech data.

[0030] In this step, the speech features of the target speech data may be extracted based on COVAREP.

[0031] 104. Input the speech features into the pre-trained speech psychological state model.

[0032] In this step, when the user's identity information is collected, the voice features and the identity information can be input into the pre-trained voice psychological state model together.

[0033] The speech psychological state model can be trained by: Obtaining voice sample data of sample users; Preprocessing the speech sample data to obtain sample processing data; Annotating the psychological symptoms of the sample processing data to obtain sample annotated data; Extracting speech sample features of the sample annotation data; Based on the psychological symptoms, classifying the speech sample features to obtain speech classification features; Inputting the speech classification features into the speech psychological state model to train the speech psychological state model; Evaluate the trained speech psychological state model.

[0034] Specifically, when the psychological symptoms of the sample processing data are annotated, the psychological symptoms may include depression, anxiety, mania, etc. The method of extracting the speech sample features of the sample annotated data may refer to the above step 103 .

[0035] 105. Determine the user's psychological state corresponding to the speech feature based on the speech psychological state model.

[0036] In this step, considering that the psychological states of people of different ages, occupations, and genders may be different, the voice features and identity information are input together into a pre-trained voice psychological state model, so that the psychological state of the user corresponding to the voice features and identity information can be determined, which is conducive to further improving the accuracy of assessing the user's psychological state.

[0037] In this embodiment, the user's voice data is obtained; the voice data is preprocessed to obtain target voice data; the voice features of the target voice data are extracted; the voice features are input into a pre-trained voice psychological state model; based on the voice psychological state model, the user's psychological state corresponding to the voice features is determined. This method no longer relies on the clinical evaluation of professional doctors, is time-saving, efficient, and easy to popularize and promote.

[0038] In one embodiment of the present specification, the voice sample feature includes at least one of the following: Basic features, pitch features, spectrum features and nonlinear features; The basic characteristics include zero crossing rate (ZCR), energy, and zero crossing rate; The fundamental pitch characteristics include a fundamental pitch period (Pitch Period) and a fundamental pitch frequency (Pitch Frequency); The spectral features include Mel-frequency cepstral coefficients (MFCCs), perceptual linear prediction coefficients (PLPs), and linear predictive coding (LPC); The nonlinear characteristics include: HNR (Harmonic-to-Noise Ratio), Jitter, and Shimmer.

[0039] In this embodiment, the above-mentioned sound features are extracted for evaluating the user's mental and psychological state because these features can reflect multiple attributes of the sound, which are closely related to the individual's emotions and psychological state. By analyzing these features, the user's mental and psychological state can be evaluated because they can reveal the physiological and psychological changes behind the sound and improve the accuracy of the evaluation of the user's mental and psychological state. HNR (Harmonic-to-Noise Ratio): Harmonic-to-Noise Ratio is a ratio that measures the ratio of harmonic components to noise components in a speech signal. It is an objective indicator used to detect pathological voices and evaluate voice quality, and can effectively reflect the glottal closure. Jitter: Frequency perturbation, which describes the change in the basic frequency of sound waves between adjacent cycles. It mainly reflects the degree of roughness and secondly the degree of hoarseness. It is consistent with the functional state of the glottal area. The frequencies of normal voice cycles are more the same and less different, so the frequency perturbation value is very small. When vocal cord lesions occur, the perturbation value increases, making the sound rough. Shimmer: Amplitude perturbation, which describes the change in the amplitude of sound waves between adjacent cycles, mainly reflecting the degree of hoarseness. Jitter and shimmer together reflect the stability of vocal cord vibration. The smaller the value, the fewer subtle changes in the acoustic signal during the phonation process.

[0040] In one embodiment of the present specification, the labeling of the psychological symptoms of the sample processing data to obtain the sample labeling data includes: identifying and annotating emotions in the sample processing data; Converting the sample processed data into text data; Performing sentiment classification on the text data to obtain various types of sentiment text data; Enhance and verify the various types of emotional text data to obtain sample annotation data.

[0041] In this embodiment, emotions may include anger, disgust, worry, happiness, sadness, surprise, nature and other emotions. Specifically, the emotions in the sample processing data can be identified and labeled based on Emotion2vec. Emotion2vec is a speech emotion representation model. Through self-supervised pre-training, Emotion2vec can extract emotion representations in different tasks, languages ​​and scenarios. The sample processing data can be converted into text data based on FunASR.

[0042] The emotional text is automatically annotated as positive, negative and neutral text emotions using the method of emotional dictionary. The text data can be sentimentally classified based on StructBERT to obtain various types of emotional text data. The StructBERT Chinese sentiment classification model infers the Chinese sentiment classification task in the general field. When the natural language text is input, the model will give the sentiment classification label (0,1) of the text and the corresponding probability. Based on the StructBERT tool, the various types of emotional text data can be enhanced and verified based on the Chinese Sentiment Classification Tool to obtain sample annotation data. The Chinese Sentiment Polarity Analysis Tool has good generalization, strong rule scalability, and is universal in all fields. The tool can automatically determine the sentiment polarity category of the input text and give the corresponding confidence. Sentiment polarity is divided into positive, negative and neutral. The sample annotation data obtained in this embodiment is very accurate and can cover people's various emotions very comprehensively, that is, it can cover people's psychological state very comprehensively.

[0043] In one embodiment of the present specification, the speech psychological state model is established in the following manner: Establishing the speech psychological state model based on convolutional neural network and recurrent neural network; Wherein, the convolutional neural network and the recurrent neural network are in a series connection state; The convolutional neural network is used to mine local spatial features of audio; The recurrent neural network is used to mine the time series features of audio.

[0044] In this embodiment, the convolutional neural network and the recurrent neural network are in series, that is, the output of the convolutional neural network can be used as the input of the recurrent neural network. The speech psychological symptom model constructed by combining the convolutional neural network (CNN) and the recurrent neural network (RNN) can more effectively identify and understand the audio features related to psychological symptoms, thereby improving the accuracy and reliability of psychological state prediction.

[0045] Specifically, the advantages of this embodiment are as follows. Dual feature extraction: Convolutional neural networks (CNNs) are good at mining local spatial features of audio, while recurrent neural networks (RNNs) are suitable for capturing the time series features of audio. By connecting these two networks in series, the deep temporal and spatial features in audio that are associated with the characteristics of psychological symptoms can be learned more comprehensively. Improving recognition accuracy: Combining CNN and RNN can make full use of the spatial and temporal information of audio signals and improve the recognition accuracy of patients with psychological symptoms. Processing complex data: The combination of CNN and RNN can process complex data with spatial and temporal structures, such as video data, sensor data, etc., which is difficult to achieve in a single network. Parameter sharing and local perception ability: RCNN realizes parameter sharing through convolutional layers, reduces the number of model parameters, and improves training efficiency and generalization ability. At the same time, the local perception ability of the convolutional layer is combined with the temporal information integration ability of the recurrent layer, so that the model has a good perception ability for both local information and long-distance dependencies. Ability to process sequence data: RCNN can effectively process sequence data of indefinite length, such as natural language text, time series, etc. By introducing a recurrent structure, the model can capture contextual information in the sequence. Wide applicability: Machine learning has a wide range of practical application scenarios in the field of speech and audio processing, including speech recognition, speech synthesis, audio processing, and audio recognition, which shows the wide applicability of the model.

[0046] In one embodiment of the present specification, the evaluating the trained speech psychological state model includes: Determine the true positive examples, true negative examples, false positive examples and false negative examples obtained by processing the speech classification features by the speech psychological state model; Adding the true example and the true counterexample to obtain a first addition result; Adding the true positive example, the true negative example, the false positive example, and the false negative example to obtain a second addition result; determining a ratio of the first addition result to the second addition result; The trained speech psychological state model is evaluated based on the ratio; wherein, The true positive example represents a speech classification feature whose mental state is actually positive and is predicted to be positive; The true negative example indicates that the mental state is actually negative and the predicted speech classification feature is negative; The false positive example represents a speech classification feature whose mental state is actually negative but is predicted to be positive; The false negative examples represent speech classification features whose mental states are actually positive but predicted to be negative.

[0047] In this embodiment, the mental state is actually negative, which may specifically include depression, anxiety, bipolar disorder, schizophrenia, personality disorder, etc. The mental state is actually positive, which may specifically include states other than the mental state is actually negative, such as a normal state (the individual has no obvious mental health problems, is emotionally stable, and can carry out daily activities normally), a positive emotion (the individual shows a positive emotional state, such as happiness, excitement, or satisfaction), a high energy level (the individual is full of energy, energetic, and has no fatigue or burnout), etc. The method of evaluating the trained speech psychological state model has the advantages of comprehensiveness (by considering the four situations of true positive examples (TP), true negative examples (TN), false positive examples (FP), and false negative examples (FN), the performance of the model can be comprehensively evaluated, rather than just focusing on a single indicator), accuracy evaluation (by calculating the first addition result (TP+TN), the total correct prediction number of the model can be obtained, which helps to evaluate the overall accuracy of the model), and identification of misclassification (by calculating the second addition result (TP+TN+FP+FN), the total number of all samples processed by the model can be obtained, thereby identifying the total number of misclassifications of the model (FP+FN)). This evaluation method is simple and comprehensive.

[0048] In one embodiment of the present specification, the evaluating the trained speech psychological state model further includes: Determining the accuracy of the speech psychological state model based on the true positive examples and the false positive examples; Determining a recall rate of the speech psychological state model based on the true positive examples and the false negative examples; The trained speech psychological state model is evaluated based on the precision and the recall.

[0049] In one embodiment of the present specification, the accuracy of the speech psychological state model is determined based on the true positive examples and the false positive examples, including the following formula: Precision = True Positives / (True Positives + False Positives); The recall rate of the speech psychological state model is determined based on the true examples and the false negative examples, including the following formula: Recall rate = true positive examples / (true positive examples + false negative examples); The trained speech psychological state model is evaluated based on the precision and the recall rate, and includes the following formula: Evaluation result = 2×(precision*recall) / (precision+recall).

[0050] In this embodiment, precision is the proportion of samples that are actually positive, which measures the proportion of samples predicted by the model as positive. Recall measures the proportion of samples that are actually positive that are correctly predicted by the model as positive. This embodiment has multiple advantages, such as the balance between precision and recall: precision measures the proportion of samples that are actually positive among samples predicted by the model, while recall measures the proportion of samples that are actually positive among samples that are correctly predicted by the model as positive. These two indicators provide different perspectives on model performance and help us understand the performance of the model in different aspects. The evaluation result is the harmonic mean of precision and recall, which takes both precision and recall into account and provides a single indicator to measure the overall performance of the model. This helps to find a balance between precision and recall, especially when there is a trade-off between the two indicators.

[0051] Based on the same general inventive concept, the present invention also protects a speech-based psychological state recognition device, such as Figure 2 As shown, Figure 2 1 is a schematic diagram of the structure of the speech-based psychological state recognition device provided by an embodiment of the present invention. The speech-based psychological state recognition device provided by the present invention is described below, and the speech-based psychological state recognition device described below and the speech-based psychological state recognition method described above can be referred to each other.

[0052] The speech-based psychological state recognition device includes an acquisition module 201 , a preprocessing module 202 , an extraction module 203 , an input module 204 and an evaluation module 205 .

[0053] The acquisition module 201 is used to acquire the user's voice data; The preprocessing module 202 is used to preprocess the voice data to obtain target voice data; The extraction module 203 is used to extract the speech features of the target speech data; The input module 204 is used to input the speech features into the pre-trained speech psychological state model; The evaluation module 205 is used to determine the user's psychological state corresponding to the speech feature based on the speech psychological state model.

[0054] Figure 3 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0055] like Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the recognition method of the mental state based on speech.

[0056] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0057] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the speech-based psychological state recognition method provided by the above-mentioned methods.

[0058] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the speech-based psychological state recognition method provided by the above-mentioned methods.

[0059] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0060] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying a psychological state based on speech, characterized in that: include: Get the user's voice data; Preprocessing the voice data to obtain target voice data; Extracting speech features of the target speech data; Inputting the speech features into a pre-trained speech psychological state model; Based on the speech psychological state model, the psychological state of the user corresponding to the speech feature is determined.

2. The method for recognizing a psychological state based on speech according to claim 1, characterized in that: The speech psychological state model is trained in the following way: Obtaining voice sample data of sample users; Preprocessing the speech sample data to obtain sample processing data; Annotating the psychological symptoms of the sample processing data to obtain sample annotated data; Extracting speech sample features of the sample annotation data; Based on the psychological symptoms, classifying the speech sample features to obtain speech classification features; Inputting the speech classification features into the speech psychological state model to train the speech psychological state model; Evaluate the trained speech psychological state model.

3. The method for recognizing a psychological state based on speech according to claim 2, characterized in that: The speech sample features include at least one of the following: Basic features, pitch features, spectrum features and nonlinear features; Wherein, the basic characteristics include zero crossing rate, energy, and zero crossing rate; The fundamental pitch characteristics include the fundamental pitch period and the fundamental pitch frequency; The spectral features include Mel frequency cepstral coefficients, perceptual linear prediction coefficients, and linear prediction coding; The nonlinear characteristics include: HNR, Jitter, and Shimmer.

4. The method for recognizing a psychological state based on speech according to claim 2, characterized in that: The step of labeling the psychological symptoms of the sample processing data to obtain sample labeling data includes: identifying and annotating emotions in the sample processing data; Converting the sample processed data into text data; Performing sentiment classification on the text data to obtain various types of sentiment text data; Enhance and verify the various types of emotional text data to obtain sample annotation data.

5. The method for recognizing a psychological state based on speech according to claim 2, characterized in that: The speech psychological state model is established in the following way: Establishing the speech psychological state model based on convolutional neural network and recurrent neural network; Wherein, the convolutional neural network and the recurrent neural network are in a series connection state; The convolutional neural network is used to mine local spatial features of audio; The recurrent neural network is used to mine the time series features of audio.

6. The method for recognizing a psychological state based on speech according to claim 2, characterized in that: The method of evaluating the trained speech psychological state model comprises: Determine the true positive examples, true negative examples, false positive examples and false negative examples obtained by processing the speech classification features by the speech psychological state model; Adding the true example and the true counterexample to obtain a first addition result; Adding the true positive example, the true negative example, the false positive example, and the false negative example to obtain a second addition result; determining a ratio of the first addition result to the second addition result; The trained speech psychological state model is evaluated based on the ratio; wherein, The true positive example represents a speech classification feature whose mental state is actually positive and is predicted to be positive; The true negative example indicates that the mental state is actually negative and the predicted speech classification feature is negative; The false positive example represents a speech classification feature whose mental state is actually negative but is predicted to be positive; The false negative examples represent speech classification features whose mental states are actually positive but predicted to be negative.

7. The method for recognizing a psychological state based on speech according to claim 6, characterized in that: The speech psychological state model after evaluating the training also includes: Determining the accuracy of the speech psychological state model based on the true positive examples and the false positive examples; Determining a recall rate of the speech psychological state model based on the true positive examples and the false negative examples; The trained speech psychological state model is evaluated based on the precision and the recall.

8. The method for recognizing a psychological state based on speech according to claim 7, characterized in that: The method of determining the accuracy of the speech psychological state model based on the true positive examples and the false positive examples comprises the following formula: Precision = True Positives / (True Positives + False Positives); The recall rate of the speech psychological state model is determined based on the true examples and the false negative examples, including the following formula: Recall rate = true positive examples / (true positive examples + false negative examples); The trained speech psychological state model is evaluated based on the precision and the recall rate, and includes the following formula: Evaluation result = 2×(precision*recall) / (precision+recall).

9. A device for identifying a psychological state based on speech, characterized in that: include: An acquisition module, used to acquire the user's voice data; A preprocessing module, used for preprocessing the voice data to obtain target voice data; An extraction module, used for extracting speech features of the target speech data; An input module, used for inputting the speech features into a pre-trained speech psychological state model; An evaluation module is used to determine the user's psychological state corresponding to the speech feature based on the speech psychological state model.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for recognizing a mental state based on speech as described in any one of claims 1 to 8 is implemented.

Images