Depression speech recognition method and system based on deep learning

Abstract

The present invention relates to the technical field of depression recognition, and in particular to a method and system for speech recognition of depression based on deep learning. The present invention clusters speech data of a training individual to obtain speech clusters, adjusts the elements in the characteristic sequence of speech attributes of the speech data based on the number of speech data in the speech cluster, and obtains the characteristic speech value of each speech attribute; constructs a characteristic word segmentation set of the training individual according to the number of occurrences of the same word segmentation in the speech cluster to which it belongs in the word cluster set of the training individual, and the number of speech clusters in which each word segmentation exists; trains a neural network in combination with the characteristic speech value and the characteristic word segmentation set of the speech attribute of the training individual, and uses the trained neural network to detect depression in the individual to be tested. The present invention trains a neural network by combining the speech features and semantic features of the language of the training individual to improve the accuracy of depression detection.

Description

Technical Field

[0001] The present invention relates to the technical field of depression recognition, and in particular to a method and system for depression speech recognition based on deep learning. Background Art

[0002] Depression is a common mental disorder characterized by persistent sadness, loss of interest or pleasure, and other symptoms. The diagnosis of depression usually relies on the subjective assessment of clinicians, which may lead to inaccurate diagnosis of depression. Deep learning is a machine learning technology that can automatically learn feature representations from large amounts of data. In the field of depression identification, deep learning is used to analyze various types of data such as patients' speech, text, and physiological signals to achieve the diagnosis of depression.

[0003] Existing technologies extract features from the patient's voice, such as pitch, rhythm, and volume, and use neural networks to analyze the differences between people with depression and healthy people, thereby diagnosing depression based on the patient's voice features. However, since the elderly and patients with chronic diseases have similar physiological or psychological states to those with depression due to their own reasons, the voice features of non-depressed people and those with depression are similar, resulting in a low accuracy rate in diagnosing depression. Summary of the invention

[0004] In order to solve the technical problem that the voice features of non-depressed people and depressed people are similar, resulting in a low accuracy rate in depression diagnosis, the purpose of the present invention is to provide a method and system for depression speech recognition based on deep learning. The technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present invention provides a method for speech recognition of depression based on deep learning, the method comprising:

[0006] Acquire speech data of the training individual and the test individual, as well as a word segmentation set of each speech data and a feature sequence of each speech attribute;

[0007] Clustering all speech data of the training individual according to the difference in elements and the difference in element fluctuation degree in the feature sequence of the same speech attribute of any two speech data of the training individual to obtain a speech cluster of the training individual; adjusting the elements in the feature sequence of each speech attribute of the speech data based on the number of speech data in the speech cluster to obtain a characteristic speech value of each speech attribute of the training individual;

[0008] The set of segmented words in the segmented word set of speech data in each speech cluster is recorded as a cluster set; according to the number of occurrences of the same segmented word in the speech cluster to which it belongs in the cluster set of all speech clusters of the training individual, and the number of speech clusters of each segmented word in the cluster set of speech clusters of the training individual, a characteristic segmented word set of the training individual is constructed;

[0009] The neural network is trained by combining the characteristic speech values ​​of all speech attributes of the training individual and the characteristic word segmentation set, and the trained neural network is used to detect depression in the individual to be tested.

[0010] Furthermore, the method for acquiring speech clusters of the training individual includes:

[0011] For each feature sequence of each voice attribute of each voice data, the mean of the difference between the remaining elements in the feature sequence except the first element and its adjacent previous element is calculated as the fluctuation concentration value; the mean of all elements in the feature sequence is taken as the feature concentration value;

[0012] Obtaining a feature distance between any two speech data of the training individual according to a difference between the fluctuation concentration value and the feature concentration value of the same speech attribute of any two speech data of the training individual;

[0013] All speech data of the training individuals are clustered based on the characteristic distance, and the obtained clusters are recorded as speech clusters.

[0014] Furthermore, the step of obtaining the characteristic distance between any two speech data of the training individuals includes:

[0015] The product of the fluctuation concentrated value and the feature concentrated value of each voice attribute of each voice data of the training individual is used as the fluctuation feature value of each voice attribute of each voice data of the training individual;

[0016] The average of the absolute values ​​of the differences between the fluctuation characteristic values ​​of the same speech attribute of any two speech data of the training individual is calculated as the characteristic distance between any two speech data of the training individual.

[0017] Furthermore, the step of obtaining the characteristic speech value of each speech attribute of the training individual includes:

[0018] The mode of all elements in the feature sequence of each speech attribute of the speech data in each speech cluster is used as the local feature value of each speech attribute of the corresponding speech cluster;

[0019] Arrange all speech clusters of the training individual in ascending order according to the total number of speech data in the speech cluster to obtain a cluster sequence; obtain the subscript value of each speech cluster in the cluster sequence and record it as the feature weight of each speech cluster;

[0020] The local feature values ​​of each speech attribute of each speech cluster are weighted by using the feature weights to obtain weighted feature values ​​of each speech attribute of each speech cluster; and the average of the weighted feature values ​​of each speech attribute of all speech clusters of the training individual is calculated as the feature speech value of each speech attribute of the training individual.

[0021] Furthermore, the step of constructing a feature segmentation set of training individuals includes:

[0022] Obtaining the commonness of each word in the word cluster set of each speech cluster of the training individual according to the number of occurrences of each word in the corresponding speech cluster and the number of speech data of each word in the word cluster set in each speech cluster;

[0023] According to the common degree of the same segmented word in the word cluster set of all speech clusters of the training individual and the number of speech clusters of each segmented word in the word cluster set where speech clusters exist, the semantic feature value of each segmented word in the word cluster set of all speech clusters of the training individual is obtained;

[0024] For the semantic feature values ​​of the segmented words in the segmented word set of all speech data of the training individual, a set consisting of segmented words corresponding to a maximum preset number of the semantic feature values ​​is used as the feature segmented word set of the training individual.

[0025] Furthermore, the obtaining of the commonness of each word in the word cluster set of each speech cluster of the training individual includes:

[0026] For each speech cluster of the training individual, select one segmentation from the word cluster set of the speech cluster as the example segmentation;

[0027] Selecting speech data with example segmentation in the segmentation set from the speech data in the speech cluster, recording it as example existence data, and taking the proportion of example existence data in all speech data in the speech cluster as the first value of the example segmentation;

[0028] The frequency of occurrence of the example segmentation word in the segmentation set of all speech data in the speech cluster is recorded as the second value of the example segmentation word;

[0029] The commonness of the example word segment is obtained according to the first value and the second value; the first value and the second value are both positively correlated with the commonness.

[0030] Furthermore, the step of obtaining the semantic feature value of each word in the word cluster set of all speech clusters of the training individual includes:

[0031] Select one segmented word from the word cluster set of the speech cluster of the training individual as the target segmented word; select a target existing cluster from all speech clusters of the training individual, and the target segmented word exists in the segmented word set of the speech data in the target existing cluster;

[0032] Calculate the average of the commonness of the target word in the word cluster set of all target clusters of the training individual as the common overall value of the target word; take the proportion of the target cluster in all speech clusters of the training individual as the occurrence value of the target word;

[0033] The semantic feature value of the target word segment is obtained according to the common overall value and the occurrence value.

[0034] Furthermore, the method for clustering all speech data of the training individuals is an agglomerative hierarchical clustering algorithm.

[0035] Furthermore, the training individuals include depression patients and normal individuals.

[0036] In a second aspect, another embodiment of the present invention provides a depression speech recognition system based on deep learning, the system comprising:

[0037] A data collection module is used to obtain speech data of training individuals and test individuals, as well as a word segmentation set of each speech data and a feature sequence of each speech attribute;

[0038] A speech feature extraction module is used to cluster all speech data of the training individual according to the difference in elements in the feature sequence of the same speech attribute of any two speech data of the training individual and the difference in the degree of element fluctuation to obtain a speech cluster of the training individual; based on the number of speech data in the speech cluster, adjust the elements in the feature sequence of each speech attribute of the speech data to obtain a characteristic speech value of each speech attribute of the training individual;

[0039] The feature segmentation screening module is used to record the set of segmentations in the segmentation set of speech data in each speech cluster as a cluster set; construct the feature segmentation set of the training individual according to the number of occurrences of the same segmentation in the speech cluster to which it belongs in the cluster set of all speech clusters of the training individual and the number of speech clusters of each segmentation in the cluster set of the speech cluster of the training individual;

[0040] The depression detection module is used to train a neural network by combining the characteristic speech value and the characteristic word segmentation set of the training individual, and use the trained neural network to perform depression detection on the individual to be tested.

[0041] The present invention has the following beneficial effects:

[0042] In an embodiment of the present invention, there are differences between depression patients and normal people in speech features and speech content; in order to capture the speech features of the training individual in different states, the speech data of the training individual are clustered to obtain speech clusters, the number of speech data in the speech cluster reflects the possibility that the speech features of the speech data in the cluster are speech features of daily communication, the number of speech data in the speech cluster is used to adjust the elements in the feature sequence of each speech attribute, the speech features of the speech features of the training individual in daily communication are analyzed, and the characteristic speech value of each characteristic attribute is obtained; there are large differences in the speech features of people in different states, but different speech contents are close due to word usage habits, combined with the difference in the use of the same segmentation in all speech clusters of the training individual, the number of speech clusters with segmentations showing the breadth of segmentation distribution is analyzed, and common words in the speech data are extracted to form a characteristic segmentation set, thereby reducing the dimension of the speech data; the neural network is trained in combination with the characteristic speech value and the characteristic segmentation set, and the neural network trained with comprehensive speech features and semantic features can more comprehensively and accurately identify the emotional state and psychological characteristics of patients with depression, while reducing the impact of environmental and individual differences on the detection results, and improving the accuracy of depression diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A flowchart of a method for speech recognition of depression based on deep learning provided by an embodiment of the present invention;

[0045] Figure 2 A flowchart of a method for acquiring a feature segmentation set provided by an embodiment of the present invention;

[0046] Figure 3 A system structure diagram of a depression speech recognition system based on deep learning provided by an embodiment of the present invention;

[0047] Figure 4 A schematic diagram of a computer device for a depression speech recognition device based on deep learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a method and system for speech recognition of depression based on deep learning proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0049] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0050] The following is a detailed description of a method and system for speech recognition of depression based on deep learning provided by the present invention in conjunction with the accompanying drawings.

[0051] Embodiment 1:

[0052] This paper proposes a method for speech recognition of depression based on deep learning, please refer to Figure 1 , which shows a flowchart of a method for speech recognition of depression based on deep learning provided by an embodiment of the present invention, the method comprising:

[0053] Step S1: Acquire speech data of the training individual and the individual to be tested, as well as a word segmentation set of each speech data and a feature sequence of each speech attribute.

[0054] Specifically, the training individuals include depression patients and normal people. A voice collection device is used to collect voice signals of the training individuals when they speak normally, and the voice signals are recorded as voice data, and multiple voice data of the training individuals are collected.

[0055] It should be noted that in order to ensure the diversity and representativeness of the data, patients with depression and normal people include people of all ages. Voice collection devices refer to microphones, voice recorders, and smartphones. The horizontal axis of the voice signal represents time, and the vertical axis represents the amplitude or intensity of the signal.

[0056] Since speech data contains rich information, such as pronunciation, emotion and intonation, which causes the frequency content and amplitude of the signal to change over time. In order to more accurately analyze the characteristics of the signal, each speech signal is evenly divided into short time frames of fixed length to capture enough information while maintaining signal stability.

[0057] The speech attributes of the speech data in this embodiment include: pitch, speaking rate and volume; the main frequency of the short-time frame is obtained as the pitch using the autocorrelation method, and the zero-crossing rate of the short-time frame is used as the speaking rate and the amplitude of the short-time frame is used as the volume, and each short-time frame corresponds to a pitch, speaking rate and volume. Each speech attribute of all short-time frames of each speech data is arranged according to the position of the short-time frame to obtain a feature sequence of each speech attribute of each speech data. Among them, the autocorrelation method and the zero-crossing rate are well known to those skilled in the art and will not be repeated here.

[0058] There are significant differences in the speech content between patients with depression and normal people. For example, patients with depression use a lot of negative words, and speech features cannot express the semantic features of speech data. Therefore, each speech data is converted into text data, and the text data is segmented. The set of all segmentations of the text data of each speech data is recorded as the segmentation set of each speech data.

[0059] It should be noted that in this embodiment, the frame length of each short-time frame is 20 milliseconds, which can be set by the implementer according to the specific situation; the hidden Markov model is selected to convert the speech signal into text data, and the JieBa word segmentation tool is used to perform word segmentation on the text data. Among them, the hidden Markov model is a well-known technology for those skilled in the art and will not be described in detail here.

[0060] Step S2: Clustering all speech data of the training individual according to the difference in elements in the feature sequence of the same speech attribute of any two speech data of the training individual and the difference in the degree of element fluctuation to obtain the speech cluster of the training individual; adjusting the elements in the feature sequence of each speech attribute of the speech data based on the number of speech data in the speech cluster to obtain the characteristic speech value of each speech attribute of the training individual.

[0061] Generally speaking, due to people's emotions, states, environment and other reasons, the training individuals may present different speech features in different speech data. In order to more comprehensively capture the speech features of the training individuals in different states, the speech data of the training individuals need to be clustered.

[0062] Preferably, in some possible implementation modes of the embodiments of the present invention, the method for obtaining the feature distance includes: for the feature sequence of each speech attribute of each speech data, calculating the mean of the difference between the remaining elements in the feature sequence except the first element and its adjacent previous element as the fluctuation concentration value; taking the mean of all elements in the feature sequence as the feature concentration value; and obtaining the feature distance of any two speech data of the training individual based on the difference between the fluctuation concentration values ​​of the same speech attribute of any two speech data of the training individual and the difference between the feature concentration values.

[0063] The fluctuation concentration value reflects the fluctuation degree of the elements in the feature sequence, and the feature concentration value reflects the overall level of the elements in the feature sequence. For any two feature sequences of the same speech attribute of speech data, if the difference between the fluctuation concentration values ​​and the feature concentration values ​​of the two feature sequences is smaller, it means that the local fluctuation intensity of the elements in the two feature sequences is closer and the global level is more consistent, then the similarity of the two feature sequences is higher, making the difference in the speech features of the two speech data smaller; in order to classify speech data with similar speech features into a cluster, the feature distance of the two speech data should be smaller.

[0064] In an embodiment of the present invention, a method for obtaining a characteristic distance includes: taking the product of a fluctuation concentration value and a characteristic concentration value of each speech attribute of each speech data of a training individual as the fluctuation characteristic value of each speech attribute of each speech data of the training individual; and calculating the mean of the absolute values ​​of the differences between the fluctuation characteristic values ​​of the same speech attribute of any two speech data of the training individual as the characteristic distance of any two speech data of the training individual.

[0065] To train individuals For example, the first The characteristic distance of speech data is expressed as follows:

[0066] ;

[0067] In the formula, To train an individual The characteristic distance of speech data; The types of speech attributes of the training individual, in this embodiment, include: pitch, speaking speed and volume, Gain experience points To train an individual The first The fluctuating concentration value of the speech attribute; To train an individual The first The characteristic concentration value of the speech attribute; To train an individual The first The fluctuating characteristic values ​​of various speech attributes; To train an individual The first The fluctuating concentration value of the speech attribute; To train an individual The first The characteristic concentration value of the speech attribute; To train an individual The first The fluctuating characteristic values ​​of various speech attributes; is the absolute value function.

[0068] If the feature distance between any two speech data of the training individual is smaller, it means that the speech features of the two speech data are more similar, and the possibility that the two speech data are clustered into one cluster is greater, then all speech data of the training individual are clustered based on the feature distance, and the obtained cluster is recorded as a speech cluster. The more speech data a speech cluster contains, the more likely it is that the speech features of the speech data in the training individual's daily communication are.

[0069] It should be noted that the present invention uses an agglomerative hierarchical clustering algorithm to cluster the speech data of the training individuals. Mean clustering algorithm and DBSCAN algorithm. Among them, the agglomerative hierarchical clustering algorithm is a well-known technology for those skilled in the art and will not be described in detail here.

[0070] The number of speech data in the speech cluster reflects the possibility that the speech features of the speech data in the cluster are the speech features of daily communication. In order to analyze the speech features of the training individuals in daily communication, the number of speech data in the speech cluster is used to adjust the elements in the feature sequence of each speech attribute to obtain the characteristic speech value of each speech attribute.

[0071] Preferably, in some possible implementation modes of the embodiments of the present invention, the method for obtaining the characteristic speech value includes: taking the mode of all elements in the characteristic sequence of each speech attribute of the speech data in each speech cluster as the local characteristic value of each speech attribute of the corresponding speech cluster; arranging all the speech clusters of the training individual in ascending order according to the total number of speech data in the speech cluster to obtain a cluster sequence; obtaining the subscript value of each speech cluster in the cluster sequence and recording it as the characteristic weight of each speech cluster; weighting the local characteristic value of each speech attribute of each speech cluster by using the characteristic weight to obtain the weighted characteristic value of each speech attribute of each speech cluster; and calculating the mean of the weighted characteristic values ​​of each speech attribute of all the speech clusters of the training individual as the characteristic speech value of each speech attribute of the training individual.

[0072] The mode of all elements in the feature sequence of each speech attribute of speech data in the speech cluster represents the overall level of each speech attribute in the speech cluster; because the speech features of speech data in the speech cluster containing more speech data are more likely to be the speech features of the training individual in daily communication, the speech cluster containing more speech data should have a greater weight when analyzing the speech features of each speech attribute. Therefore, the feature weight of the speech cluster when performing speech feature analysis is set by the subscript value of each speech cluster in the cluster sequence. The feature speech value represents the speech feature information of each speech attribute of the training individual in daily communication.

[0073] It should be noted that, in this embodiment, the subscript of the cluster sequence starts from 1, that is, the subscript value of the first element in the cluster sequence is 1.

[0074] Step S3: Record the set of segmentations in the segmentation set of speech data in each speech cluster as a cluster set; construct a characteristic segmentation set of the training individual according to the number of occurrences of the same segmentation in the speech cluster to which it belongs in the cluster set of all speech clusters of the training individual, and the number of speech clusters of each segmentation in the cluster set of speech clusters of the training individual.

[0075] Depressed patients not only differ from normal people in voice pitch, speech speed and volume, but also in voice content. For example, a large number of negative words appear in the voice content of depressed patients. Analyzing only voice features will lead to poor generalization ability of the model when constructing a speech recognition model for depression. In order to make the model more representative, the semantic features of the training individual's voice data are determined by analyzing the similarities of the word segmentation sets of all voice data in the same voice cluster, thereby screening the feature word segmentation set.

[0076] See also Figure 2 , which shows a flowchart of a method for acquiring a feature segmentation set provided by an embodiment of the present invention, the method comprising:

[0077] Step S310: Obtain the commonness of each word in the word cluster set of each speech cluster of the training individual according to the number of occurrences of each word in the corresponding speech cluster of each speech cluster of the training individual and the number of speech data of each word in the word cluster set in each speech cluster.

[0078] In an embodiment of the present invention, a method for obtaining commonness includes: for each speech cluster of a training individual, selecting any segmentation from a word cluster set of the speech cluster as an example segmentation; selecting speech data in which the example segmentation exists in the segmentation set from speech data in the speech cluster, recording it as example existence data, and taking the proportion of the example existence data in all speech data in the speech cluster as a first value of the example segmentation; taking the frequency of occurrence of the example segmentation in the segmentation set of all speech data in the speech cluster as a second value of the example segmentation; obtaining the commonness of the example segmentation according to the first value and the second value; and both the first value and the second value are positively correlated with the commonness.

[0079] The proportion of example existence data in all speech data in the speech cluster reflects the breadth of distribution of the example segmentation in the speech cluster. If the first value of the example segmentation is larger, it means that the example segmentation is more universal or representative in the speech cluster, and the possibility that the example segmentation is a common word in the speech cluster is greater. The frequency of the example segmentation in the segmentation set of all speech data in the speech cluster directly reflects the commonness of the example segmentation in the speech data in the speech cluster. If both the first value and the second value are larger, the example segmentation is used frequently in the speech cluster and appears in multiple speech data, indicating that the possibility that the example segmentation is a common word of the training individual in the speech cluster is greater, and the more common the example segmentation is, the more it can express the semantic characteristics of the speech of patients with depression.

[0080] Therefore, the first value and the second value of the example participle are both positively correlated with the commonness. In the embodiment of the present invention, the product of the first value and the second value of the example participle is normalized to obtain the commonness of the example participle.

[0081] In the embodiment of the present invention, the correlation between the first value, the second value and the commonness may also be constructed through other basic mathematical operations, which will not be limited or elaborated herein.

[0082] It should be noted that in the embodiment of the present invention, the Norm function is used to perform normalization processing, and other normalization methods may also be selected, such as function conversion, maximum and minimum normalization, etc., which are not limited here. The method for obtaining the commonness of all segmentations in the word cluster set of the speech cluster of all training individuals is the same as the method for obtaining the commonness of the example segmentations.

[0083] Step S320: Obtain the semantic feature value of each word in the word cluster set of all speech clusters of the training individual according to the commonness of the same word in the word cluster set of all speech clusters of the training individual and the number of speech clusters of each word in the word cluster set where speech clusters exist.

[0084] People may have large differences in speech features between different speech data due to mood, environment, etc., but the differences in text data between different speech data are small due to people's word usage habits. For example, words such as catchphrases and negative words will appear in different speech data. Therefore, the semantic features of each word are analyzed based on the difference in the usage of the same word in all speech clusters of the training individuals and the number of speech clusters of each word in the word cluster set with speech clusters, and the semantic feature value is obtained.

[0085] In an embodiment of the present invention, a method for obtaining a speech feature value includes: selecting any segmentation from a word cluster set of a speech cluster of a training individual as a target segmentation; selecting a target existence cluster from all speech clusters of the training individual, wherein the target segmentation exists in a segmentation set of speech data within the target existence cluster; calculating the average common degree of the target segmentation in the word cluster set of all target existence clusters of the training individual as the common overall value of the target segmentation; taking the proportion of the target existence cluster in all speech clusters of the training individual as the occurrence value of the target segmentation; and obtaining the semantic feature value of the target segmentation based on the common overall value and the occurrence value.

[0086] If the overall common value of the target segmentation is larger, it means that the training individual uses the target segmentation a lot under different speech characteristics, and the target segmentation can better reflect the semantic characteristics of the speech of the patient with depression. The proportion of the target cluster in all speech clusters of the training individual reflects the breadth of the distribution of the target segmentation in all speech clusters of the training individual, indicating that the target segmentation is more likely to be a common word in the daily life of the training individual, and can better reflect the semantic characteristics of the speech of the patient with depression.

[0087] Therefore, the common overall value and the occurrence value of the target word are positively correlated with the semantic feature value. In the embodiment of the present invention, the product of the common overall value and the occurrence value of the target word is used as the semantic feature value of the target word.

[0088] It should be noted that the method for obtaining the semantic feature values ​​of all the segmented words in the word cluster set of each speech cluster of all training individuals is the same as the method for obtaining the semantic feature values ​​of the target segmented words.

[0089] Step S330: For the semantic feature values ​​of the segmented words in the segmented word set of all speech data of the training individual, a set consisting of the segmented words corresponding to the largest preset number of semantic feature values ​​is used as the feature segmented word set of the training individual.

[0090] The words in the feature word set are common words used by the training individual in daily life. The common words in the speech data of the training individual are extracted through semantic feature values, the dimension of the speech data is reduced, and the accuracy of the neural network training is improved.

[0091] It should be noted that, in the embodiment of the present invention, the total number of segmented words in the segmented word set of all speech data of the training individual is counted, and the value after the total number is rounded up is used as the preset number.

[0092] Step S4: training a neural network by combining the characteristic speech values ​​and characteristic word segmentation sets of all speech attributes of the training individual, and using the trained neural network to perform depression detection on the individual to be tested.

[0093] The characteristic speech value presents the speech characteristics of each speech attribute of the training individual. The words in the characteristic word segmentation set represent the commonly used vocabulary of the training individual in daily life, reflecting the semantic characteristics of the training individual. Combining the characteristic speech value and the characteristic word segmentation set to train the depression speech recognition model can more comprehensively and accurately identify the emotional state and psychological characteristics of patients with depression, while reducing the impact of environmental and individual differences on the detection results.

[0094] The present invention labels the depression labels of training patients through convolutional neural networks. The input of the neural network is the characteristic speech values ​​of all speech attributes of the training individual and the segmentation in the characteristic segmentation set, and the output is the depression label of the training individual.

[0095] The relevant contents of the convolutional neural network include: the data set of the neural network is divided into a training set and a validation set; the training process of the neural network is a process of labeling the depression status of the training individual, and the specific labeling process is: labeling the depression label of the depressed patient as 1, and labeling the depression label of the normal person as 0; the loss function of the neural network is a cross entropy function. The convolutional neural network is a well-known technology for those skilled in the art, and will not be described in detail here.

[0096] In order to adapt the trained neural network to patients with different physical qualities, the present invention collects characteristic data points of at least two training patients during the drug anesthesia period, and trains the neural network using the training individuals according to the above method.

[0097] It should be noted that in the implementation of the present invention, at least two training individuals are used to train the neural network, and the training individuals participating in the neural network training must include patients with depression and normal people, and patients with depression and normal people need to include all age groups. The method for obtaining the characteristic speech values ​​and characteristic word segmentation sets of all speech attributes of the individual to be tested and the training individual is the same.

[0098] So far, the present invention is completed.

[0099] Embodiment 2:

[0100] This paper proposes a depression speech recognition system based on deep learning, please refer to Figure 3 , which shows a system structure diagram of a depression speech recognition system based on deep learning provided by an embodiment of the present invention, the system comprising:

[0101] The data collection module 510 is used to obtain the speech data of the training individual and the individual to be tested, as well as the word segmentation set of each speech data and the feature sequence of each speech attribute;

[0102] The speech feature extraction module 520 is used to cluster all speech data of the training individual according to the difference of elements in the feature sequence of the same speech attribute of any two speech data of the training individual and the difference of the element fluctuation degree to obtain the speech cluster of the training individual; adjust the elements in the feature sequence of each speech attribute of the speech data based on the number of speech data in the speech cluster to obtain the characteristic speech value of each speech attribute of the training individual;

[0103] The characteristic word segmentation screening module 530 is used to record the set of word segments in the word segmentation set of the speech data in each speech cluster as a word cluster set; construct the characteristic word segmentation set of the training individual according to the number of occurrences of the same word segmentation in the speech cluster to which it belongs in the word cluster set of all speech clusters of the training individual and the number of speech clusters of each word segmentation in the word cluster set of the speech cluster of the training individual;

[0104] The depression detection module 540 is used to train a neural network by combining the characteristic speech value and the characteristic word segmentation set of the training individual, and use the trained neural network to perform depression detection on the individual to be tested.

[0105] It should be noted that the device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the depression speech recognition system based on deep learning and the depression speech recognition method embodiment based on deep learning provided in the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0106] Embodiment 3:

[0107] Figure 4 A computer device schematic diagram of a depression speech recognition device based on deep learning provided by an embodiment of the present invention. For example, Figure 4 As shown, the computer device includes: a memory 601, a processor 602, and a computer program 603 stored in the memory 601 and running on the processor 602, wherein when the processor 602 executes the computer program 603, the computer device can execute any one of the deep learning-based depression speech recognition methods introduced above.

[0108] In addition, an embodiment of the present application also protects a device, which may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to execute a deep learning-based depression speech recognition method provided in an embodiment of the present application.

[0109] In this embodiment, the functional modules of the device can be divided according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0110] It should be understood that the device provided in this embodiment is used to execute the above-mentioned deep learning-based depression speech recognition method, and therefore can achieve the same effect as the above-mentioned implementation method.

[0111] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to a device, the processing module may be used to control and manage the actions of the device. The storage module may be used to support the device to execute mutual program codes, etc.

[0112] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits included in the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0113] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0114] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for speech recognition of depression based on deep learning, characterized in that: The method includes: Acquire speech data of the training individual and the test individual, as well as a word segmentation set of each speech data and a feature sequence of each speech attribute; Clustering all speech data of the training individual according to the difference in elements and the difference in element fluctuation degree in the feature sequence of the same speech attribute of any two speech data of the training individual to obtain a speech cluster of the training individual; adjusting the elements in the feature sequence of each speech attribute of the speech data based on the number of speech data in the speech cluster to obtain a characteristic speech value of each speech attribute of the training individual; The set of segmented words in the segmented word set of speech data in each speech cluster is recorded as a cluster set; according to the number of occurrences of the same segmented word in the speech cluster to which it belongs in the cluster set of all speech clusters of the training individual, and the number of speech clusters of each segmented word in the cluster set of speech clusters of the training individual, a characteristic segmented word set of the training individual is constructed; The neural network is trained by combining the characteristic speech values ​​of all speech attributes of the training individual and the characteristic word segmentation set, and the trained neural network is used to detect depression in the individual to be tested; The method for acquiring speech clusters of the training individual comprises: For each feature sequence of each voice attribute of each voice data, the mean of the difference between the remaining elements in the feature sequence except the first element and its adjacent previous element is calculated as the fluctuation concentration value; the mean of all elements in the feature sequence is taken as the feature concentration value; Obtaining a feature distance between any two speech data of the training individual according to a difference between the fluctuation concentration value and the feature concentration value of the same speech attribute of any two speech data of the training individual; Clustering all speech data of the training individuals based on the characteristic distance, and recording the obtained clusters as speech clusters; The step of obtaining the characteristic speech value of each speech attribute of the training individual includes: The mode of all elements in the feature sequence of each speech attribute of the speech data in each speech cluster is used as the local feature value of each speech attribute of the corresponding speech cluster; Arrange all speech clusters of the training individual in ascending order according to the total number of speech data in the speech cluster to obtain a cluster sequence; obtain the subscript value of each speech cluster in the cluster sequence and record it as the feature weight of each speech cluster; The local feature values ​​of each speech attribute of each speech cluster are weighted by using the feature weights to obtain weighted feature values ​​of each speech attribute of each speech cluster; and the average of the weighted feature values ​​of each speech attribute of all speech clusters of the training individual is calculated as the feature speech value of each speech attribute of the training individual.

2. According to the method for speech recognition of depression based on deep learning according to claim 1, it is characterized in that: The step of obtaining the characteristic distance between any two speech data of the training individuals includes: The product of the fluctuation concentrated value and the feature concentrated value of each voice attribute of each voice data of the training individual is used as the fluctuation feature value of each voice attribute of each voice data of the training individual; The average of the absolute values ​​of the differences between the fluctuation characteristic values ​​of the same speech attribute of any two speech data of the training individual is calculated as the characteristic distance between any two speech data of the training individual.

3. According to the method of speech recognition for depression based on deep learning in claim 1, it is characterized in that: The step of constructing a feature segmentation set of training individuals includes: Obtaining the commonness of each word in the word cluster set of each speech cluster of the training individual according to the number of occurrences of each word in the corresponding speech cluster and the number of speech data of each word in the word cluster set in each speech cluster; According to the common degree of the same segmented word in the word cluster set of all speech clusters of the training individual and the number of speech clusters of each segmented word in the word cluster set where speech clusters exist, the semantic feature value of each segmented word in the word cluster set of all speech clusters of the training individual is obtained; For the semantic feature values ​​of the segmented words in the segmented word set of all speech data of the training individual, a set consisting of segmented words corresponding to a maximum preset number of the semantic feature values ​​is used as the feature segmented word set of the training individual.

4. A method for depression speech recognition based on deep learning according to claim 3, characterized in that: The obtaining of the commonness of each word in the word cluster set of each speech cluster of the training individual includes: For each speech cluster of the training individual, select one segmentation from the word cluster set of the speech cluster as the example segmentation; Selecting speech data with example segmentation in the segmentation set from the speech data in the speech cluster, recording it as example existence data, and taking the proportion of example existence data in all speech data in the speech cluster as the first value of the example segmentation; The frequency of occurrence of the example segmentation word in the segmentation set of all speech data in the speech cluster is recorded as the second value of the example segmentation word; The commonness of the example word segment is obtained according to the first value and the second value; the first value and the second value are both positively correlated with the commonness.

5. The method for speech recognition of depression based on deep learning according to claim 3, characterized in that: The step of obtaining the semantic feature value of each word in the word cluster set of all speech clusters of the training individual includes: Select one segmented word from the word cluster set of the speech cluster of the training individual as the target segmented word; select a target existing cluster from all speech clusters of the training individual, and the target segmented word exists in the segmented word set of the speech data in the target existing cluster; Calculate the average of the commonness of the target word in the word cluster set of all target clusters of the training individual as the common overall value of the target word; take the proportion of the target cluster in all speech clusters of the training individual as the occurrence value of the target word; The semantic feature value of the target word segment is obtained according to the common overall value and the occurrence value.

6. The method for depression speech recognition based on deep learning according to claim 1, characterized in that: The method for clustering all speech data of training individuals is an agglomerative hierarchical clustering algorithm.

7. The method for depression speech recognition based on deep learning according to claim 1, characterized in that: The training individuals include depression patients and normal individuals.

8. A depression speech recognition system based on deep learning, characterized in that: The system includes: A data collection module is used to obtain speech data of training individuals and test individuals, as well as a word segmentation set of each speech data and a feature sequence of each speech attribute; A speech feature extraction module is used to cluster all speech data of the training individual according to the difference in elements in the feature sequence of the same speech attribute of any two speech data of the training individual and the difference in the degree of element fluctuation to obtain a speech cluster of the training individual; based on the number of speech data in the speech cluster, adjust the elements in the feature sequence of each speech attribute of the speech data to obtain a characteristic speech value of each speech attribute of the training individual; The feature segmentation screening module is used to record the set of segmentations in the segmentation set of speech data in each speech cluster as a cluster set; construct the feature segmentation set of the training individual according to the number of occurrences of the same segmentation in the speech cluster to which it belongs in the cluster set of all speech clusters of the training individual and the number of speech clusters of each segmentation in the cluster set of the speech cluster of the training individual; A depression detection module, used to train a neural network in combination with the characteristic speech value of the training individual and the characteristic word segmentation set, and use the trained neural network to perform depression detection on the individual to be tested; The method for acquiring speech clusters of the training individual comprises: For each feature sequence of each voice attribute of each voice data, the mean of the difference between the remaining elements in the feature sequence except the first element and its adjacent previous element is calculated as the fluctuation concentration value; the mean of all elements in the feature sequence is taken as the feature concentration value; Obtaining a feature distance between any two speech data of the training individual according to a difference between the fluctuation concentration value and the feature concentration value of the same speech attribute of any two speech data of the training individual; Clustering all speech data of the training individuals based on the characteristic distance, and recording the obtained clusters as speech clusters; The step of obtaining the characteristic speech value of each speech attribute of the training individual includes: The mode of all elements in the feature sequence of each speech attribute of the speech data in each speech cluster is used as the local feature value of each speech attribute of the corresponding speech cluster; Arrange all speech clusters of the training individual in ascending order according to the total number of speech data in the speech cluster to obtain a cluster sequence; obtain the subscript value of each speech cluster in the cluster sequence and record it as the feature weight of each speech cluster; The local feature values ​​of each speech attribute of each speech cluster are weighted by using the feature weights to obtain weighted feature values ​​of each speech attribute of each speech cluster; and the average of the weighted feature values ​​of each speech attribute of all speech clusters of the training individual is calculated as the feature speech value of each speech attribute of the training individual.

Images