Method and system for recognizing depressive mood

Abstract

The application provides a depression recognition method and system, relates to the technical field of machine learning, and comprises the following steps: respectively performing slicing processing on each first voice data in sample data through different numbers of slices; inputting initial features of the sliced sample slices into a preset emotion recognition model to be trained; comparing predicted labels with real labels of the first voice data to obtain a target number of sample slices; inputting target features output from an intermediate layer into a fusion feature network model; training the model; slicing second voice data of a to-be-tested person into voice slices conforming to the target number; identifying the second voice data sliced into the voice slices through the target number determined in the training to expand the data quantity; and obtaining a recognition result of the to-be-tested person through the trained model, so that the required time for recognizing depression can be reduced, and the problem of long recognition period of depression in the prior art is solved.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a method and system for recognizing depressive mood. Background Technology

[0002] Depressive mood refers to feelings of depression, sadness, and low mood caused by various factors. Losing a job, suffering a major setback, facing a major life turning point, or other intense emotional events can all lead to depressive mood.

[0003] Prolonged periods of depression can threaten human health, thus necessitating its identification. Currently, the identification of depressive mood primarily relies on scale assessments combined with professional analysis. However, the high technical threshold for conducting such professional analysis limits the number of qualified personnel, resulting in a lengthy identification process for depressive mood. Summary of the Invention

[0004] This application provides a method and system for recognizing depressive emotions. It can segment sample data into sample slices, determine the appropriate target number of sample slices during the training of a preset emotion recognition model, and recognize depressive emotions through the trained preset emotion recognition model and the fusion feature network model, thereby improving the accuracy of depressive emotion recognition.

[0005] The embodiments of this application are implemented as follows:

[0006] In a first aspect, embodiments of this application provide a method for recognizing depressive mood, the method comprising the following steps:

[0007] Acquire sample data, which includes the first speech data of each participant and the real labels of the first speech data;

[0008] The first speech data is segmented by different numbers of slices to determine the sample slices corresponding to each segmentation process.

[0009] The initial features corresponding to the sample slices are input into the preset emotion recognition model to be trained, and the predicted labels output by the preset emotion recognition model are determined. The predicted labels represent the depression emotion score corresponding to the first speech data.

[0010] Based on the predicted labels and the real labels, determine the target number of the trained preset emotion recognition model and sample slices. The target number is the number of sample slices corresponding to when the predicted labels and the real labels meet the prediction evaluation index.

[0011] The target features are input into the fusion feature network model to determine the trained fusion feature network model, where the target features are the results of the intermediate layer output of the trained preset emotion recognition model.

[0012] The subject's second voice data is segmented into a target number of voice slices;

[0013] Based on speech slices, a pre-trained pre-set emotion recognition model, and a trained fusion feature network model, the recognition result of the test subject is determined. The recognition result includes at least the depression emotion score corresponding to the second speech data.

[0014] In some possible implementations, the first speech data is segmented with different numbers of slices to determine the sample slices corresponding to each segmentation, including:

[0015] According to the preset segmentation strategy, each first speech data is segmented into a different number of sample slices, and the ratio of the number of depressed speech slices and the number of non-depressed speech slices in the sample slices is within a preset ratio range.

[0016] The sample slices are preprocessed to determine the preprocessed sample slices. The preprocessing includes at least one of filtering, endpoint detection, and pre-emphasis. Filtering is used to filter the sample slices, endpoint detection is used to filter the silent segments based on the intersection of the audio and silent segments in the sample slices, and pre-emphasis is used to enhance the high-frequency information in the sample slices.

[0017] In some possible implementations, the preset segmentation strategy includes a first duration segmentation strategy, a second duration segmentation strategy, and a third duration segmentation strategy, wherein the first duration is shorter than the second duration, and the second duration is shorter than the third duration;

[0018] According to the preset segmentation strategy, each speech data is segmented into a different number of sample slices, including:

[0019] Based on the first duration segmentation strategy, each voice data is segmented into a first number of sample slices;

[0020] Based on the second duration segmentation strategy, each voice data is segmented into a second number of sample slices;

[0021] Based on the third duration segmentation strategy, each speech data is segmented into a third number of sample slices.

[0022] In some possible implementations, the trained fusion feature network model is determined, including:

[0023] Obtain the target features of sample slices output from the intermediate layer of the pre-trained pre-defined emotion recognition model;

[0024] The target features are input into the fusion feature network model to determine the target label of the speech data. The target label represents the depression prediction score corresponding to the speech data.

[0025] Based on the target label and the real label of the speech data, the trained fusion feature network model is determined.

[0026] In some possible implementations, before segmenting each first speech data with different numbers of slices, the process includes:

[0027] The first speech data is preprocessed to determine the preprocessed first speech data. The preprocessing includes at least one of filtering, endpoint detection, and pre-emphasis. Filtering is used to filter the first speech data, endpoint detection is used to filter silent segments based on the intersection of spoken and silent segments in the first speech data, and pre-emphasis is used to enhance the high-frequency information in the first speech data.

[0028] In some possible implementations, the recognition result of the test subject is determined based on speech slices, a pre-trained pre-defined emotion recognition model, and a pre-trained fusion feature network model, including:

[0029] The initial features of the speech slice are input into the pre-trained pre-set emotion recognition model to determine the target features of the speech slice output by the intermediate layer in the pre-trained pre-set emotion recognition model.

[0030] The target features of the speech slices are input into a trained fusion feature network model to determine the recognition result of the subject.

[0031] In some possible implementations, the preset emotion recognition model to be trained includes a first emotion recognition model and a second emotion recognition model;

[0032] The initial features corresponding to the sample slices are input into the preset emotion recognition model to be trained, and the predicted labels output by the preset emotion recognition model are determined, including:

[0033] The initial features of the sample slices are input into the first emotion recognition model to determine the first predicted label output by the first emotion recognition model.

[0034] The initial features of the sample slices are input into the second emotion recognition model to determine the second predicted label output by the second emotion recognition model;

[0035] Based on the predicted and true labels, determine the target number of the pre-trained pre-defined emotion recognition model and sample slices, including:

[0036] Based on the first predicted label and the real label, determine the first target number of the trained first emotion recognition model and the sample slice corresponding to the first emotion recognition model;

[0037] Based on the second predicted label and the real label, determine the second target number of the sample slices corresponding to the trained second emotion recognition model and the second emotion recognition model.

[0038] In some possible implementations, the target features are input into a fusion feature network model to determine the trained fusion feature network model, including:

[0039] Obtain the first feature output from the intermediate layer of the trained first emotion recognition model;

[0040] Obtain the second feature output from the intermediate layer of the trained second emotion recognition model. The target feature includes the first feature and the second feature.

[0041] The first feature and the second feature are merged to determine the fused feature;

[0042] The fused features are input into the fused feature network model to determine the trained fused feature network model.

[0043] In some possible implementations, the subject's second speech data is segmented into a target number of speech slices, including:

[0044] The second speech data is divided into a first speech slice that meets the first target number, and the second speech data is divided into a second speech slice that meets the second target number.

[0045] Based on speech slices, a pre-trained pre-defined emotion recognition model, and a trained fusion feature network model, the recognition results for the test subject are determined, including:

[0046] Based on the first speech slice, the second speech slice, the trained first emotion recognition model, the trained second emotion recognition model, and the trained fusion feature network model, the recognition result of the test subject is determined.

[0047] Secondly, embodiments of this application provide a depressive mood recognition system, which includes a front-end device and a back-end processing device, wherein the front-end device and the back-end processing device are connected via a network;

[0048] The background processing device includes a first memory and a first processor. The first memory stores a computer program, and the first processor executes the computer program to implement the steps of the depressive mood recognition method of any of the first aspects above.

[0049] The front-end device includes a display and a second processor; the display is used to show the recognition result of the subject, which is obtained through the back-end processing device; the second processor is used to receive the voice signal of the subject and send the voice signal of the subject to the back-end processing device.

[0050] Thirdly, this application provides a device for recognizing depressive mood, the device comprising:

[0051] The data processing module is used to acquire sample data, which includes the first voice data of each participant and the real labels of the first voice data.

[0052] The data processing module is also used to segment each first speech data with different numbers of slices and determine the sample slices corresponding to each segmentation process.

[0053] The training module is used to input the initial features corresponding to the sample slices into the preset emotion recognition model to be trained, and to determine the predicted label output by the preset emotion recognition model. The predicted label represents the depression emotion score corresponding to the first speech data.

[0054] The training module is also used to determine the target number of the pre-trained pre-set emotion recognition model and sample slices based on the predicted labels and the real labels. The target number is the number of sample slices corresponding to when the predicted labels and the real labels meet the prediction evaluation index.

[0055] The training module is also used to input the target features into the fusion feature network model and determine the trained fusion feature network model, wherein the target features are the results of the intermediate layer output of the trained preset emotion recognition model;

[0056] The prediction module is used to segment the subject's second speech data into speech slices that meet the target number;

[0057] The prediction module is also used to determine the recognition result of the test subject based on speech slices, a pre-trained preset emotion recognition model and a pre-trained fusion feature network model. The recognition result includes at least the depression emotion score corresponding to the second speech data.

[0058] Fourthly, this application provides a computer storage medium storing a computer program that, when executed by a processor, implements the steps of the depressive mood recognition method of any one of the first aspects described above.

[0059] Fifthly, this application provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the depressive mood recognition method of any one of the first aspects described above.

[0060] This application can identify depressive mood without relying on scale examinations and professional analysis. Compared with existing technologies that rely on scale examinations and professional analysis, the solution provided in this application requires less time to train the emotion recognition model and the fusion feature network model, and to identify depressive mood using the trained emotion recognition model and the fusion feature network model. Furthermore, the solution of this application can repeatedly apply the trained emotion recognition model and the fusion feature network model, and can simultaneously identify the depressive mood of multiple people using the trained emotion recognition model and the fusion feature network model. All of these can reduce the time required for depressive mood identification, thereby solving the problem of long cycle for identifying depressive mood in existing technologies.

[0061] Furthermore, this application segments each first speech data in the acquired sample data with different numbers of slices to obtain sample slices corresponding to each segmentation. The sample slices expand the data volume of the sample data while retaining relatively complete signal features. A preset emotion recognition model is trained based on these sample slices. The predicted labels output by the preset emotion recognition model are compared with the true labels of the first speech data in the sample data to determine the target number of sample slices with good training performance. The target features output from the intermediate layer of the trained preset emotion recognition model are input into a fusion feature network model to determine the trained fusion feature network model. During the recognition process, the second speech data of the test subject is segmented into speech slices that meet the target number to provide better training results. Based on the speech slices, the trained preset emotion recognition model, and the trained fusion feature network model, the recognition result of the test subject is determined. Therefore, this application also has high accuracy in recognizing depressive emotions. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 A flowchart illustrating a method for recognizing depressive mood provided in an embodiment of this application is shown;

[0064] Figure 2 This paper illustrates a flowchart of the training portion of a depressive mood recognition method provided in an embodiment of this application.

[0065] Figure 3 This illustration shows a schematic diagram of the training process of a model based on a convolutional neural network provided in an embodiment of this application;

[0066] Figure 4 This illustration shows a schematic diagram of feature fusion in a feature network model provided in an embodiment of this application;

[0067] Figure 5 The diagram illustrates a flowchart of the prediction part in a depressive mood recognition method provided in an embodiment of this application.

[0068] Figure 6 An embodiment of this application also illustrates a depressive mood recognition device;

[0069] Figure 7 This paper shows a schematic diagram of the structure of a depressive mood recognition system provided in an embodiment of this application;

[0070] Figure 8 A schematic diagram of the structure of another depressive mood recognition system provided in an embodiment of this application is shown. Detailed Implementation

[0071] To make the objectives, implementation methods and advantages of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0072] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0073] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0074] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0075] Depressive mood refers to feelings of sadness, grief, and low mood caused by various factors. Losing a job, experiencing a major setback, facing a significant life turning point, or other intense emotional events can all lead to depressive mood. Prolonged immersion in depressive mood can threaten human health; therefore, it is necessary to identify depressive mood. Currently, the identification of depressive mood mainly relies on scale testing combined with professional analysis. However, due to the high technical threshold for conducting professional analysis, the number of people capable of performing such analysis is limited, resulting in a long identification period for depressive mood.

[0076] With the development of machine learning technology, depressive moods can be identified through behavioral features such as voice signals, facial expressions, and actions. Among these, voice signals are easy to collect and contain a large amount of information, making them suitable for integration with machine learning technology to provide depressive mood recognition.

[0077] In related technologies, the main approach involves segmenting the speech signal into frames, extracting audio signal features such as Mel-frequency cepstral coefficients, formants, and zero-crossing rates, and then training these features using traditional machine learning methods or deep learning methods such as support vector machines, decision trees, and Gaussian mixture models to provide recognition results. However, frame-level audio feature recognition methods weaken the continuous semantic information between audio segments in the speech data. Furthermore, directly using the entire speech signal for evaluation can lead to the model ignoring truly relevant features during training due to the potential presence of redundant information unrelated to depressive mood. This results in the loss of key features and poor accuracy in assessing depressive mood.

[0078] To address the aforementioned issues, this application provides a method and system for recognizing depressive emotions. The method segments each first speech data point in the acquired sample data with different numbers of slices, obtaining sample slices corresponding to each segmentation. These sample slices expand the sample data volume while retaining relatively complete signal features. The initial features corresponding to the sample slices are input into a pre-trained emotion recognition model to determine the predicted labels output by the model. Then, the predicted labels are compared with the true labels of the first speech data in the sample data to determine the target number of the trained pre-trained emotion recognition model and sample slices. The target features output from the intermediate layer of the trained pre-trained emotion recognition model are then input into a fusion feature network model to determine the trained fusion feature network model. After training all models, the second speech data of the test subject is segmented into speech slices that meet the target number. The second speech data is segmented into speech slices for recognition using the target number determined during training, reducing the amount of data processed in the training and recognition processes. Finally, based on the speech slices, the trained pre-trained emotion recognition model, and the trained fusion feature network model, the recognition result of the test subject is determined.

[0079] This application embodiment can identify depressive mood without relying on scale examinations and professional analysis. Compared with existing technologies that rely on scale examinations and professional analysis, the solution provided in this application embodiment requires less time to train the emotion recognition model and the fusion feature network model, and to identify depressive mood using the trained emotion recognition model and the fusion feature network model. Furthermore, the solution in this application embodiment can repeatedly apply the trained emotion recognition model and the fusion feature network model, and can simultaneously identify the depressive mood of multiple people. All of these can reduce the time required for depressive mood identification, thereby solving the problem of long cycle for identifying depressive mood in the prior art.

[0080] Furthermore, since this application uses audio segments and fusion features when performing depressive mood recognition, the depressive mood recognition method disclosed in this application can also be considered as an online depressive mood recognition method based on audio segments and feature fusion.

[0081] The following describes in detail the depressive mood recognition method and system of this application with reference to the accompanying drawings.

[0082] Figure 1 This document illustrates a flowchart of a method for recognizing depressive mood according to an embodiment of this application. Figure 1 As shown in the embodiment of this application, a method for recognizing depressive mood is provided.

[0083] This method for identifying depressive mood includes the following steps:

[0084] S110. Obtain sample data, which includes the first speech data of each participant and the real labels of the first speech data.

[0085] It should be understood that the sample data can be data from publicly available depression datasets, such as the DAIC-WOZ database (AVEC 2017) and the Extended-DAIC database (AVEC 2019). The datasets collect the participants' voice data in a question-and-answer manner and provide the participants with real labels based on the voice data (i.e., the first voice data). These real labels are used to characterize the depressive mood score assessed by a professional physician based on the first voice data.

[0086] In some embodiments, the sample data may also be collected by professional analysts through question-and-answer sessions during the actual process of identifying depressive moods, and the corresponding voice data may be assigned real labels.

[0087] In some embodiments, the first speech data may be preprocessed to determine the preprocessed first speech data. The preprocessing includes at least one of filtering, endpoint detection, and pre-emphasis. Filtering is used to filter the first speech data, endpoint detection is used to filter silent segments based on the intersection of spoken and silent segments in the first speech data, and pre-emphasis is used to enhance the high-frequency information in the first speech data.

[0088] The filtering function can filter out audio signals in the first speech data that are outside the preset range, and retain the frequency range related to the assessment of depressive mood, for example, retaining audio signals with a frequency range between 80Hz and 3800Hz.

[0089] Endpoint detection first identifies the intersection points between silent and spoken segments in the first speech data, then segments the first speech data based on these intersection points, retaining the spoken segments and removing the silent segments. For features present in the first speech data that only appear in spoken segments, endpoint detection can remove silent segments and improve the accuracy of the training data.

[0090] Pre-emphasis increases the resolution of high-frequency signals in the first speech data. The first speech data contains a lot of information in the consonants, and the consonants have a high frequency. By amplifying the high-frequency signals, more information can be obtained.

[0091] In some embodiments, pre-emphasis can be implemented using a first-order high-pass filter, H of which can be expressed by the following equation:

[0092] H(z) = 1 - αz -1

[0093] In the formula, z is the first speech data, and α is the pre-emphasis coefficient; for example, α is set to 0.96.

[0094] In some embodiments, the preprocessing of the first voice data described above can be performed in step 120. That is, the preprocessing of the data can be performed before or after the segmentation process.

[0095] S120. The first speech data is segmented by different numbers of slices to determine the sample slices corresponding to each segmentation process.

[0096] Segmenting each first speech data with different numbers of slices means segmenting each first speech data multiple times, and segmenting it with different numbers of slices each time, that is, segmenting the first speech data into different numbers of sample slices each time.

[0097] The first voice data is collected from the test subjects using a question-and-answer method. In other words, the first voice data contains several data points of varying lengths. By segmenting the first voice data, sample slices of similar lengths can be obtained, and the number of positive and negative sample slices can be kept in a relatively balanced state.

[0098] In step 120, the first speech data is divided into different sample slices through segmentation processing. Compared with the frame-segmentation processing of speech signals in related technologies, the sample slices contain more information than each frame in the speech signal. Therefore, more semantic information can be retained to improve the accuracy of training models.

[0099] The segmentation process divides the first speech data into speech slices of appropriate size, so that initial features can be extracted from the speech slices in the subsequent process, thereby obtaining initial features containing more speech information. It can also expand the total number of samples, thereby training a model with more accurate recognition performance.

[0100] In some embodiments, each first speech data can be divided into different numbers of sample slices according to a preset segmentation strategy. The ratio of the number of depressive speech slices to the number of non-depressive speech slices within the sample slices is within a preset ratio range.

[0101] In some embodiments, the preset ratio range can be between 0.9 and 1.1. For example, through segmentation processing, the ratio of the number of depressed voice segments and the number of non-depressed voice segments in the sample slice is close to 1. When the number of depressed voice segments and the number of non-depressed voice segments in the sample slice reach sample balance, that is, when the number of depressed voice segments and the number of non-depressed voice segments in the sample slice are balanced.

[0102] It should be understood that there is no limit to the duration of different types of speech segments, including those for depressive and non-depressive speech segments. In other words, it is not necessary for different types of speech segments to have the same duration.

[0103] In some embodiments, if the preset segmentation strategy includes a first duration segmentation strategy, a second duration segmentation strategy, and a third duration segmentation strategy, wherein the first duration is less than the second duration, and the second duration is less than the third duration.

[0104] According to the preset segmentation strategy, each voice data is segmented into a different number of sample slices, including: segmenting each voice data into a first number of sample slices according to the first duration segmentation strategy; segmenting each voice data into a second number of sample slices according to the second duration segmentation strategy; and segmenting each voice data into a third number of sample slices according to the third duration segmentation strategy.

[0105] In different training processes, the first number of sample slices, the second number of sample slices, or the third number of sample slices can be used as input data for the model, and the output can be used to determine which number of sample slices produces the best model performance.

[0106] In some embodiments, a portion of the first voice data may not be segmented.

[0107] As an example, each first speech data is divided into four types: several short-term sample slices, several medium-term sample slices, several long-term sample slices, or complete sample slices without segmentation. For each segmented sample slice, the label of the sample slice is assigned the same real label as the first speech data.

[0108] In some embodiments, if the first speech data is not preprocessed in step 110, the sample slices can be preprocessed after the slice processing in step 120 to determine the preprocessed sample slices. The preprocessing includes at least one of filtering, endpoint detection, and pre-emphasis. Filtering is used to filter the sample slices, endpoint detection is used to filter the silent segments based on the intersection of the audible and silent segments in the sample slices, and pre-emphasis is used to enhance the high-frequency information in the sample slices.

[0109] The embodiments of this application can remove redundant information unrelated to depressive mood through preprocessing. Compared with related technologies that directly use the whole speech signal for training and evaluation, the model training accuracy is higher, and the amount of data after segmentation is less than the amount of data of the whole speech signal, thus improving the efficiency of training and recognition.

[0110] The specific preprocessing method is the same as in step 110, and will not be repeated here.

[0111] S130. Input the initial features corresponding to the sample slices into the preset emotion recognition model to be trained, and determine the predicted label output by the preset emotion recognition model.

[0112] The predicted label represents the depression score corresponding to the first voice data.

[0113] The initial features corresponding to the sample slices are the audio features contained in the sample slices. For extracting audio features from the sample slices, the following can be extracted from the eGeMAPS feature set (GeMAPS feature set has a total of 62 features, all of which are HSF features, calculated from 18 LLD features): fundamental frequency F0 (F0 generally refers to the fundamental frequency), pitch (log F0, calculated on a half-frequency scale, starting from 27.5Hz), jitter (the deviation within a single consecutive fundamental frequency period; the deviation measures the difference between the observed variable and a specific value; if no specific value is specified, the mean of the variable is usually used), bandwidth of the first formant, center frequencies of the first three formants, the ratio of the center frequencies of the first three formants to the fundamental energy, Hammarberg Index (the strongest energy peak from 0-2kHz divided by the strongest energy peak from 2-5kHz), Spectral Slope (two slopes obtained by linear regression on two regions of the linear power spectrum, 0-500Hz and 500-1500Hz), and Spectral... The LLD features, including flux, AlphaRatio (energy sum of 50-1000Hz divided by energy sum of 1-5kHz), Harmonic difference H1-H2 (energy of the first fundamental harmonic H1 divided by energy of the second fundamental harmonic), MFCC1-4, and bandwidth of the second and third formants, were calculated. Then, statistical function values ​​such as standard deviation and mean slope were calculated for each LLD feature. The original features were combined with the values ​​after applying the statistical functions to obtain a total of 88 features. A 1-dimensional label value representing the depression score was added after the 88 features to generate the initial features.

[0114] In some embodiments, a preset emotion recognition model is constructed, and the initial features corresponding to the sample slices (depressed speech slices and non-depressed speech slices) are input into the preset emotion recognition model to be trained for training. The output layer of the preset emotion recognition model outputs the predicted label representing the predicted depression emotion score.

[0115] The evaluation metrics for the model can be root mean squared error (RMSE) and mean absolute error (MAE). After optimizing the model parameters to obtain a better preset emotion recognition model, the model parameters are fixed.

[0116] In some embodiments, the preset emotion recognition model can be one model or multiple models.

[0117] In some embodiments, the preset emotion recognition model to be trained includes a first emotion recognition model and a second emotion recognition model; then, inputting the initial features corresponding to the sample slice into the preset emotion recognition model to be trained and determining the predicted label output by the preset emotion recognition model includes: inputting the initial features of the sample slice into the first emotion recognition model and determining the first predicted label output by the first emotion recognition model; inputting the initial features of the sample slice into the second emotion recognition model and determining the second predicted label output by the second emotion recognition model.

[0118] As an example, the preset emotion recognition models to be trained include models based on Convolutional Neural Networks (CNNs) and models based on Long Short-Term Memory (LSTM). The CNN-based and LSTM-based models can extract contextual information from sample slices to a certain extent. The initial features of the sample slices are input into the CNN-based model to determine the convolutional prediction labels output by the CNN-based model; the initial features of the sample slices are input into the LSTM-based model to determine the long short-term prediction labels output by the LSTM-based model.

[0119] Among them, the convolutional neural network-based model used for audio feature analysis refers to a network model constructed using a sequential structure, consisting of convolutional layers and fully connected layers. The sample slices are audio data, which are data extended in the time direction. Therefore, the convolutional layers of the convolutional neural network-based model use convolution in a one-dimensional direction.

[0120] The feature file is fed into a convolutional neural network-based model, which includes three convolutional layers for extracting audio features. Each convolutional layer is followed by a LeakyReLU function layer, a MaxPooling layer, and a Dropout layer. The activation function layer prevents gradient vanishing, the MaxPooling layer reduces the dimensionality of the data, and the Dropout layer prevents overfitting during training. There are three fully connected layers, each followed by a LeakyReLU activation function layer, and finally, the output layer.

[0121] For example, the horizontal convolutional kernel can be calculated by combining the speech features before and after a certain moment in the speech information. The kernel size is 8*1, the number of channels is set to 64, and the convolution operation mode is selected. After the convolutional layer, the pooling operation is used to reduce the dimensionality of the features. In order to improve the training, the regularization (Dropout) operation is used to randomly disable the update of some neuron parameters during the training process. The feature file is subjected to matrix transformation (Reshape) operation so that the features can be fed into the CNN model for training. Among them, the convolutional layer, pooling layer, and fully connected layer are all intermediate layers, i.e., hidden layers.

[0122] The Long Short-Term Memory (LSTM) network model used for audio feature analysis refers to a network model constructed using a sequential structure, consisting of LSTM layers and fully connected layers. Each fully connected layer is followed by a Leaky ReLU activation function layer. The feature file is subjected to matrix transformation (reshape) operations so that the feature file can be fed into the LSTM layer for training. Both the LSTM layer and the fully connected layer are intermediate layers, i.e., hidden layers.

[0123] The LeakyReLU function is used to prevent gradient vanishing and neuron death during training. Its calculation method can be represented as follows:

[0124]

[0125] In the formula, x is the output value of the neuron, which takes the value in the real number range, f(x) is the value of neuron x after passing through the LeakyReLU activation function, and α is a coefficient, which can be 0.01.

[0126] It should be understood that ReLU is a commonly used activation function that can make the output of neurons nonlinear, thereby improving the expressive power of neural networks. However, the ReLU function also has a problem: when the output of the neuron (i.e., x in the above formula) is negative, the derivative of the ReLU function is 0, which may lead to the "dead ReLU problem," that is, the neuron stops updating during training.

[0127] The LeakyReLU function is an improvement on the ReLU function. The LeakyReLU function prevents neurons from "slipping" because it introduces a small slope in the negative range. This ensures that even if the neuron's output is negative, its derivative is not zero, allowing the neuron to continue updating during training. The slope coefficient of the LeakyReLU function is typically set to a small value, such as 0.01 or 0.2, to guarantee that the derivative in the negative range is not too small while also minimizing its impact on the neuron's output.

[0128] In some embodiments, in order to enable both the convolutional neural network-based model and the long short-term memory network-based model to be trained normally, this invention treats them as regression models during the training phase. That is, the output layer results are scores representing the degree of depressive mood. After the training process achieves good results, the model is saved, and only the output of the last intermediate layer is used as the target feature (which is also the audio feature). During the model training process, the loss functions used are root mean square error (RMSE) and mean absolute error (MAE).

[0129] For a pre-defined emotion recognition model that includes multiple models, it is necessary to consider the specific evaluation requirements. Although a larger number of models may improve accuracy, a larger number of models will also bring about the cost of model storage and model computation.

[0130] S140. Based on the predicted labels and the real labels, determine the target number of the trained preset emotion recognition model and sample slices.

[0131] The target number is the number of sample slices that correspond to the prediction and evaluation metrics when the predicted label and the true label meet the prediction evaluation criteria.

[0132] It should be understood that during the model training process, it is necessary to find the appropriate number of slices (i.e., the number of targets) in each network of the preset emotion recognition model, and determine the best-performing model and the corresponding number of targets.

[0133] In some embodiments, the loss functions used during model training are root mean square error (RMSE) and mean absolute error (MAE). That is, RMSE and MAE are determined by predicting the labels and the true labels, and then RMSE and MAE are used to evaluate the model’s performance and determine the target number of trained preset emotion recognition models and sample slices.

[0134] The RMSE can be calculated using the following formula:

[0135]

[0136] In the formula, m is the total number of samples, and y i and These represent the true label and the predicted label, respectively, reflecting the difference between the predicted and the true results.

[0137] MAE can be calculated using the following formula:

[0138]

[0139] In the formula, m is the total number of samples, and y i and These represent the true label and the predicted label, respectively, reflecting the difference between the predicted result and the average level of the overall sample.

[0140] In some embodiments, the preset emotion recognition model to be trained includes a first emotion recognition model and a second emotion recognition model. For step 140, determining the target number of the trained preset emotion recognition model and sample slices based on predicted labels and real labels includes: determining the first target number of the trained first emotion recognition model and the sample slices corresponding to the first emotion recognition model based on the first predicted labels and real labels; determining the second target number of the trained second emotion recognition model and the sample slices corresponding to the second emotion recognition model based on the second predicted labels and real labels; and determining the target number corresponding to each first emotion recognition model and second emotion recognition model respectively.

[0141] For the example above, the preset emotion recognition model to be trained includes a model based on a convolutional neural network and a model based on a long short-term memory network. In step 140, determining the target number of the trained preset emotion recognition model and the sample slices based on the predicted labels and the real labels includes: determining the convolution target number of the trained convolutional neural network model and the corresponding sample slices based on the convolutional predicted labels and the real labels; and determining the long and short-term target number of the trained long short-term memory network model and the corresponding sample slices based on the long short-term predicted labels and the real labels.

[0142] The first step in using either model is to train it on the data. Once the training results are satisfactory, the model parameters can be frozen. The frozen parameter model can then be used to predict speech emotion.

[0143] The training process for the models based on convolutional neural networks and long short-term memory networks involves processing the audio information by performing sample balancing and slicing operations, training the model according to the number of slices, comparing the training loss under different conditions, selecting the optimal model, freezing its parameters, and retaining the output of its intermediate layers as the optimal target feature.

[0144] It's important to understand that in machine learning, datasets are typically divided into training, validation, and test sets. The training set is used to train the model. We use the training set to train the model's parameters so that it can predict input data. The training set is usually a large dataset containing all the information the model needs, allowing it to adapt to the data and learn patterns, thus improving its generalization ability. The validation set is primarily used to tune the model's hyperparameters. During training, we need to select some hyperparameters (such as learning rate, regularization coefficient, etc.) to optimize the model's performance. We use the validation set to evaluate the model's performance under different hyperparameters and then select the optimal hyperparameters to improve the model's generalization ability. The validation set is usually partitioned from the training set; its data is independent of the training set's data but drawn from the same data distribution. The test set is used to evaluate the model's performance. After the model training is complete, we use the test set to evaluate the model's generalization ability on new data. The test set is also usually partitioned from the training set; its data is independent of the training and validation sets' data and has a similar data distribution to the training and validation sets. The division of the test set, training set, and validation set can usually follow a preset ratio, such as 6:2:2, but this is not fixed and needs to be analyzed based on the specific amount of data.

[0145] S150. Input the target features into the fusion feature network model to determine the trained fusion feature network model.

[0146] The target feature is the result of the intermediate layer output of the pre-trained pre-defined emotion recognition model.

[0147] In some embodiments, a fusion feature network model is constructed, the target features are input into the fusion feature network model, and the final label is the depression score output by the fusion feature network model.

[0148] The fusion feature network model consists of 5 fully connected layers, with the last layer using an activation function to ensure that the final output score is greater than 0.

[0149] In some embodiments, target features of sample slices output from intermediate layers in a pre-trained pre-defined emotion recognition model are obtained; the target features are input into a fusion feature network model to determine the target label of the speech data, whereby the target label represents the depression prediction score corresponding to the speech data; and the pre-trained fusion feature network model is determined based on the target label and the real label of the speech data.

[0150] In some embodiments, the preset emotion recognition model to be trained includes a first emotion recognition model and a second emotion recognition model. For step 150, inputting the target feature into the fusion feature network model to determine the trained fusion feature network model includes: obtaining the first feature output by the intermediate layer in the trained first emotion recognition model; obtaining the second feature output by the intermediate layer in the trained second emotion recognition model, wherein the target feature includes the first feature and the second feature; fusing the first feature and the second feature to determine the fused feature; and inputting the fused feature into the fusion feature network model to determine the trained fusion feature network model.

[0151] For the example above, the preset emotion recognition model to be trained includes a model based on a convolutional neural network and a model based on a long short-term memory network. In step 150, the target features are input into the fusion feature network model to determine the trained fusion feature network model, including: obtaining the first high-level audio feature (one of the target features) output from the intermediate layer of the trained convolutional neural network model; obtaining the second high-level audio feature (one of the target features) output from the intermediate layer of the trained long short-term memory network model; fusing the first high-level audio feature and the second high-level audio feature to determine the fusion feature; inputting the fusion feature into the fusion feature network model to output the final label (i.e., the final predicted depression emotion score) to determine the trained fusion feature network model.

[0152] The feature fusion network model fuses high-level audio features (i.e., target features) extracted by convolutional neural network (CNN) and long short-term memory (LSTM) network models. CNN and LSM models extract audio features differently, and combining their features yields more information-rich features, improving the accuracy of depressive mood recognition. The feature fusion method involves first concatenating the extracted high-level audio features, then feeding the concatenated features into a feature fusion network. Through processing in hidden layers, the final prediction result is output.

[0153] Steps 110 to 150 above constitute the training portion of the depressive mood recognition method. Figure 2 This document illustrates a flowchart of the training portion of a depressive mood recognition method provided in an embodiment of this application. Figure 2As shown, the training part includes: first, preparing sample data as audio input, then performing speech slicing, sample balancing, and initial feature extraction; after feature extraction, model training can begin. As mentioned earlier, this module consists of a model based on a convolutional neural network and a model based on a long short-term memory network. The output layers of both models represent the predicted depression scores. The evaluation metrics are RMSE and MAE. After optimizing the model parameters to obtain a better depression recognition model, the model parameters are fixed, and the output of the last layer of the intermediate layer of the model is selected as the target feature for extraction. The features of the two models need to be concatenated. This feature integrates global audio information, thus improving the accuracy of depression detection. After advanced audio feature extraction, the model fusion stage begins. The model fusion network contains 5 fully connected layers. The input is the concatenated features, and the output is the predicted score. The evaluation metrics are RMSE (root mean square error) and MAE (mean absolute error).

[0154] Figure 3 The diagram illustrates the training process of a convolutional neural network-based model according to an embodiment of this application. Figure 3 As shown, during the training phase, the model receives initial features and then passes them through a three-layer convolutional neural network. To prevent gradient vanishing and overfitting during network training, each convolutional layer is followed by a LeakyReLU activation function layer, a MaxPooling layer, and a Dropout layer. After information is extracted by the convolutional layers, the data is flattened by tensors and then fed into a fully connected layer (FC). The depression score of the sample is used as the true label, and RMSE and MAE are selected as loss functions to optimize the model parameter settings. When the model is fitted, the optimal model during training is selected for testing on the validation set. Training is repeated until the model's loss on the validation set reaches its optimum. The model parameters are then frozen, and FC3 in the fully connected layer is selected as the target feature for audio and fed into the model fusion network.

[0155] Figure 4 This illustration shows a schematic diagram of feature fusion in a feature network model according to an embodiment of this application. Features extracted from a convolutional neural network (CNN) model and features from a long short-term memory (LSTM) network model are concatenated and then fed into the feature network model to obtain the final label (i.e., the final predicted score). The features from the CNN-based model are the last fully connected layer in the intermediate layer. Figure 4 In FC3, the features from the model based on Long Short-Term Memory (LSTM) networks are the output, i.e. Figure 4 The output section.

[0156] The depressive mood recognition method in this application also includes a testing section, such as... Figure 1As shown, it also includes: S160, dividing the second speech data of the test subject into speech slices that meet the target number.

[0157] In the testing section, there is no need to balance the data samples; only data splitting is required.

[0158] In some embodiments, the preset emotion recognition model to be trained includes a first emotion recognition model and a second emotion recognition model. In step 160, the second speech data of the test subject is segmented into speech slices that meet the target number, which includes: segmenting the second speech data into first speech slices that meet the first target number, and segmenting the second speech data into second speech slices that meet the second target number.

[0159] For the example above, the preset emotion recognition model to be trained includes a model based on a convolutional neural network and a model based on a long short-term memory network. In step 160, the second speech data of the test subject is divided into speech slices that meet the target number, including: dividing the second speech data into convolutional speech slices that meet the convolution target number, and dividing the second speech data into long short-term memory speech slices that meet the long short-term memory target number.

[0160] In some embodiments, segmenting the second speech data of the test subject into a target number of speech slices includes: determining a target segmentation strategy corresponding to the target number based on the target number; and segmenting the second speech data into speech slices based on the target segmentation strategy.

[0161] S170. Based on speech slices, a pre-trained pre-set emotion recognition model, and a pre-trained fusion feature network model, the recognition result of the test subject is determined.

[0162] The recognition results include at least the depression score corresponding to the second voice data.

[0163] In some embodiments, the initial features of the speech slice are input into a pre-trained preset emotion recognition model to determine the target features of the speech slice output by the intermediate layer in the pre-trained preset emotion recognition model; the target features of the speech slice are input into a pre-trained fusion feature network model to determine the recognition result of the test subject.

[0164] In some embodiments, the mood improvement method corresponding to the depression score of the second voice data can be determined based on the depression score of the second voice data and the preset improvement strategy.

[0165] In some embodiments, the depression score corresponding to the second voice data corresponds to the depression assessment index (PHQ-8) score. The PHQ-8 score ranges from 0 to 24 points, and is divided into depression levels according to the principle that every 5 points increase represents no depression, mild depression, moderate depression, moderate to severe depression, and severe depression, respectively.

[0166] In some embodiments, the depression score corresponding to the second voice data is greater than 9 points in the index assessment score, indicating that the person has a depression.

[0167] In some embodiments, the preset emotion recognition model to be trained includes a first emotion recognition model and a second emotion recognition model. For step 170, determining the recognition result of the test subject based on the speech slice, the trained preset emotion recognition model, and the trained fusion feature network model includes: determining the recognition result of the test subject based on the first speech slice, the second speech slice, the trained first emotion recognition model, the trained second emotion recognition model, and the trained fusion feature network model.

[0168] For the example above, the preset emotion recognition model to be trained includes a model based on a convolutional neural network and a model based on a long short-term memory network. In step 170, the recognition result of the test subject is determined based on speech slices, the trained preset emotion recognition model, and the trained fusion feature network model. This includes: determining the recognition result of the test subject based on convolutional speech slices, long short-term speech slices, the trained convolutional neural network model, the trained long short-term memory network model, and the trained fusion feature network model.

[0169] Steps 160 to 170 above constitute the prediction part of the depressive mood recognition method. Figure 5 This illustration shows a flowchart of the prediction part in a depressive mood recognition method provided in an embodiment of this application. Figure 5 As shown, the prediction part includes: first, inputting second speech data, which can be recorded online or uploaded locally; slicing the second speech data; then extracting initial features; feeding the initial feature file into the model fusion network for prediction; and outputting the prediction score.

[0170] The segmented speech slices retain more semantic information while reducing redundant information irrelevant to the identification of depressive emotions, thus improving the accuracy of depressive emotion recognition while ensuring its effectiveness.

[0171] Based on the aforementioned depressive mood recognition method, and employing the same technical concept... Figure 6 An embodiment of this application also provides a depressive mood recognition device, such as... Figure 6As shown, the depressive mood recognition device 600 includes a data processing module 610, a training module 620, and a prediction module 630, wherein:

[0172] The data processing module is used to acquire sample data, which includes the first speech data of each participant and the real labels of the first speech data; it is also used to segment each first speech data with different numbers of slices to determine the sample slices corresponding to each segmentation.

[0173] The training module is used to input the initial features corresponding to the sample slices into the preset emotion recognition model to be trained, and to determine the predicted label output by the preset emotion recognition model. The predicted label represents the depression emotion score corresponding to the first speech data. It is also used to determine the target number of the trained preset emotion recognition model and sample slices based on the predicted label and the real label. The target number is the number of sample slices corresponding to when the predicted label and the real label meet the prediction evaluation index. It is also used to input the target features into the fusion feature network model to determine the trained fusion feature network model. The target features are the results of the intermediate layer output of the trained preset emotion recognition model.

[0174] The prediction module is used to segment the second speech data of the test subject into speech slices that meet the target number; it is also used to determine the recognition result of the test subject based on the speech slices, the pre-trained preset emotion recognition model and the pre-trained fusion feature network model, and the recognition result includes at least the depression emotion score corresponding to the second speech data.

[0175] It should be understood that the depression emotion recognition device provided in the above embodiments is only illustrated by the division of the above functional modules during training and prediction. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0176] Moreover, the depression emotion recognition device and depression emotion recognition method provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the above-described embodiments of the depression emotion recognition method, which will not be repeated here.

[0177] Based on the above-described method for recognizing depressive moods, and employing the same technical concept, this application also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the above-described method for recognizing depressive moods.

[0178] Based on the above-described method for recognizing depressive moods, and employing the same technical concept, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described method for recognizing depressive moods.

[0179] Based on the aforementioned depressive mood recognition method and employing the same technical concept, this application provides a depressive mood recognition system. Figure 7 The diagram shows a structural schematic of a depressive mood recognition system provided in an embodiment of this application. Figure 7 As shown, the system includes a front-end device and a back-end processing device, which are connected via a network. The back-end processing device includes a first memory and a first processor. The first memory stores a computer program, and the first processor executes the computer program to implement the steps of the aforementioned depressive mood recognition method. The front-end device includes a display and a second processor; the display shows the recognition result of the test subject, which is obtained through the back-end processing device; the second processor receives the test subject's voice signal and sends it to the back-end processing device.

[0180] The front-end device can be a display device with a display, a handheld device, etc. In some embodiments, the front-end device also has a second memory for storing the collected voice data.

[0181] In some embodiments, this application also provides a depressive mood recognition system. Figure 8 This application provides a schematic diagram of the structure of another depressive mood recognition system, as shown in the embodiment of the present application. Figure 8 As shown, the system includes a front-end device and a back-end processing device, which are connected via a network. The front-end device includes a display module and an audio input module, while the back-end processing device includes a preprocessing module, a feature extraction module, a feature analysis module, and a feature fusion prediction module.

[0182] The audio input module in the front-end device can record voice data (i.e., the second voice data of the test subject) online and upload the voice data to the preprocessing module in the back-end processing device. Then, the voice data is processed by the preprocessing module, feature extraction module, feature analysis module and feature fusion prediction module in the back-end processing device to obtain the recognition result corresponding to the voice data. The recognition result is then returned to the display module of the front-end device to display the corresponding recognition result and can also provide emotion improvement methods.

[0183] The preprocessing module is used for sample slicing and sample balancing, as well as preprocessing of speech data. This preprocessing module balances the depressive and non-depressive data in the sample data and expands the number of original audio samples.

[0184] The feature extraction module is used to extract the corresponding speech features (i.e., initial features) from the sample slices or speech slices output by the processing module.

[0185] The feature analysis module and the feature fusion prediction module are used to train and extract high-level features of the audio; the high-level features extracted by the preset emotion recognition model are fused to give the final label.

[0186] In some embodiments, the voice data in the front-end device may also be recorded data stored in the front-end device.

[0187] In some embodiments, the front-end device includes a series of web interfaces, and the corresponding controls in the interfaces can realize functions such as voice data selection, voice data recording, voice data uploading, and analysis and processing. For example, by selecting the corresponding control for voice data, a data browser is displayed. After selecting the voice data to be uploaded, the selected data is uploaded by the corresponding control for uploading voice data. This completes the uploading of the second voice data of the subject in the front-end device to the back-end processing device. The analysis and processing control is used to complete the analysis of the voice data, and the device waits for the recognition result returned by the back-end processing device.

[0188] In some embodiments, the front-end device further includes a microphone. When the corresponding control for recording voice data is selected, microphone permission is obtained and recording is performed. After the recording is completed, the voice data obtained from the recording can be uploaded. The voice data can be analyzed by the corresponding control and the recognition result can be returned by the back-end processing device.

[0189] The embodiments of this application do not limit the way in which the functions in the front-end device are implemented; they can be implemented through mechanical buttons or touch screen controls, etc.

[0190] In some embodiments, voice data may be stored as a file on the front-end device and transmitted between the front-end device and the back-end processing device in the form of a file.

[0191] In some embodiments, the display module in the front-end device has a series of information display functions, such as displaying recognition results, displaying depression assessment indicators, and displaying mood improvement suggestions, and displays them on the web interface of the display screen. The result display function intuitively displays the predicted score, the depression assessment indicator display function divides the predicted score into several levels, which represent different levels of depression severity, and the mood improvement suggestion function provides different mood improvement suggestions based on the level of depression severity to help improve the depression condition.

[0192] The depressive mood recognition system provided in this application embodiment facilitates the collection and evaluation of voice data online. The segmentation and processing of voice data can improve the accuracy and efficiency of depressive mood recognition.

[0193] The following paragraphs will compare and list the Chinese terms used in this application specification and their corresponding English terms to facilitate reading and understanding.

[0194] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the discussion in some embodiments above is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the embodiments and various different variations of the embodiments suitable for specific application considerations.

Claims

1. A method for recognizing depressive mood, characterized in that, include: Acquire sample data, which includes the first voice data of each participant and the real labels of the first voice data; The first speech data is segmented by different numbers of slices to determine the sample slices corresponding to each segmentation process. The initial features corresponding to the sample slices are input into a preset emotion recognition model to be trained, and the predicted label output by the preset emotion recognition model is determined. The predicted label represents the depression score corresponding to the first speech data. The preset emotion recognition model to be trained includes a first emotion recognition model and a second emotion recognition model. The initial features are input into the first emotion recognition model to determine the first predicted label output by the first emotion recognition model, and the initial features are input into the second emotion recognition model to determine the second predicted label output by the second emotion recognition model. The first emotion recognition model is a model based on a convolutional neural network, and the second emotion recognition model is a model based on a long short-term memory network. Based on the first predicted label and the real label, determine the first target number of the trained first emotion recognition model and the sample slices corresponding to the first emotion recognition model; based on the second predicted label and the real label, determine the second target number of the trained second emotion recognition model and the sample slices corresponding to the second emotion recognition model. Obtain the first feature output from the intermediate layer of the trained first emotion recognition model; Obtain the second feature output from the intermediate layer of the trained second emotion recognition model; The first feature and the second feature are fused to determine the fused feature; The fused features are input into the fused feature network model to determine the trained fused feature network model; The second speech data of the test subject is divided into a first speech slice that meets the first target number, and the second speech data is divided into a second speech slice that meets the second target number. Based on the first speech slice, the second speech slice, the trained first emotion recognition model, the trained second emotion recognition model, and the trained fusion feature network model, the recognition result of the test subject is determined, and the recognition result includes at least the depression score corresponding to the second speech data.

2. The method for recognizing depressive mood according to claim 1, characterized in that, The step of segmenting each of the first speech data with different numbers of slices to determine the sample slices corresponding to each segmentation process includes: According to a preset segmentation strategy, each of the first voice data is segmented into a different number of sample slices, and the ratio of the number of depressed voice slices and the number of non-depressed voice slices in the sample slices is within a preset ratio range. The sample slice is preprocessed to determine the preprocessed sample slice. The preprocessing includes at least one of filtering, endpoint detection, and pre-emphasis. The filtering is used to filter the sample slice. The endpoint detection is used to filter the silent segment based on the intersection of the audio segment and the silent segment in the sample slice. The pre-emphasis is used to enhance the high-frequency information in the sample slice.

3. The method for recognizing depressive mood according to claim 2, characterized in that, The preset segmentation strategy includes a first duration segmentation strategy, a second duration segmentation strategy, and a third duration segmentation strategy, wherein the first duration is less than the second duration, and the second duration is less than the third duration; The step of dividing each voice data into a different number of sample slices according to a preset segmentation strategy includes: According to the first duration segmentation strategy, each of the voice data is segmented into a first number of sample slices; Based on the second duration segmentation strategy, each of the speech data is segmented into a second number of sample slices; Based on the third duration segmentation strategy, each of the speech data is segmented into a third number of sample slices.

4. The method for recognizing depressive mood according to claim 2, characterized in that, The process of determining the trained fusion feature network model includes: Obtain the target features of the sample slice output by the intermediate layer in the trained preset emotion recognition model, wherein the target features include the first feature and the second feature; The target features are input into the fusion feature network model to determine the target label of the speech data, and the target label represents the depression prediction score corresponding to the speech data. Based on the target label and the real label of the speech data, the trained fusion feature network model is determined.

5. The method for recognizing depressive mood according to claim 1, characterized in that, Before segmenting each of the first speech data with different numbers of slices, the process includes: The first speech data is preprocessed to determine the preprocessed first speech data. The preprocessing includes at least one of filtering, endpoint detection, and pre-emphasis. The filtering is used to filter the first speech data. The endpoint detection is used to filter silent segments based on the intersection of spoken and silent segments in the first speech data. The pre-emphasis is used to enhance the high-frequency information in the first speech data.

6. The method for recognizing depressive mood according to claim 1, characterized in that, The determination of the recognition result of the test subject based on the speech slice, the trained preset emotion recognition model, and the trained fusion feature network model includes: The initial features of the speech slice are input into the trained preset emotion recognition model to determine the target features of the speech slice output by the intermediate layer in the trained preset emotion recognition model. The target features include the first feature and the second feature. The target features of the speech slice are input into the trained fusion feature network model to determine the recognition result of the subject.

7. A depressive mood recognition system, characterized in that, The depressive mood recognition system includes a front-end device and a back-end processing device, which are connected via a network. The background processing device includes a first memory and a first processor. The first memory stores a computer program, and the first processor executes the computer program to implement the steps of the depressive mood recognition method according to any one of claims 1 to 6. The front-end device includes a display and a second processor; The display is used to show the identification results of the test subject, which are obtained through the background processing device; The second processor is used to receive the voice signal of the person being examined and send the voice signal of the person being examined to the background processing device.

Images