Depression detection method and system based on SGCN graph neural network learning

Abstract

The invention provides a depression detection method and system based on SGCN graph neural network learning, and the method comprises the steps: collecting a voice sample of a testee, and carrying out the preprocessing; related features are extracted from the preprocessed voice sample, and feature selection or dimension reduction processing is carried out according to prior knowledge and feature importance of depression related research; establishing a machine learning model for depression detection by using the marked training set; optimizing the model through cross validation and parameter adjustment; evaluating the optimized model by using an independent test set; by embedding the model into a practical application, including a mobile application program or an online platform, the feasibility and effect of the model in the real world are verified. The method can be used for a data joint training model of patients in different scenes, and can ensure the completeness and privacy of data, high prediction accuracy and rapid diagnosis.

Description

Technical Field

[0001] The present invention relates to the technical field of graph neural network learning, and in particular, to a depression detection method and system based on SGCN graph neural network learning. Background Art

[0002] In order to better understand and treat depression, researchers have conducted extensive research over the past few decades. The technologies currently involved in research on depression include: (1) Neuroimaging research: Neuroimaging techniques, such as functional magnetic resonance imaging (fMRI), electroencephalogram (EEG), and positron emission tomography (PET), allow researchers to observe and analyze the structural and functional differences in the brains of patients with depression, thereby revealing its pathological mechanisms. (2) Molecular biology and genetic research: Molecular biology and genetic research help to reveal the genetic basis of depression and its related molecular mechanisms. By studying gene expression, single nucleotide polymorphisms, etc., researchers can identify gene variants associated with the risk of depression and the prediction of the course of the disease. (3) Environmental and psychosocial factors research: In addition to genetic factors, environmental and psychosocial factors also play a key role in the occurrence and development of depression. Researchers use surveys, questionnaires, and psychological assessments to analyze the impact of different environmental and psychosocial factors on depression, such as stress, social support, and childhood trauma. (4) Drug treatment research: Drug treatment is one of the commonly used treatments for depression. Researchers conduct clinical trials and drug efficacy studies to explore and evaluate the efficacy and safety of various antidepressants, as well as their effects on patients' symptoms and quality of life. (5) Psychotherapy research: Psychotherapy is widely used in the treatment of depression, including cognitive behavioral therapy, illustration therapy, and psychodynamic therapy. Researchers study the effectiveness and mechanisms of different psychotherapies in order to provide patients with more personalized and effective treatment plans.

[0003] Patent document CN111447874A (application number: CN201880078144.5) discloses a discrimination device, a method for discriminating depressive symptoms, a method for judging the level of depressive symptoms, a method for classifying patients with depression, a method for judging the therapeutic effect of depressive symptoms, and a brain activity training device. Patent document CN111141863A (application number: CN201811314630.0) discloses a depression diagnostic marker and a preparation method thereof. The above two inventions are mainly proposed from the two aspects of brain science and medicine, but they are not analyzed from the perspective of behavior and machine learning speech. Summary of the invention

[0004] In view of the defects in the prior art, the purpose of the present invention is to provide a depression detection method and system based on SGCN graph neural network learning.

[0005] The depression detection method based on SGCN graph neural network learning provided by the present invention includes:

[0006] Step 1: Collect the subject’s speech samples;

[0007] Step 2: Preprocess the collected speech samples, including removing noise, standardizing the audio level, and reducing the frequency band range of the speech;

[0008] Step 3: Extract relevant features from the preprocessed speech samples;

[0009] Step 4: Among the extracted features, perform feature selection or dimensionality reduction based on prior knowledge of depression-related research and feature importance;

[0010] Step 5: Use the labeled training set to build a machine learning model for depression detection;

[0011] Step 6: Optimize the model through cross-validation and parameter adjustment, and use indicators including accuracy, recall, and F1 score for performance evaluation;

[0012] Step 7: Use an independent test set to evaluate the optimized model and calculate the confusion matrix, ROC curve, and AUC to evaluate the overall classification effect of the model;

[0013] Step 8: Verify the feasibility and effectiveness of the model in the real world by embedding it into practical applications, including mobile apps or online platforms.

[0014] Preferably, step 1 comprises: using a data acquisition device including a microphone and a mobile application to record the voice of the subject, ensuring that the acquisition environment is quiet, and noting the age, gender and depression condition of each subject.

[0015] Preferably, step 2 includes: selecting only individuals that have labels, physiological and behavioral information and features at the same time; if the missing data of a channel in the feature set exceeds half, deleting the channel; if the missing data in a row is less than half, filling the gaps with the average value of other channels in the row; accumulating the weights of different channels from high to low until the sum of the weights reaches a threshold of 0.5, and then taking the average of the accumulated channel values; filtering out data with an absolute value greater than 100 and replacing it with the median of the feature.

[0016] Preferably, step 4 includes: deleting information irrelevant to the correlation, including patient number and speech duration; constructing an adjacency matrix based on behavioral and physiological information, using the Pearson coefficient to calculate the correlation between each patient and all other patients, and if the coefficient exceeds a set threshold, the two patients are considered to be related in the adjacency matrix and an undirected connection is created.

[0017] Preferably, step 5 comprises:

[0018] Use SGCN training model to analyze and judge the input voice data to realize depression prediction function, including: converting audio files into graph structures, and then using graph convolutional neural network GCN to extract features and train the model; using summation GCN as OUTPUT function, and extracting more comprehensive representations from multiple vectors or multiple features to complete the binary classification task. The function of GCN is: in, is the adjacency matrix; is the degree matrix; H (l) represents the nodes of each layer; σ represents the activation function; W (l) Represents the network weight matrix of the lth layer in the network;

[0019] The three-layer structure is adopted. The process is as follows: obtain the characteristic representation X of the Graph node, calculate the adjacency matrix Input it into the three-layer GCN network to get the prediction results for each label: in, and The weight matrices of the first and second layers respectively; X represents the feature matrix; A represents the adjacency matrix; SoftMax() and ReLU() are activation functions;

[0020] Depression and non-depression are classified by binary cross entropy, and the expression is:

[0021]

[0022] Among them, y i is the label value of sample i, p i It is expressed as the probability that sample i is predicted as the positive class, and N is the number of samples predicted in this batch.

[0023] The depression detection system based on SGCN graph neural network learning provided by the present invention includes:

[0024] Module M1: Collecting speech samples from subjects;

[0025] Module M2: pre-processing the collected speech samples, including removing noise, standardizing the audio level, and reducing the frequency band range of the speech;

[0026] Module M3: extract relevant features from the preprocessed speech samples;

[0027] Module M4: Among the extracted features, feature selection or dimensionality reduction is performed based on prior knowledge of depression-related research and feature importance;

[0028] Module M5: Use the labeled training set to build a machine learning model for depression detection;

[0029] Module M6: Optimize the model through cross-validation and parameter adjustment, and use indicators including accuracy, recall and F1 score for performance evaluation;

[0030] Module M7: Use an independent test set to evaluate the optimized model, calculate the confusion matrix, ROC curve and AUC to evaluate the overall classification effect of the model;

[0031] Module M8: Validate the feasibility and effectiveness of the model in the real world by embedding it into practical applications, including mobile apps or online platforms.

[0032] Preferably, the module M1 includes: using a data acquisition device including a microphone and a mobile application to record the voice of the subject, ensuring that the acquisition environment is quiet, and noting the age, gender and depression condition of each subject.

[0033] Preferably, the module M2 includes: only selecting individuals that have labels, physiological and behavioral information and features at the same time; if the missing data of a channel in the feature set exceeds half, deleting the channel; if the missing data in a row is less than half, filling the gaps with the average value of other channels in the row; accumulating the weights of different channels from high to low until the sum of the weights reaches a threshold of 0.5, and then taking the average of the accumulated channel values; filtering out data with an absolute value greater than 100 and replacing it with the median of the feature.

[0034] Preferably, the module M4 includes: deleting information irrelevant to the correlation, including patient number and speech duration; constructing an adjacency matrix based on behavioral and physiological information, using the Pearson coefficient to calculate the correlation between each patient and all other patients, and if the coefficient exceeds a set threshold, the two patients are considered to be related in the adjacency matrix and an undirected connection is created.

[0035] Preferably, the module M5 comprises:

[0036] Use SGCN training model to analyze and judge the input voice data to realize depression prediction function, including: converting audio files into graph structures, and then using graph convolutional neural network GCN to extract features and train the model; using summation GCN as OUTPUT function, and extracting more comprehensive representations from multiple vectors or multiple features to complete the binary classification task. The function of GCN is: in, is the adjacency matrix; is the degree matrix; H(l) is the node representation of each layer; σ represents the activation function; W (l)Represents the network weight matrix of the lth layer in the network;

[0037] The three-layer structure is adopted. The process is as follows: obtain the characteristic representation X of the Graph node, calculate the adjacency matrix Input it into the three-layer GCN network to get the prediction results for each label: in, and The weight matrices of the first and second layers respectively; X represents the feature matrix; A represents the adjacency matrix; SoftMax() and ReLU() are activation functions;

[0038] Depression and non-depression are classified by binary cross entropy, and the expression is:

[0039]

[0040] Among them, y i is the label value of sample i, p i It is expressed as the probability that sample i is predicted as the positive class, and N is the number of samples predicted in this batch.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) GCN is used to process speech signal features, which can use the connection and graph structure information between nodes for message transmission and feature aggregation. By considering the characteristics and relationships of the node's neighbor nodes, GCN can capture the node's context information more comprehensively.

[0043] (2) The present invention aims to solve the problem of voice diagnosis of depression and provides a prediction method based on graph neural network. The method can be used for joint data training models of patients in different scenarios, which can ensure the completeness and privacy of the data, as well as high prediction accuracy and rapid diagnosis. It also explores the relationship between depression and voice intersection and uses voice analysis technology to assist in the diagnosis, prediction and treatment of depression. The application of voice analysis and machine learning methods in the field of depression will provide more accurate, real-time and non-invasive evaluation methods, thereby helping doctors and patients better understand and manage depression. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0045] Figure 1 Build graph for audio graph structure;

[0046] Figure 2 Implement graph for SGCN neural network framework;

[0047] Figure 3 This is the confusion matrix diagram of the prediction results;

[0048] Figure 4 ROC curve plot for classification of depressed and non-depressed patients;

[0049] Figure 5 Flowchart of the depression detection method based on graph neural network learning. DETAILED DESCRIPTION

[0050] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0051] Embodiment 1:

[0052] like Figure 5 The present invention provides a method for detecting depression based on graph neural network learning, comprising the following steps:

[0053] 1) Construction of graph structure

[0054] The construction of the graph structure is as follows Figure 1 As shown in the figure, the graph neural network is constructed for the compressed speech audio file. Figure 2 As shown in the figure, for the compressed speech audio file, the graph neural network is constructed. The audio file is first converted into a graph structure, and then the graph convolutional neural network is used to extract features and train the model. The audio signal is segmented, and each small segment of the signal is converted into a graph structure, where each node represents a time point of the audio signal, and the connection between the nodes represents the temporal adjacency relationship. The information on each node is processed, for example, by calculating time domain or frequency domain features, or by converting the signal into an image for processing. The local features of the audio signal are extracted, and then the node features in each round of GCN are updated according to the adjacency matrix and the node features, gradually integrating the information of the entire graph. Then the global features and relationships are extracted. Finally, depending on the task, an appropriate loss function is defined and trained using the back propagation algorithm. Since the compressed speech corpus is continuous data, it is processed as a basic temporal graph structure with edges on both sides of the central node. The graph structure constructed using the cosine similarity metric is used as the graph structure for depressive speech emotion recognition. The process of generating a graph based on cosine similarity is as follows:

[0055]

[0056] Among them, X i×j is the feature matrix, A i×j is the adjacency matrix, xi is the i-th row of the feature matrix X, is the threshold hyperparameter; s ij is the cosine similarity between node i and node j; a ij is the corresponding element of the adjacency matrix a.

[0057] 2) Model Architecture

[0058] The GCN method that best fits the task and aggregates the rich distribution is selected, and the nodes of the design space batch normalization, culling, activation, aggregation, layer connection, pre-processing layer message passing layer and post-processing layer are representatively calculated as follows:

[0059]

[0060] in, W p ∈R z×d;W e (k) ∈R z×z is a learnable weight matrix; b p ∈Rz is a learnable bias vector. h i (k) is the node representation of node i at the kth layer, k = 1, ..., K; z is the number of hidden units. The node representation cannot be directly used for graph classification tasks. Therefore, given the final iterative node representation, the output function is used to generate a graph representation. Finally, the graph symbol is predicted by the FC layer:

[0061]

[0062] Where W0∈RC×z is the learnable weight matrix; b0∈RC is the number of classes. OUTPUT can be a simple function of the variable, such as sum or mean, or maximum set. In the present invention, only two classification tasks are performed, and a more comprehensive representation is extracted from multiple vectors or multiple features to complete the task, so summation is used as the OUTPUT function.

[0063] The summation GCN is a variant of the semi-supervised GCN (semi-GCN). The equation of the semi-GCN is:

[0064]

[0065] 3) Feature extraction

[0066] The model has a GNN feature encoder f:S→Z, which takes the raw audio S as input and returns an embedding Z. These embeddings are used as node attributes in the graph. Labeled and unlabeled training data are used when building the graph, spectrograms are extracted using spectral graph convolution, etc., and then the node embeddings of the graph are built, and the model is not bound to any specific embeddings. During the audio feature embedding process, each audio clip is split into non-overlapping 960 millisecond segments. As input to the pre-trained GCN model. Each log-Meier spectrogram is processed using the 128-dimensional features extracted from the audio and averaged over all segments to form the final vector representation of each audio clip.

[0067] 4) Dataset

[0068] SJTU-LWD-DACD (Depression Audio Consultation Dataset from SJTU LiWeidong Lab), a text questionnaire (comorbidities and related risk factors, version 2.0) and interview audio dataset of "Major Depression Assessment" for 1,635 adult women. Finally, 1,440 data were collected, of which 1,135 were valid data, including 588 non-depressed patients with a total audio duration of 29 hours; 547 depressed patients with a total audio duration of 77 hours. The diagnosis was provided by professional doctors in the hospital.

[0069] 5) Test diagnostic accuracy

[0070] Different classifiers such as MFCC, eGeMaps and ComParE were used for the audio features of patients with depression and non-depression, and the BiLSTM classifier and the classifier SGCN were compared. The experimental results are shown in Table 1. The input high-dimensional audio features were statistically analyzed, some dimensional features were removed, and then input into the SGCN classifier for training. It can effectively filter out effective features and ensure accuracy. The best feature combination is "alcohol", "mania", and "social life", but the other differences are not obvious, and the experiment is highly random. The randomness of the experiment is very large. At the same time, confusion experiments were conducted on the ComParE and phonation feature sets. The experimental results are shown in Table 1. Figure 3 As shown in Table 1 and Table 2. From the experimental results, it can be seen that the predicted TP (True Positive) and TN (True Negative) are very high, reaching more than 93%. This also shows that the method of the present invention is effective in classifying patients with depression and non-depression. The confusion matrix, ROC curve and AUC can be calculated to evaluate the overall classification effect of the model. The confusion matrix for the classification of patients with depression and non-depression is shown in Figure 3 As shown, the ROC curve is Figure 4 shown.

[0071] Embodiment 2:

[0072] The present invention provides a method for detecting depression based on graph neural network learning, comprising:

[0073] 1. Construction of depression feature map

[0074] Step S1: Get audio feature embedding. Input spectrogram, H input ×W input ×C input Represents height, width, and number of channels respectively. Output H output ×W output ×C output The calculation process is three steps. output =(H intput -F+2P) / S+1,W output =(W input -F+2P) / S+1,C output =K, F is the side length of the spectrum graph, S is the stride, and K is the number of channels.

[0075] Step S2: embed the obtained features into a representation and input them into the network:

[0076]

[0077] Input feature embedding of the initial layer nodes: The embedding result obtained in the last layer is

[0078] 2. Model Construction

[0079] Step S1: The model is divided into three layers, the first layer is the node input layer, the second layer is the hidden layer, and the third layer is the classification layer. Finally, a trainable weight W is obtained. k , and the current node embedding representation during the iteration Multiply by the trainable weight B k .

[0080] Step S2: Get a matrix form. Then we have:

[0081] Among them, A v，： is the v rows of the adjacency matrix A of the audio feature graph.

[0082] Step S3: Construct a diagonal matrix D. v，v = Deg(v) = |N(v)|, whose inverse matrix is ​​D v，v =Deg(v)=|N(v)|, after iteration the matrix form is:

[0083]

[0084] 3. Training Process

[0085] Since the audio features in this article have been labeled, traditional supervised training is adopted. y is the classification label of depression, the node of the graph. The classification function uses SoftMax.

[0086] Table 1 Comparison of accuracy between BiLSTM classifier and SGCN classifier

[0087]

[0088] Table 2: Comparison of accuracy between BiLSTM classifier and SGCN classifier

[0089]

[0090]

[0091] Those skilled in the art know that, in addition to implementing the system, device and its various modules provided by the present invention in a purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, the system, device and its various modules provided by the present invention can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing the method and structures within the hardware component.

[0092] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A depression detection method based on SGCN graph neural network learning, characterized in that: include: Step 1: Collect the subject’s speech samples; Step 2: Preprocess the collected speech samples, including removing noise, standardizing the audio level, and reducing the frequency band range of the speech; Step 3: Extract relevant features from the preprocessed speech samples; Step 4: Among the extracted features, perform feature selection or dimensionality reduction based on prior knowledge of depression-related research and feature importance; Step 5: Use the labeled training set to build a machine learning model for depression detection; Step 6: Optimize the model through cross-validation and parameter adjustment, and use indicators including accuracy, recall and F1 score for performance evaluation; Step 7: Use an independent test set to evaluate the optimized model and calculate the confusion matrix, ROC curve, and AUC to evaluate the overall classification effect of the model; Step 8: Verify the feasibility and effectiveness of the model in the real world by embedding it into practical applications, including mobile apps or online platforms.

2. The method for detecting depression based on SGCN graph neural network learning according to claim 1, characterized in that: The step 1 includes: using data collection equipment including a microphone and a mobile application to record the voice of the subject, ensuring that the collection environment is quiet, and noting the age, gender and depression condition of each subject.

3. The method for detecting depression based on SGCN graph neural network learning according to claim 1, characterized in that: The step 2 includes: selecting only individuals that have labels, physiological and behavioral information and features at the same time; if more than half of the vacant data in a channel in the feature set are missing, deleting the channel; if less than half of the vacant data in a row are missing, filling the vacancies with the average value of other channels in the row; accumulating the weights of different channels from high to low until the sum of the weights reaches a threshold of 0.5, and then taking the average of the accumulated channel values; filtering out data with an absolute value greater than 100 and replacing it with the median of the feature.

4. The method for detecting depression based on SGCN graph neural network learning according to claim 1, characterized in that: The step 4 includes: deleting information irrelevant to the correlation, including patient number and speech duration; constructing an adjacency matrix based on behavioral and physiological information, using the Pearson coefficient to calculate the correlation between each patient and all other patients, and if the coefficient exceeds a set threshold, the two patients are considered to be related in the adjacency matrix and an undirected connection is created.

5. The method for detecting depression based on SGCN graph neural network learning according to claim 1, characterized in that: The step 5 comprises: Use SGCN training model to analyze and judge the input voice data to realize depression prediction function, including: converting audio files into graph structures, and then using graph convolutional neural network GCN to extract features and train the model; using summation GCN as OUTPUT function, and extracting more comprehensive representations from multiple vectors or multiple features to complete the binary classification task. The function of GCN is: in, is the adjacency matrix; is the degree matrix; H (l) represents the nodes of each layer; σ represents the activation function; W (l) Represents the network weight matrix of the lth layer in the network; The three-layer structure is adopted. The process is as follows: obtain the characteristic representation X of the Graph node, calculate the adjacency matrix Input it into the three-layer GCN network to get the prediction results for each label: in, and The weight matrices of the first and second layers respectively; X represents the feature matrix; A represents the adjacency matrix; SoftMax() and ReLU() are activation functions; Depression and non-depression are classified by binary cross entropy, and the expression is: Among them, y i is the label value of sample i, p i It is expressed as the probability that sample i is predicted as the positive class, and N is the number of samples predicted in this batch.

6. A depression detection system based on SGCN graph neural network learning, characterized in that: include: Module M1: Collecting speech samples from subjects; Module M2: pre-processing the collected speech samples, including removing noise, standardizing the audio level, and reducing the frequency band range of the speech; Module M3: extract relevant features from the preprocessed speech samples; Module M4: Among the extracted features, feature selection or dimensionality reduction is performed based on prior knowledge of depression-related research and feature importance; Module M5: Use the labeled training set to build a machine learning model for depression detection; Module M6: Optimize the model through cross-validation and parameter adjustment, and use indicators including accuracy, recall and F1 score for performance evaluation; Module M7: Use an independent test set to evaluate the optimized model, calculate the confusion matrix, ROC curve and AUC to evaluate the overall classification effect of the model; Module M8: Validate the feasibility and effectiveness of the model in the real world by embedding it into practical applications, including mobile apps or online platforms.

7. The depression detection system based on SGCN graph neural network learning according to claim 6 is characterized in that: The module M1 includes: using data acquisition equipment including a microphone and a mobile application to record the voice of the subject, ensuring that the acquisition environment is quiet, and noting the age, gender and depression condition of each subject.

8. The depression detection system based on SGCN graph neural network learning according to claim 6 is characterized in that: The module M2 includes: only selecting individuals with labels, physiological and behavioral information and features; if more than half of the vacant data in a channel in the feature set are missing, the channel is deleted; if less than half of the vacant data in a row are missing, the vacancies are filled with the average value of other channels in the row; the weights of different channels are accumulated from high to low until the sum of the weights reaches a threshold of 0.5, and then the average of the accumulated channel values ​​is taken; data with an absolute value greater than 100 is filtered out and replaced with the median of the feature.

9. The depression detection system based on SGCN graph neural network learning according to claim 6 is characterized in that: The module M4 includes: deleting information irrelevant to the correlation, including patient number and speech duration; constructing an adjacency matrix based on behavioral and physiological information, using the Pearson coefficient to calculate the correlation between each patient and all other patients, and if the coefficient exceeds a set threshold, the two patients are considered related in the adjacency matrix and an undirected connection is created.

10. The depression detection system based on SGCN graph neural network learning according to claim 6 is characterized in that: The module M5 comprises: Use SGCN training model to analyze and judge the input voice data to realize depression prediction function, including: converting audio files into graph structures, and then using graph convolutional neural network GCN to extract features and train the model; using summation GCN as OUTPUT function, and extracting more comprehensive representations from multiple vectors or multiple features to complete the binary classification task. The function of GCN is: in, is the adjacency matrix; is the degree matrix; H (l) represents the nodes of each layer; σ represents the activation function; W (l) Represents the network weight matrix of the lth layer in the network; The three-layer structure is adopted. The process is as follows: obtain the characteristic representation X of the Graph node, calculate the adjacency matrix Input it into the three-layer GCN network to get the prediction results for each label: in, and The weight matrices of the first and second layers respectively; X represents the feature matrix; A represents the adjacency matrix; SoftMax() and ReLU() are activation functions; Depression and non-depression are classified by binary cross entropy, and the expression is: Among them, y i is the label value of sample i, p i It is expressed as the probability that sample i is predicted as the positive class, and N is the number of samples predicted in this batch.

Images