Depression tendency detection system based on natural gait timing analysis

Abstract

The present invention provides a depression tendency detection system based on natural gait time series analysis, and relates to the technical field of depression tendency detection. The system includes a data acquisition module, a data processing module, a feature extraction and sorting module, a model construction module, a model training and optimization module, and a depression tendency detection result output module. By collecting the psychological state data and gait data of the subject, and extracting the gait features of the gait data, then training the depression tendency detection model, determining the optimal number of features and the optimal feature subset, thereby obtaining the optimal depression tendency detection model for individual depression tendency detection. Compared with the traditional depression detection technology based on questionnaire scales and doctor interviews, this system can be more scientific, objective, and effective in assisting diagnostic decision-making, and the system is more accurate in assessing depression tendency, the method is simple, and the cost is low. It is suitable for early screening of the mental health status of a large population, individual early warning, and assisting medical staff in decision-making.

Description

Technical Field

[0001] The present invention relates to the technical field of depression tendency detection, and in particular to a depression tendency detection system based on natural gait time series analysis. Background Art

[0002] The early stage of depression is called depressive tendency. People with depressive tendency show depression but have not yet reached the clinical diagnosis standard. Early detection, early diagnosis and early treatment are the key to curing depression. Therefore, a new method is urgently needed to detect the depressive tendency of individuals in a non-intrusive, accurate and objective manner.

[0003] At present, the clinical identification and diagnosis of depression is mainly based on certain diagnostic criteria, combined with the individual's medical history, interview situation and questionnaire scale scores to give a comprehensive diagnosis result; and some non-contact depression detection methods are mainly based on clinical semi-structured depression interview videos, extracting facial motion units and facial feature point data, and after a series of processing, outputting depression detection results.

[0004] However, the method of filling out questionnaires cannot truly reflect the mental health status of individuals because they may exaggerate or weaken their depression state voluntarily or involuntarily. Some non-contact depression detection methods require individuals to be stimulated to produce specific emotions before facial behavior data can be obtained. This method requires the collection of close-up, frontal facial video data of individuals, which has certain limitations in application scenarios, and the self-protection awareness of individuals may also lead to inaccurate data collection. In addition, most depression detection methods such as the above two methods are based on data collected at a certain point in time to detect and judge the depression state of individuals. In reality, most depression data of individuals are nonlinear, non-stationary and dynamically changing. Therefore, the method of obtaining individual depression detection results only through single detection data will lead to inaccurate detection results and large errors. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the deficiencies of the prior art, the present invention provides a depression tendency detection system based on natural gait timing analysis, which solves the problem of low accuracy of depression detection results in the prior art depression detection technology.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] The present invention provides a depression tendency detection system based on natural gait time series analysis, the system comprising:

[0010] The data collection module is used to continuously collect the gait video data of the subjects in different time periods and obtain the depression tendency labels of the subjects;

[0011] A data processing module, used for processing the gait video data to obtain the gait data of the subject in a complete gait cycle;

[0012] A feature extraction and sorting module, used for extracting the gait features of the subject based on the gait data, and sorting the gait features by calculating the importance and relevance of the gait features;

[0013] A model building module, used to build a depression tendency detection model, wherein the depression tendency detection model includes an input layer, a hidden layer, a Softmax layer, and an output layer;

[0014] A model training and optimization module, which trains the depression tendency detection model based on the sorted gait features and the depression tendency labels to obtain an optimal depression tendency detection model;

[0015] The depression tendency detection result output module is used for the subject to perform depression tendency detection based on the optimal depression tendency detection model and output the depression tendency detection result.

[0016] Preferably, the hidden layer comprises a stacked 2-layer LSTM network structure.

[0017] Preferably, the data acquisition module includes a gait data acquisition unit and a psychological state acquisition unit.

[0018] Preferably, the gait data collection unit includes a video camera, and the video camera is used to collect the gait data of the subject walking naturally in a designated area at different time periods;

[0019] The psychological state collection unit includes an input unit, and the PHQ-9 scale filled in by the subject is input using the scale input unit.

[0020] Preferably, when the data processing module processes the gait video data to obtain the gait data of the subject in a complete gait cycle, it includes:

[0021] S31, detecting and tracking the coordinates of the two-dimensional skeleton joint points of the subject in the gait video frame, and acquiring the three-dimensional coordinates of the body joint points of the subject in the gait video through binocular camera depth estimation based on the two-dimensional skeleton joint point coordinates;

[0022] S32, performing coordinate system conversion, standardization, and filtering and denoising processing on the three-dimensional coordinates in the gait video;

[0023] S33, intercepting the gait video processed in S32 to obtain gait data of the subject in a complete gait cycle; the gait data includes joint coordinate data.

[0024] Preferably, the gait features include: spatiotemporal features, motion features, time domain features and frequency domain features.

[0025] Preferably, when the feature extraction and sorting module extracts the gait features of the subject based on the gait data and sorts the gait features by calculating the importance and relevance of the gait features, it includes:

[0026] S41, calculating the importance of the gait features using a random forest algorithm, and calculating the correlation of the gait features using a Pearson correlation analysis;

[0027] S42: sum up the absolute values ​​of the importance and correlation of the gait features and sort them from high to low.

[0028] Preferably, when the model training and optimization module trains the depression tendency detection model based on the sorted gait features and the depression tendency labels to obtain the optimal depression tendency detection model, it includes:

[0029] S51, based on the sorted gait features, successively select the top m i+1 =m i +1 gait feature combined with the depressive tendency label to train the depressive tendency detection model until all the gait features are selected and the model training is completed; wherein, m i Indicates that the number of gait features currently selected is m i pcs, m i The initial value of is 0;

[0030] S52, compare the AUC values ​​of the corresponding gait feature quantity obtained by running the model after each model training in S51, and select the quantity m corresponding to the maximum AUC value i As the optimal number of features m good ; Based on the comprehensive ranking of gait characteristics, select the top m good gait features as the optimal feature subset;

[0031] S53, based on the optimal number of features m good and the optimal number of features m good The corresponding optimal feature subset and the depression tendency detection model after the depression tendency label training are the optimal depression tendency detection model.

[0032] Preferably, the depressive tendency detection result includes the probability of the depressive tendency belonging to different depression severity level categories.

[0033] (III) Beneficial effects

[0034] The invention provides a depressive tendency detection system based on natural gait time series analysis.

[0035] Compared with the prior art, it has the following beneficial effects:

[0036] 1. A depression tendency detection system based on natural gait time series analysis proposed by the present invention includes a data acquisition module, a data processing module, a feature extraction and sorting module, a model building module, a model training and optimization module, and a depression tendency detection result output module. By collecting the psychological state data and gait data of the subject, and extracting the gait features of the gait data, then training the depression tendency detection model, determining the optimal number of features and the optimal feature subset, thereby obtaining the optimal depression tendency detection model for individual depression tendency detection. Compared with the traditional depression detection technology based on questionnaire scales and doctor interviews, this system can assist in diagnostic decision-making more scientifically, objectively and effectively, and compared with the non-contact facial depression state detection method, the system only needs to collect the gait video data of the subject in the natural state, without emotional induction or adding other interference to the subject, and can achieve a more accurate assessment of its behavioral data and depression tendency, and its detection method is simple and low-cost, suitable for early screening of the mental health status of a large population, early warning of individuals, and providing auxiliary decision-making for psychiatric specialists.

[0037] 2. The present invention takes into account the dynamic variability of an individual's depressive state. During data collection, video data of the subjects walking naturally in a designated area is collected at a fixed time each week to form time series data. The PHQ-9 scale score filled in by the subjects is used as their depressive tendency label. Then, an LSTM network model is constructed to analyze the correlation between gait characteristics and depressive tendencies, and the time series information of the features is mined to improve the accuracy of depression detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 is a structural diagram of a depression tendency detection system based on natural gait timing analysis in an embodiment of the present invention;

[0040] Figure 2 Schematic diagram of gait data collection in an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of using OpenPoseBody25 to obtain two-dimensional motion information of body joints in an embodiment of the present invention;

[0042] Figure 4 Schematic diagram of a binocular camera depth estimation model in an embodiment of the present invention;

[0043] Figure 5 The figure is a framework diagram of the LSTM model constructed in the embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The embodiment of the present application solves the problem of low accuracy of depression detection results in existing depression detection technologies by providing a depression tendency detection system based on natural gait timing analysis, and achieves the purpose of accurately judging an individual's depression tendency by collecting individual gait indicators.

[0046] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0047] The existing depression detection technology has limitations in data collection, data collection is not objective, is susceptible to interference, and does not take into account the nonlinear, non-stationary and dynamic characteristics of depression data, which will result in low accuracy of the final depression detection results. In order to solve these problems, the present invention proposes a depression tendency detection system based on natural gait time series analysis, which includes a data acquisition module, a data processing module, a feature extraction and sorting module, a model construction module, a model training and optimization module, and a depression tendency detection result output module. The system collects the psychological state data and gait data of the subject, and extracts the gait features of the gait data, and then uses the gait features to train the depression tendency detection model, determine the optimal number of features and the optimal feature subset, and obtain the optimal depression tendency detection model for individual depression tendency detection. The system can well solve the problems existing in the prior art, can more accurately detect the depression tendency of an individual, and assist doctors in making decisions.

[0048] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0049] Embodiment 1:

[0050] The present invention proposes a depressive tendency detection system based on natural gait time series analysis, see Figure 1 The system comprises:

[0051] The data collection module is used to continuously collect the gait video data of the subjects in different time periods and obtain the depression tendency labels of the subjects;

[0052] A data processing module, used for processing the gait video data to obtain the gait data of the subject in a complete gait cycle;

[0053] A feature extraction and sorting module, used for extracting the gait features of the subject based on the gait data, and sorting the gait features by calculating the importance and relevance of the gait features;

[0054] A model building module, used to build a depression tendency detection model, wherein the depression tendency detection model includes an input layer, a hidden layer, a Softmax layer, and an output layer;

[0055] A model training and optimization module, which trains the depression tendency detection model based on the sorted gait features and the depression tendency labels to obtain an optimal depression tendency detection model;

[0056] The depression tendency detection result output module is used for the subject to perform depression tendency detection based on the optimal depression tendency detection model and output the depression tendency detection result.

[0057] It can be seen that the depression tendency detection system based on natural gait timing analysis proposed in this embodiment includes a data acquisition module, a data processing module, a feature extraction and sorting module, a model building module, a model training and optimization module, and a depression tendency detection result output module. By collecting the psychological state data and gait data of the subject, and extracting the gait features of the gait data, and then training the depression tendency detection model, the optimal number of features and the optimal feature subset are determined, so as to obtain the optimal depression tendency detection model for individual depression tendency detection. Compared with the traditional depression detection technology based on questionnaire scales and doctor interviews, this system can assist in diagnostic decision-making more scientifically, objectively and effectively, and compared with the non-contact facial depression state detection method, the system only needs to collect the gait video data of the subject in the natural state, without emotional induction or adding other interference to the subject, and can achieve a more accurate assessment of its behavioral data and depression tendency, and its detection method is simple and low-cost, suitable for early screening of the mental health status of a large population, early warning of individuals, and providing auxiliary decision-making for psychiatric specialists.

[0058] The following is combined with Figure 1-5, and a specific explanation of each module in the system, to explain in detail the implementation process of an embodiment of the present invention. A depression tendency detection system based on natural gait timing analysis of the present invention includes: a data acquisition module, a data processing module, a feature extraction and sorting module, a model building module, a model training and optimization module, and a depression tendency detection result output module. Specifically, the specific role of each module and its specific implementation process when working are as follows:

[0059] The data collection module is used to continuously collect the gait video data of the subjects in different time periods and obtain the depression tendency labels of the subjects.

[0060] Generally, a person's depression state changes dynamically. Compared with a single test, the gait data of the subject is collected multiple times in different time periods to analyze his depression state in the past period of time and detect his current depression tendency. This can take into account the temporal nature of depression changes and has higher accuracy. Therefore, when collecting data in this embodiment, the psychological state and gait data of the subject are collected once a week at a fixed time, and collected 4 times in total for 28 consecutive days. The data collection module mainly includes two aspects when collecting data:

[0061] On the one hand, the PHQ-9 scale data filled in by the subjects are collected, and then the depression severity level corresponding to the scale score is used as label data. In this embodiment, according to the PHQ-9 scale score results, the severity level of depression tendency can be divided into five categories: no depression, mild depression, moderate depression, moderate to severe depression and severe depression.

[0062] On the other hand, a video camera is used to collect gait video data of the subject when he / she walks naturally in a designated area. When a video camera is used to collect gait video data of the subject, the specific scheme of this embodiment is as follows: Figure 2 As shown, a 10-meter-long, well-lit corridor is prepared, and two groups of cameras C1 and C2 are installed at intervals of 20 cm on both sides of the ceiling midline corresponding to 7 meters and 10 meters of the corridor. Each group includes two cameras, and the camera resolution is 1920×1080, the frame rate is 30fps, and the vertical viewing angle is 60°. In order to obtain the best data acquisition field of view, the inclination angle between the camera and the ground is set to θ=30°. When formally collecting the subject's gait data, the subject faces the camera and walks from one end of the corridor to the other end, walking naturally, and only one walk is required. When using this method to collect the subject's gait data, there is no need to give the individual a certain stimulus to guide the individual to produce specific emotions. It is simple and convenient;

[0063] This method uses a camera installed on the ceiling to collect gait video data of an individual walking naturally, which can avoid the problem that when collecting facial video data of the subject at close range and from the front, the collected data cannot truly reflect the mental health status of the subject due to self-protection awareness, hiding symptoms, rebellious psychology, etc. Therefore, the method of collecting gait data of the subject in this embodiment can more naturally and truly reflect his mental state.

[0064] The data processing module is used to process the gait video data to obtain the gait data of the subject in a complete gait cycle.

[0065] When the data processing module processes the gait video data of the subject obtained by the data acquisition module, the data processing module mainly includes the following steps:

[0066] (1) Obtain 3D coordinate information. Use OpenPoseBody25 to detect the coordinates of the 2D skeleton joints of the human body in the tracking video frame, such as Figure 3 As shown in the figure, the two-dimensional coordinate information of 25 body joints such as nose, neck, shoulder, elbow, wrist, and hip is detected. Figure 4 As shown in Figure 1, the depth value of each joint point is obtained through the binocular camera depth estimation model, and the three-dimensional coordinate information of the body joint point can be obtained after alignment. The specific process of depth estimation is as follows: P is a point in space, P 1 and P 2 is the image point of point P on the left and right image planes, f is the focal length, O L and O R is the optical center of the left and right cameras. After calibration, the left and right cameras reach parallel epipolar lines and parallel optical axes. L and X R is the distance between the two imaging points on the left and right image planes and the left edge of the image. Then the depth value Z of the spatial point P can be calculated:

[0067]

[0068]

[0069] Among them, b is the distance of the optical center of the binocular camera, that is, the baseline of the binocular camera; d is X L -X R , the difference between the horizontal coordinates of the left and right images is called parallax.

[0070] (2) Coordinate system conversion. Considering that the camera is located on the ceiling and the angle θ between it and the ground is 30°, in order to avoid the overlap of individual body joints due to the tilt angle and thus affect the analysis, the original coordinates are first rotated to the horizontal direction. (x', y', z') and (x, y, z) represent the coordinates after rotation transformation and the original coordinates, respectively.

[0071]

[0072] (3) Standardization. In order to eliminate the interference of the subject's body shape and walking position changes on the gait data, the above-mentioned converted coordinate data needs to be standardized. The individual's spinal joint points are used as the coordinate origin, and the body joint points are converted to the same coordinate system.

[0073] Suppose the coordinates of the vertebral joint point in a certain frame of image are p' 0 (x' 0 ,y' 0 ,z' 0 ), the coordinates of the other joint points are p'(x',y',z'), and the coordinates of the other joint points after coordinate transformation are p"(x",y",z"), and there exists p"(x",y",z")=p'(x',y',z')-p' 0 (x' 0 ,y' 0 ,z' 0 ), repeat the operation for all frames to achieve standardized processing of all transformed coordinate data.

[0074] (4) Filtering and denoising: Use low-pass filtering to remove high-frequency components and noise in video data.

[0075]

[0076] Where x is the three-dimensional coordinate record value of each joint point, g(t) is the filter function, n is the number of frames, and y(n) is the filtered signal. In this embodiment, a Gaussian filter is used for denoising, and the convolution kernel

[0077] (5) Clip capture. A gait cycle is defined as the process from the left heel leaving the ground to the left heel touching the ground again when the subject walks. Studies have shown that the average natural gait frequency of normal people is about 95-125 steps per minute, and a gait cycle is about 0.79-1.04 seconds. Considering that people with depressive tendencies walk slower and have longer gait cycles, in order to ensure that at least a complete gait cycle video is captured, continuous gait data with a length of 3 seconds and a total of 90 frames is captured for analysis. The specific operations are as follows:

[0078] 1) In order to ensure the validity of the video data and the appropriate proportion of the screen during the subject's walking, the initial effective shooting range of the camera is set to 5 meters, with the distance from the vertebral joint point to the camera group C 1The depth value of 5 meters is used as the demarcation point, the corresponding timestamp is found, and the data before this moment is deleted. Then, the OpenPoseBody25 body joint detection confidence of 95% is used as the threshold. If the confidence is ≥ 95%, it is used as the starting point of valid video data; if the confidence is < 95%, it is moved back along the video frame until the threshold is reached, which is used as the starting point of valid video data, and the data before this moment is deleted.

[0079] 2) In order to align the video data, the starting point is the left foot in front and the left heel is on the ground. Find the local minimum point of the left heel coordinate on the y-axis and find the corresponding timestamp T 1 , taking this as the starting point, a gait segment with a duration of 3 seconds is captured, that is, 90 frames.

[0080] 3) Considering the different walking speeds of the subjects, if camera group C 1 If the effective video data is less than 3 seconds, camera group C will be used 2 The video data is spliced ​​and supplemented.

[0081] After the data processing module performs the above-mentioned coordinate acquisition, coordinate system conversion, standardization, filtering and denoising, and segment interception on the gait video data of the subject, the gait data of each subject within at least one complete gait cycle can be obtained. The gait data is the joint coordinate data of a fixed number of frames.

[0082] The feature extraction and sorting module is used to extract the gait features of the subject based on the gait data, and sort the gait features by calculating the importance and relevance of the gait features.

[0083] 1) Extract the spatiotemporal features, motion features, time domain features and frequency domain features of gait data.

[0084] Since the gait characteristics associated with depression are increased body sway, reduced stride length and slower walking speed, in order to extract feature values ​​that can characterize the depressive state from the data, this embodiment focuses on the behavioral data of 25 body joints and mainly extracts four aspects of the subject's gait data: spatiotemporal characteristics, motion characteristics, time domain characteristics and frequency domain characteristics.

[0085] (1) Spatial-temporal feature extraction.

[0086] F1_speed: defined as the movement speed of the midpoint of the hip along the z-axis;

[0087] Step width F1_width: defined as the position difference between the left and right heels on the x-axis when both feet touch the ground at the same time in a gait cycle;

[0088] Body sway F1_sway: defined as the maximum position difference between the left shoulder and the right shoulder on the x-axis during a gait cycle;

[0089] Head posture F1_head: defined as the angle between the line connecting the nose and the neck and the plane formed by the y-axis and the z-axis in one gait cycle;

[0090] Left / right arm swing F1_Larm / Rarm: defined as the maximum position difference between the left and right wrists on the z-axis in a gait cycle;

[0091] Left / right leg stride F1_Lstride / Rstride: defined as the maximum position difference between the left and right feet on the z-axis in a gait cycle;

[0092] Left / right leg raising F1_Lraise / Rraise: defined as the maximum position difference between the left and right feet on the y-axis during a gait cycle.

[0093] Since the step frequency, stride length, etc. of each subject are different, the average value of the spatiotemporal characteristics in the corresponding different gait cycles is calculated for each subject, and finally 10 spatiotemporal characteristics are obtained.

[0094] (2) Movement feature extraction.

[0095] Considering that the wrists, ankles, and head move with large amplitudes during walking, the movement speed and acceleration of the five body joints, namely the left / right wrist, left / right ankle, and nose, are calculated. The specific formula is as follows:

[0096]

[0097] Acceleration i =|Speed i -Speed i-1 |

[0098] Among them, D i Indicates the coordinate position of the joint point in the i-th frame. Get the joint point speed sequence (Speed 1 ,Speed 2 ,...,Speed 89 ), joint acceleration sequence (Acceleration 2 ,Acceleration 3 ,...,Acceleration 89 ), the maximum value, minimum value, mean value and standard deviation of each sequence are selected as motion features, and finally 5×2×4=40 motion features are obtained.

[0099] (3) Time domain feature extraction.

[0100] Gait data is a time-related motion process, so it can be processed using time-frequency analysis techniques in signal processing. Extract time domain features and calculate the mean of 25 body joints on the x, y, and z axes. Standard Deviation (S), Kurtosis (K) and Skewness (SK). Among them, the mean measures the central tendency of the data; the standard deviation measures the dispersion of the data; the skewness measures the symmetry of the data; and the kurtosis measures the abnormal values ​​of the data distribution. The specific calculation formula is as follows:

[0101]

[0102]

[0103]

[0104]

[0105] The statistical features of the subject’s 25 body joints on the x, y, and z axes are calculated, and finally 25×3×4=300 time domain features are obtained.

[0106] (4) Frequency domain feature extraction.

[0107] Frequency domain features can better reflect the regularity and periodicity of gait data. The time domain signal is converted into a frequency domain signal through Fourier transform, and the signal mean, mean, standard deviation, kurtosis and skewness of amplitude and phase are calculated, and finally 25×3×(1+2×4)=675 frequency domain features are obtained.

[0108] 2) Comprehensively rank the gait features by calculating their importance and relevance.

[0109] After the above feature extraction, a large number of spatiotemporal features, motion features, time domain features, and frequency domain features of the subject's gait data are extracted. In order to remove redundant features that affect the model effect and make the model simpler and more efficient, it is necessary to screen out important features. Specifically, the spatiotemporal features, motion features, time domain features, and frequency domain features are first merged into an overall feature set, and then the feature importance and Pearson correlation analysis of the random forest algorithm are combined to obtain a comprehensive feature ranking. Specifically:

[0110] (1) Calculate feature importance. Random forest is a supervised classification algorithm that learns the relationship between data features and labels. It randomly extracts n samples with replacement from the original training sample set N to generate a new training sample set for training decision trees. After repeated attempts, a random forest consisting of multiple trees is obtained. The feature importance of random forest can select the features that have the greatest impact on the results. The specific process is as follows:

[0111] 1.1) Calculate the Gini index for each feature:

[0112]

[0113] Among them, K means that there are K categories at the feature node e, p ek Indicates the proportion of category k in node e.

[0114] 1.2) Considering that there are more samples at the node, the Gini index of the node is weighted, and the feature X is added with the weight of the node j The feature importance at node e is:

[0115]

[0116] Among them, n represents the number of samples of the node, N represents the total number of samples, and GI f and GI t They represent the Gini index of the two new nodes after the node splits.

[0117] 1.3) If feature X j The node that appears in decision tree i is in set E, then X j The importance of the i-th tree is:

[0118]

[0119] 1.4) If there are h trees and m features in the random forest, after normalization, the final feature X j The importance in random forest is:

[0120]

[0121] (2) Calculate feature correlation. Perform Pearson correlation analysis to calculate the correlation between each feature Feature (F) and the depression label class Class (C):

[0122]

[0123] Among them, cov(F,C) represents the covariance of F and C; σ(F) and σ(C) are the standard deviations of F and C respectively.

[0124] (3) Comprehensive ranking of features. Since the importance and relevance of features are in the same dimension, the importance and absolute value of the relevance of the features are summed up and then ranked to obtain the comprehensive ranking of the features.

[0125] The model building module is used to build a depression tendency detection model, wherein the depression tendency detection model includes an input layer, a hidden layer, a Softmax layer, and an output layer.

[0126] This embodiment constructs a depression tendency detection model based on the Long Short-Term Memory (LSTM) network. The LSTM network is a deep learning model commonly used to process time series data. According to the gait characteristics of the subjects, the psychological state of the subjects at different times is dynamically modeled through the LSTM network to construct important gait indicators related to depression tendencies. In addition, after obtaining the gait data of the subjects over a period of time, their depression state trends can be analyzed according to the model, which is helpful for judging whether they have depression tendencies and how severe they are. It can be used to warn individuals and provide auxiliary decision-making for psychiatric medical staff. Therefore, this embodiment constructs a depression tendency detection model based on the LSTM network. The overall architecture of the depression tendency detection model is as follows: Figure 5 As shown, it is divided into 4 parts: input layer, hidden layer, Softmax layer and output layer.

[0127] Input layer: Combine gait data and depression label data to construct time series data for neural network training. The input sequence length is fixed. The input sequence is: X 1 ={a 1},X 2 ={a 2},X 3 ={a 3},X 4 ={a 4}, where a i represents the gait characteristics of the subject at the i-th acquisition moment, a i =(x 1 ,x 2 ,...,x M ), where x 1 ,x 2 ,...,x M Represents the M features in the gait feature subset.

[0128] Hidden layer: In order to prevent overfitting and improve the generalization ability of the network, this embodiment adopts a stacked 2-layer LSTM network structure. The number of neurons in the hidden layer q can be obtained according to the empirical calculation formula: Where a and b are the number of neurons in the input layer and output layer respectively; c is an integer ranging from 1 to 10.

[0129] Softmax layer: learns the information transmitted by the hidden layer, calculates the probability of the data to be classified belonging to each category, and divides the severity of depression tendency into five categories according to the PHQ-9 scale score results: no depression, mild depression, moderate depression, moderate to severe depression and severe depression.

[0130] Output layer: The final output is the depression tendency detection category of the given gait feature. The output sequence is O 1 =(b 1 ),O 2 =(b 2 ),O 3 =(b 3 ),O 4 =(b 4 ), where b i represents the depression tendency of the subject at the i-th collection time, b i =(y 1 ,y 2 ,...,y 5 ), where y 1 ,y 2 ,...,y 5 Represents the probability of a depressive tendency belonging to one of the five severity levels.

[0131] The model training and optimization module trains the depression tendency detection model based on the sorted gait features and the depression tendency labels to obtain an optimal depression tendency detection model.

[0132] The model training and optimization process mainly uses gait feature subsets with different feature numbers and depression tendency labels to train the depression tendency detection model, so as to select the optimal number of features and the corresponding optimal feature subset. The depression tendency detection model is trained based on the optimal number of features, the corresponding optimal feature subset and the depression tendency label, and the optimal depression tendency detection model can be obtained. It mainly includes the following steps:

[0133] S51, based on the gait features after the above sorting, select the top m i+1 =m i +1 gait feature combined with the depression tendency label to train the depression tendency detection model until all gait features are selected and the model training is completed, and all training processes are terminated; wherein, m i Indicates that the number of the currently selected gait features is m i The corresponding optimal feature subset is the first m i Features. iThe initial value of can be 0 (i.e., the model is currently trained from the first feature), or any other positive integer selected according to actual needs. In actual operation, in order to reduce the number of training times, according to previous experimental experience, m can be taken i The initial value of is 20, and then 21, 22, ..., 100 are taken as the number of features. That is, the first time the first 20 gait features are taken in combination with the depression tendency label to build and train the model to obtain the training result, and the second time the first 21 features are taken in combination with the depression tendency label to build and train the model to obtain the training result...

[0134] For n subjects, data was collected 4 times for 28 consecutive days to obtain the gait characteristics and corresponding depression tendency time series data of each subject, and these data were used as sample data to train and optimize the depression tendency detection model based on the LSTM network constructed above. Each time, the feature subset selected according to the above operation was used as input, and the depression tendency label obtained from the corresponding subject questionnaire scale score was used as output. The sample data was divided into a training set and a test set at a ratio of 80% and 20%, and the ADAM algorithm was used for optimization, and the cross entropy was used as the loss function to train the depression tendency detection model.

[0135] S52, compare the AUC values ​​of the corresponding gait feature quantity obtained by running the model after each depression tendency detection model training in S51, and select the quantity m corresponding to the maximum AUC value i As the optimal number of features m good ; Based on the comprehensive ranking of gait characteristics, select the top m good gait features as the optimal feature subset.

[0136] The AUC values ​​of the corresponding gait feature quantity obtained by running the model after each training of the depression tendency detection model in S51 are compared, where the m corresponding to the maximum AUC value is i The value is the optimal number of features m good The specific process is as follows:

[0137] In order to measure the performance of the trained depression tendency detection model, the ROC curve and AUC value were selected to comprehensively evaluate the algorithm performance.

[0138] The ROC curve is a line connecting points with specificity (FPR) on the horizontal axis and sensitivity (TPR) on the vertical axis.

[0139]

[0140]

[0141] TP, TN, FP, and FN are positive samples correctly classified by the model, negative samples correctly classified by the model, positive samples incorrectly classified by the model, and negative samples incorrectly classified by the model. AUC is the area under the ROC curve. The AUC value is calculated. The larger the AUC value, the better the performance of the current classification algorithm.

[0142] S53, based on the optimal number of features m good and the optimal feature subset to obtain the optimal depression tendency detection model.

[0143] Finally, the optimal number of features m corresponding to the maximum AUC value is selected good The training model is the model with the best training results. The final model determines that the number of features is the optimal number of features m. good , thus the optimal depression tendency detection model can be established, and the model training and optimization process is completed.

[0144] The depression tendency detection result output module is used for the subject to perform depression tendency detection based on the optimal depression tendency detection model and output the depression tendency detection result.

[0145] The optimal depression tendency detection model trained and optimized by the above-mentioned model training and optimization module can be used to detect the depression tendency of an individual. During the specific detection, after collecting the subject's gait data for multiple consecutive times in different time periods, the data is then used as the input of the optimal depression tendency detection model to obtain the individual's depression tendency detection result. The depression tendency detection result includes the probability of the depression tendency belonging to different depression severity level categories, that is, the probability of the depression tendency belonging to the five categories of no depression, mild depression, moderate depression, moderate to severe depression and severe depression. The category with the highest probability can be used as the depression tendency corresponding to the individual.

[0146] The depression tendency detection system of the present invention is based on natural gait timing analysis. The detection method is simple and low-cost, and is suitable for early screening of mental health conditions of a large population. It can quickly and accurately determine whether an individual has a depressive tendency and how severe it is, thereby providing an early warning to the individual; in addition, the depression tendency detection results can also provide auxiliary decision-making for psychiatric medical staff.

[0147] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0148] 1. A depression tendency detection system based on natural gait time series analysis proposed by the present invention includes a data acquisition module, a data processing module, a feature extraction and sorting module, a model building module, a model training and optimization module, and a depression tendency detection result output module. By collecting the psychological state data and gait data of the subject, and extracting the gait features of the gait data, then training the depression tendency detection model, determining the optimal number of features and the optimal feature subset, thereby obtaining the optimal depression tendency detection model for individual depression tendency detection. Compared with the traditional depression detection technology based on questionnaire scales and doctor interviews, this system can assist in diagnostic decision-making more scientifically, objectively and effectively, and compared with the non-contact facial depression state detection method, the system only needs to collect the gait video data of the subject in the natural state, without emotional induction or adding other interference to the subject, and can achieve a more accurate assessment of its behavioral data and depression tendency, and its detection method is simple and low-cost, suitable for early screening of the mental health status of a large population, early warning of individuals, and providing auxiliary decision-making for psychiatric specialists.

[0149] 2. The present invention takes into account the dynamic variability of an individual's depressive state. During data collection, video data of the subjects walking naturally in a designated area is collected at a fixed time each week to form time series data. The PHQ-9 scale score filled in by the subjects is used as their depressive tendency label. Then, an LSTM network model is constructed to analyze the correlation between gait characteristics and depressive tendencies, and the time series information of the features is mined to improve the accuracy of depression detection.

[0150] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0151] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A depression tendency detection system based on natural gait time series analysis, characterized in that: The system comprises: The data collection module is used to continuously collect the gait video data of the subjects in different time periods and obtain the depression tendency labels of the subjects; A data processing module, used for processing the gait video data to obtain the gait data of the subject in a complete gait cycle; A feature extraction and sorting module, used for extracting the gait features of the subject based on the gait data, and sorting the gait features by calculating the importance and relevance of the gait features; A model building module, used to build a depression tendency detection model, wherein the depression tendency detection model includes an input layer, a hidden layer, a Softmax layer, and an output layer; A model training and optimization module, which trains the depression tendency detection model based on the sorted gait features and the depression tendency labels to obtain an optimal depression tendency detection model; A depression tendency detection result output module is used for the subject to perform depression tendency detection based on the optimal depression tendency detection model and output the depression tendency detection result; When the model training and optimization module trains the depression tendency detection model based on the sorted gait features and the depression tendency labels to obtain the optimal depression tendency detection model, the model training and optimization module includes: S51, based on the sorted gait features, successively select the top m i+1 =m i +1 gait feature combined with the depressive tendency label to train the depressive tendency detection model until all the gait features are selected and the model training is completed; wherein, m i Indicates that the number of gait features currently selected is m i pcs, m i The initial value of is 0; S52, compare the AUC values ​​of the corresponding gait feature quantity obtained by running the model after each model training in S51, and select the quantity m corresponding to the maximum AUC value i As the optimal number of features m good ; Based on the comprehensive ranking of gait characteristics, select the top m good gait features as the optimal feature subset; S53, based on the optimal number of features m good and the optimal number of features m good The corresponding optimal feature subset and the depression tendency detection model after the depression tendency label training are the optimal depression tendency detection model.

2. The system according to claim 1, characterized in that The hidden layer includes a stacked 2-layer LSTM network structure.

3. The system according to claim 1, characterized in that The data acquisition module includes a gait data acquisition unit and a psychological state acquisition unit.

4. The system according to claim 3, characterized in that The gait data collection unit includes a video camera, which is used to collect gait data of the subject walking naturally in a designated area at different time periods; The psychological state collection unit includes an input unit, and the PHQ-9 scale filled in by the subject is input by the input unit.

5. The system according to claim 1, wherein: When the data processing module processes the gait video data to obtain the gait data of the subject in a complete gait cycle, it includes: S31, detecting and tracking the coordinates of the two-dimensional skeleton joint points of the subject in the gait video frame, and acquiring the three-dimensional coordinates of the body joint points of the subject in the gait video through binocular camera depth estimation based on the two-dimensional skeleton joint point coordinates; S32, performing coordinate system conversion, standardization, and filtering and denoising processing on the three-dimensional coordinates in the gait video; S33, intercepting the gait video processed in S32 to obtain gait data of the subject in a complete gait cycle; the gait data includes joint coordinate data.

6. The system according to claim 1, characterized in that The gait features include: spatiotemporal features, motion features, time domain features and frequency domain features.

7. The system according to claim 6, characterized in that The feature extraction and sorting module extracts the gait features of the subject based on the gait data, and sorts the gait features by calculating the importance and relevance of the gait features, including: S41, calculating the importance of the gait features using a random forest algorithm, and calculating the correlation of the gait features using a Pearson correlation analysis; S42: sum up the absolute values ​​of the importance and correlation of the gait features and sort them from high to low.

8. The system of claim 1, wherein: The depressive tendency detection result includes the probability of the depressive tendency belonging to different depression severity level categories.

Images