Depression gait recognition method and system based on multi-scale adaptive graph convolution

Abstract

The application relates to the technical field of artificial intelligence, and particularly discloses a depression gait recognition method and system based on multi-scale adaptive graph convolution. A gait skeleton sequence containing skeleton data at each moment is acquired; each skeleton data is modeled as a graph structure, a joint topology partition strategy is formulated, and initial spatial features in the skeleton data are extracted through graph convolution. A sample-level graph structure is dynamically generated, a predefined physical connected graph and a task-level graph structure are fused to form a new adjacency matrix, adaptive spatial features are generated through graph convolution. A multi-scale time convolution layer is inputted, M convolution branches with different sampling intervals d are executed in parallel, and time sequence features are formed through splicing. Adaptive spatial features and time sequence features are fused through a double-flow adaptive fusion module, a classification layer is inputted, and a depression state recognition result is outputted. The application aims to realize deeper feature extraction, break through the limitation of space-time dynamic modeling, improve the accuracy of depression state recognition, and enhance the generalization ability of the model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a depression gait recognition method and system based on multi-scale adaptive graph convolution. BACKGROUND

[0002] At present, in the field of human behavior recognition, gait-based recognition technology is widely concerned due to its non-invasive and natural collection characteristics.

[0003] In the prior art, the gait recognition method usually extracts gait sequences from natural walking videos and combines time domain and frequency domain features to construct a model. However, this method mainly relies on hand-designed feature extraction methods, which is difficult to fully exploit the complex spatial topological correlation and fine temporal dynamic evolution rules contained in gait data. In addition, hand-crafted features have limited ability to express high-order spatiotemporal dependencies implied in behavior patterns, resulting in insufficient ability to analyze fine-grained behavior semantics. These technical bottlenecks not only reduce the accuracy of behavior recognition, but also weaken the generalization ability of the model, especially in application scenarios that require high-precision recognition of specific psychological states, this limitation is particularly prominent.

[0004] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present application, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] In view of the problem of relying on hand-designed shallow features and insufficient spatiotemporal dynamic modeling in the prior art, the present application provides a depression gait recognition method and system based on multi-scale adaptive graph convolution, aiming to realize deeper feature extraction and break through the limitations of spatiotemporal dynamic modeling, thereby improving the accuracy of depression state recognition and enhancing the generalization ability of the model.

[0006] To achieve the above purpose, the present application provides the following technical solutions:

[0007] The depression gait recognition method based on multi-scale adaptive graph convolution comprises:

[0008] obtaining a gait skeleton sequence; wherein each time's skeleton data contains a joint node and a C-dimensional feature vector of each joint node;

[0009] modeling each skeleton data as a graph structure , and formulating a joint topological partition strategy to extract spatial features in the skeleton data through graph convolution to generate initial spatial features ; wherein, is a set of joint nodes, is the joint node; an edge set defined based on human skeleton physical connectivity;

[0010] based on initial spatial features dynamically generating a sample-level graph structure, fusing a predefined physical connectivity graph, a task-level graph structure and the sample-level graph structure to form a new adjacency matrix, and generating adaptive spatial features through a graph convolution operation ; the predefined physical connectivity graph is a topology inherent to the skeleton; the task-level graph structure is a trainable parameter matrix, which is updated through back propagation;

[0011] inputting the adaptive spatial features into a multi-scale temporal convolution layer, and performing in parallel convolution branches with different sampling intervals d, and splicing the outputs of the convolution branches to form time sequence features ;

[0012] fusing the adaptive spatial features and the time sequence features through a dual-flow adaptive fusion module to generate fusion features ;

[0013] inputting the fusion features into a classification layer for processing, and outputting a depression state recognition result.

[0014] As a preferred scheme of the present application, the joint topology partition strategy comprises:

[0015] dividing a neighborhood joint set of an i-th joint node into a root joint subset , a trunk subset and a limb subset , ;

[0016] a joint category label function is defined as:

[0017] ;

[0018] wherein, is a neighborhood joint node of the joint node ; =1,2,..., ; =1,2,..., .

[0019] As a preferred scheme of the present application, the spatial features in the skeleton data are extracted through a graph convolution to generate initial spatial features , which comprises:

[0020] The initial neighborhood joint set of the first joint node The initial spatial feature of the first joint node is generated by weighted average of the features of all joint nodes in the initial neighborhood joint set , and the formula is:

[0021] ;

[0022] In the formula, is the input feature of the joint node in the initial neighborhood joint set is the class label function of the node relative to is the first normalization factor; is the weight function; =1,2,..., ; =1,2,..., ;

[0023] The initial spatial feature is generated, and the formula is:

[0024] .

[0025] As a preferred scheme of the present application, the generating adaptive spatial feature includes:

[0026] The initial spatial feature of the first joint node is input into two 1x1 convolutional layers to generate first embedding feature and second embedding feature ; Based on the first embedding feature and the second embedding feature

[0027] , the sample-level graph structure is generated, and the formula is: ;

[0028] ;

[0029] The predefined physical connectivity graph and the task-level graph structure are normalized to generate the normalized predefined physical connectivity graph and the normalized task-level graph structure ; ​​​​​​​

[0030] Fusion normalization of predefined physical connectivity graphs Normalized task-level graph structure and sample-level graph structure This forms a new adjacency matrix. The formula is:

[0031] ;

[0032] Based on the new adjacency matrix Extended Joint nodes The neighborhood range is given by the formula:

[0033] ;

[0034] In the formula, For the first Joint nodes The extended neighborhood; For the new adjacency matrix The mth row, the mth row Column elements; The first preset threshold;

[0035] Based on extended neighborhood , for the Joint nodes Perform graph convolution operation to generate the first... Joint nodes Adaptive spatial features The formula is:

[0036] ;

[0037] In the formula, To expand the neighborhood Joint nodes; To expand the neighborhood joint nodes Input features; For nodes Compared to Category label function; It is the second normalization factor; =1,2,..., ;

[0038] Generate adaptive spatial features The formula is:

[0039] .

[0040] As a preferred embodiment of the present invention, the task-level graph structure is a trainable parameter matrix, which is updated by back propagation, and the formula is:

[0041] ;

[0042] wherein, is a depression classification loss function, and η is a learning rate.

[0043] As a preferred scheme of the present application, the adaptive spatial feature and the time sequence feature are fused by the double-flow adaptive fusion module to generate a fusion feature , and the fusion feature includes:

[0044] The dimensions of the adaptive spatial feature and the time sequence feature are aligned by a channel adaptation convolution to generate spatial features and time sequence features with a standard number of channels;

[0045] The spatial feature is taken as a modulation stream to calculate the attention weight of the spatial feature relative to the time sequence feature , and the formula is:

[0046] ;

[0047] wherein, is a normalization function; is a linear transformation layer; is an activation function;

[0048] The fusion feature is generated, and the formula is:

[0049] ;

[0050] wherein, is an element-wise multiplication; is a channel adaptation convolution.

[0051] As a preferred scheme of the present application, the fusion feature is input into a classification layer for processing to output a depression state recognition result, and specifically includes:

[0052] The fusion feature F is input into the classification layer, and after mapping processing by a full connection layer, a depression probability distribution is output by using a Softmax function, wherein, , is a depression probability. for non-depression probability;

[0053] when the depression probability exceeds a second preset threshold, the depression state is determined.

[0054] The recognition system of the depression gait recognition method based on multi-scale adaptive graph convolution comprises:

[0055] A data collection module is configured to acquire a gait skeleton sequence; wherein, the skeleton data at each time point comprises N joint nodes and a C-dimensional feature vector of each joint node.

[0056] A spatial feature extraction module is configured to model each skeleton data as a graph structure , and formulate a joint topology partition strategy to extract spatial features in the skeleton data through graph convolution to generate initial spatial features ; wherein, is a set of joint nodes, is the th joint node; is an edge set defined based on the physical connectivity of the human skeleton;

[0057] A spatial feature updating module is configured to dynamically generate a sample-level graph structure based on the initial spatial features , fuse a predefined physical connectivity graph, a task-level graph structure and the sample-level graph structure to form a new adjacency matrix, and generate adaptive spatial features through graph convolution operation ; the predefined physical connectivity graph is a topology structure inherent to the skeleton; the task-level graph structure is a trainable parameter matrix which is updated through back propagation;

[0058] A temporal feature extraction module is configured to input the adaptive spatial features into a multi-scale temporal convolution layer, and perform parallel execution on a plurality of convolution branches with different sampling intervals d, and splice the outputs of the convolution branches to form temporal features .

[0059] A feature fusion module is configured to fuse the adaptive spatial features and the temporal features through a dual-flow adaptive fusion module to generate fusion features .

[0060] A state determination module is configured to input the fusion features into a classification layer for processing, and output a depression state recognition result.

[0061] A computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements steps of a depression gait recognition method based on multi-scale adaptive graph convolution when executing the computer program.

[0062] A computer readable storage medium stores a computer program, and the computer program implements steps of a depression gait recognition method based on multi-scale adaptive graph convolution when executed by a processor.

[0063] Compared with the prior art, the beneficial effects of the present application are: by introducing task-level and sample-level adaptive graph structures, dynamic modeling of high-order spatial-temporal dependencies between skeleton joint nodes is realized; by fusing a predefined physical connectivity graph, a task-level graph structure and a sample-level graph structure, a new adjacency matrix is formed, and the skeleton connection mode is automatically adjusted, thereby effectively extracting deep spatial features; multi-branch time convolution with different sampling intervals is adopted, and dynamic features in different time windows are processed in parallel, thereby significantly enhancing the modeling capability of the model for time sequence features such as action rhythm changes and behavior periodicity, and accurately capturing gait dynamic changes related to depression state; with the help of a dual-flow adaptive fusion module, spatial features and time sequence features are optimized collaboratively, single-modal noise is effectively suppressed, depression-specific behavior patterns are strengthened, more discriminative fusion features are generated, and the accuracy of depression state recognition is greatly improved; by using graph convolution combined with multi-scale time convolution, discriminative features are automatically learned from the skeleton sequence in an end-to-end architecture, and the adaptability and generalization capability of the model are effectively improved.

[0064] The foregoing general description and the following description are merely exemplary and explanatory, and are not intended to limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0065] One or more embodiments are exemplarily illustrated by corresponding drawings, which do not constitute limitations on the embodiments, elements with the same reference numerals in the drawings are shown as similar elements, the drawings do not constitute proportional limits, and wherein:

[0066] Figure 1 is a flowchart of a depression gait recognition method based on multi-scale adaptive graph convolution provided by an embodiment of the present application;

[0067] Figure 2 is a structural schematic diagram of a joint topology partitioning strategy provided by an embodiment of the present application;

[0068] Figure 3 is a flowchart of an adaptive spatial feature generation method provided by an embodiment of the present application;

[0069] Figure 4 is a flowchart of a feature fusion method provided by an embodiment of the present application;

[0070] Figure 5 is a schematic diagram of a multi-scale time convolution layer provided by an embodiment of the present application;

[0071] Figure 6 is a modular structure schematic diagram of a recognition system of a depression gait recognition method based on multi-scale adaptive graph convolution provided by an embodiment of the present application;

[0072] Figure 7 is a skeleton connection topological graph of a control group provided by an embodiment of the present application;

[0073] Figure 8 is a skeleton connection topological graph of a research group provided by an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to enable a person skilled in the art to better understand the features and technical contents of the embodiments of the present disclosure, the implementation of the embodiments of the present disclosure will be described in detail below with reference to the drawings, which are only used for reference and do not limit the embodiments of the present disclosure. In the following technical description, in order to facilitate explanation, a plurality of details are provided to provide a full understanding of the disclosed embodiments. However, one or more embodiments can still be implemented without these details. In other cases, well-known structures and devices can be simplified to illustrate.

[0075] The terms "first", "second", and the like in the specification and claims of the embodiments of the present disclosure and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion.

[0076] Unless otherwise specified, the term "a plurality of" means two or more.

[0077] In the embodiments of the present disclosure, the character " / " represents an "or" relationship between the objects before and after it. For example, A / B represents: A or B.

[0078] The term "and / or" is a description of the association between objects, which means that there can be three relationships. For example, A and / or B, which means: A or B, or, A and B, three relationships.

[0079] The term "corresponding" can refer to an association or binding relationship, A corresponding to B means that there is an association or binding relationship between A and B.

[0080] The gait recognition method in the prior art extracts a gait sequence from a natural walking video, and combines time domain and frequency domain features to construct a behavior recognition model. However, the method relies on a manually designed feature extraction method, and it is difficult to fully mine the complex spatial topological correlation and fine time dynamic evolution law contained in the gait sequence, which limits the ability to capture deep behavior semantics, and further restricts the further improvement of the model performance.

[0081] The application provides a depression gait recognition method based on multi-scale adaptive graph convolution. The method realizes dynamic modeling of high-order space-time dependence between skeletal joints by introducing task-level and sample-level adaptive graph structures. By fusing a pre-defined physical connected graph, a task-level graph structure and a sample-level graph structure, a new adjacency matrix is constructed to automatically adjust the skeleton connection mode, thereby effectively extracting deep spatial features such as depression-related behavior coupling features across joints and body parts.

[0082] In addition, the method uses multi-branch time convolution with different sampling intervals to process dynamic features in different time windows in parallel, enhances the modeling ability of the model for time sequence features such as action rhythm changes and behavior periodicity, and effectively captures the dynamic changes of gait related to depression state. Through the cooperative optimization of spatial features and time sequence features by the dual-flow adaptive fusion module, the single-modal noise is effectively suppressed, the depression-specific behavior pattern is strengthened, and more discriminative fusion features are generated, thereby significantly improving the accuracy of depression state recognition.

[0083] By using graph convolution combined with multi-scale time convolution, the method automatically learns discriminative features from the skeleton sequence in an end-to-end structure, effectively improving the adaptability and generalization ability of the model.

[0084] As shown in Figure 1 , it is an embodiment of the application, which provides a depression gait recognition method based on multi-scale adaptive graph convolution, comprising:

[0085] S1, acquiring a gait skeleton sequence; wherein each time point of the skeleton data contains joint nodes and a C-dimensional feature vector of each joint node;

[0086] Further, the gait skeleton data is collected by a Kinect V2 depth camera, and the collected data is subjected to coordinate rotation and Gaussian filtering and other preprocessing steps to improve the signal-to-noise ratio.

[0087] S2, modeling each skeleton data as a graph structure , and formulating a joint topological partition strategy to extract spatial features in the skeleton data through graph convolution to generate initial spatial features ; wherein, is a set of joint nodes, For the first Each joint node; It is an edge set defined based on the physical connectivity of the human skeleton;

[0088] Developing a joint topology partitioning strategy includes:

[0089] The i-th joint node Neighborhood joint set Divided into root joint subset Trunk subset and limb subset , ;

[0090] Joint category labeling function Defined as:

[0091] ;

[0092] In the formula, joint nodes , neighboring joint nodes; =1,2,..., ; =1,2,..., .

[0093] Furthermore, the joint topology partitioning strategy aims to assist the model in more accurately understanding the distance relationships between skeletal structures by assigning prior semantic annotation information to adjacent joint nodes. Based on previous in-depth analysis of depressive gait characteristics, the study found that limb joints such as the hands and feet can provide additional kinematic information compared to trunk joints. Based on this finding, this invention proposes a joint topology partitioning strategy. Figure 2 As shown, taking the red box as an example, the yellow nodes belong to the root joint subset. The blue nodes belong to the trunk subset. The orange nodes belong to the limbs subset. The defined body subset based on the node labels in Kinet. Includes {0, 1, 2, 3, 4, 8, 12, 16, 20}.

[0094] In one embodiment, step S2 specifically includes:

[0095] S21, Model each skeleton data as a graph structure And formulate a joint topology partitioning strategy;

[0096] S22, the first Joint nodes Initial neighborhood joint set The features of all joint nodes within the node are weighted and averaged to generate the first... Initial spatial features of each joint node The formula is:

[0097] ;

[0098] In the formula, For the initial neighborhood joint set Internal joint nodes Input features; For nodes Compared to Category label function; It is the first normalization factor; For weighting functions; =1,2,..., ; =1,2,..., ;

[0099] Furthermore, As a configurable normalization factor, it aims to eliminate feature magnitude bias caused by changes in neighborhood size. Its configuration methods include, but are not limited to, the following two schemes:

[0100] Option 1: [The following is a possible solution] Set as static data. That is, it is equal to a constant, and is suitable for scenarios where the node degree distribution is uniform;

[0101] The second option: Dynamic settings This makes the output features equivalent to the mean of the neighborhood features, thereby improving topological generalization.

[0102] S23, Generate initial spatial features The formula is:

[0103] .

[0104] Furthermore, the features of any joint can be obtained by weighted averaging of the features of all nodes in its neighborhood. Specifically, graph convolution updates the features of the target node by weighted summing of the features of its neighboring nodes, where the weights are determined by the class labels of the neighboring nodes. Through this operation, the network can effectively capture the spatial relationships between joints.

[0105] S3, Initial Spatial Features Dynamically generate sample-level graph structures, fuse predefined physically connected graphs, task-level graph structures, and the sample-level graph structures to form a new adjacency matrix, and generate adaptive spatial features through graph convolution operations. The predefined physical connectivity graph is the inherent topology of the skeleton; the task-level graph structure is a trainable parameter matrix, updated through backpropagation.

[0106] Task-level graph structure The trainable parameter matrix is ​​updated via backpropagation, using the following formula:

[0107] ;

[0108] In the formula, Let η be the loss function for depression classification, and η be the learning rate.

[0109] like Figure 3 As shown, in one embodiment, step S3 specifically includes:

[0110] S31, the first Joint nodes initial spatial features Input two 1×1 convolutional layers to generate the first embedded features. Second Embedded Features ;

[0111] S32, based on the first embedding feature Second Embedded Features Generate sample-level graph structure The formula is:

[0112] ;

[0113] Furthermore, this step aims to learn the unique relationships between the joint nodes of the current sample. For example, in a depressive gait, the asymmetry of arm swing is a typical manifestation of such relationships.

[0114] S33, for predefined physical connectivity graphs and task-level graph structure Perform normalization processing to generate a normalized predefined physical connectivity graph. Normalized task-level graph structure ;

[0115] Furthermore, a commonly used normalization method is row normalization. Normalization aims to eliminate deviations in joint node degrees and effectively prevent gradient explosion.

[0116] S34, Merging and normalizing predefined physical connectivity graphs Normalized task-level graph structure and sample-level graph structure This forms a new adjacency matrix. The formula is:

[0117] ;

[0118] Furthermore, Essential biomechanical constraints used to maintain gait analysis, such as the natural connection between the knee and ankle joints, Used to optimize global depressive features, such as shortened stride. Used to detect individual abnormalities, such as reduced swing of one arm.

[0119] S35, based on the new adjacency matrix Extended Joint nodes The neighborhood range is given by the formula:

[0120] ;

[0121] In the formula, For the first Joint nodes The extended neighborhood; For the new adjacency matrix The mth row, the mth row Column elements; The first preset threshold;

[0122] Furthermore, in practical applications, The value of needs to be adjusted according to the specific situation. Its range is [0,1], and the typical value is 0.5. The screening threshold used to control the connection strength of joint nodes is adjusted. It can flexibly adapt to gait patterns of depression of varying severity; Include The newly added joint nodes are filtered, such as the wrist joint abnormally associated with the knee joint; when At that time, the joint nodes join in The neighborhood of the gait is used to capture abnormal joint linkages in depressive gait.

[0123] S36, based on extended neighborhood , for the Joint nodes Perform graph convolution operation to generate the first... Joint nodes Adaptive spatial features The formula is:

[0124] ;

[0125] In the formula, To expand the neighborhood Joint nodes; To expand the neighborhoodJoint node of Input feature of Node of Category label function of Second normalization factor of Category label function of Second normalization factor of =1,2,..., ;

[0126] Further, As a configurable normalization factor, it aims to eliminate the feature magnitude deviation caused by the change of neighborhood size. Its configuration methods include but are not limited to the following two schemes:

[0127] The first scheme: Set it to a static value, That is, equal to a constant, suitable for scenarios where node degree distribution is uniform.

[0128] The second scheme: dynamic Adjustment to ensure that the output feature is equal to the mean value of the neighborhood feature, thereby improving the topological generalization.

[0129] S37, generate adaptive spatial features The formula is:

[0130] ;

[0131] S4, input adaptive spatial features into a multi-scale time convolution layer, which performs convolution branches with different sampling intervals d in parallel, and splices the outputs of each convolution branch to form time sequence features ;

[0132] Further, as shown in Figure 4 In one embodiment, the multi-scale time convolution layer designs five convolution branches with different sampling intervals d, which are: basic convolution branch, dilated convolution branch, maximum pooling branch, dynamic convolution kernel branch and residual connection branch. The sampling interval d of each branch is set to 1, 2, 3 and 4 respectively.

[0133] In the case of the same sampling interval d, it will be verified through the experiment of comparing the dynamic convolution kernel size With the fixed convolution kernel size Which convolution kernel size is more suitable for this application scenario. Assuming that the fixed convolution kernel size of the five branches is set to 3, and the dynamic convolution kernel size is set to 1, 3, 5 and 9 respectively.

[0134] As shown in Table 1, the comparative experiment results show that the parameter amount of the fixed convolution kernel is significantly lower than that of the dynamic convolution kernel, and is reduced by about 18.4%; the specificity of the fixed convolution kernel is 3.4 percentage points higher than that of the dynamic convolution kernel; the accuracy of the two is exactly the same, and is 89.62%. Although the sensitivity of the dynamic convolution kernel is higher, it is at the expense of specificity, and requires an additional 28% of the parameter amount to support. In the F1 score, the dynamic convolution kernel only leads by 0.0038, and the advantage is negligible. Therefore, the fixed convolution kernel realizes the accuracy comparable to the dynamic convolution kernel with lower calculation cost and higher specificity. Although the dynamic convolution kernel slightly occupies the advantage in sensitivity, the cost of the explosive growth of the parameter amount and the decline of the specificity makes the fixed convolution kernel a more efficient and robust choice in most actual scenarios.

[0135] Table 1 Performance comparison table of dynamic convolution kernel and fixed convolution kernel

[0136]

[0137] In summary, for the application scenario, the convolution kernel size is preferably a fixed value, specifically, the recommended convolution kernel size is 3.

[0138] S5, fusing the adaptive spatial feature and the temporal feature through a double-flow adaptive fusion module to generate a fused feature .

[0139] As Figure 5 shown, in one embodiment, step S5 specifically includes:

[0140] S51, aligning the dimensions of the adaptive spatial feature and the temporal feature through a channel adaptation convolution to generate spatial features and temporal features of a standard channel number;

[0141] Further, the channel adaptation convolution unifies the adaptive spatial feature and the temporal feature to a preset standard channel number through a 1x1 convolution, eliminating the fusion deviation caused by the dimension difference.

[0142] S52, taking the spatial feature as a modulation flow, calculating the attention weight of the spatial feature relative to the temporal feature , the formula is:

[0143] ;

[0144] In the formula, This is the normalization function; It is a linear transformation layer; For activation functions;

[0145] Furthermore, this formula aims to extract spatial features Middle Learning Attention Weight Distribution , generated The weights will be used to weight and fuse temporal features, thereby achieving adaptive modulation of temporal features by spatial information. Specifically, First, spatial characteristics Perform the first linear projection to extract... Abstract representation of, This not only enhanced the model's nonlinear expressive power but also effectively solved the gradient vanishing problem. Subsequently, The generated weights will be combined with time-series features Matching The linearized output is then normalized to ensure that the output value range is within [0,1], thus guaranteeing the reasonableness of the weighting coefficients. Attention weights. The value range of is [0,1]. When The time signature is a characteristic associated with high confidence depression.

[0146] S53, Generate fusion features The formula is:

[0147] ;

[0148] In the formula, This is element-wise multiplication; Channel-adaptive convolution.

[0149] S6 inputs the fused feature F into the classification layer for processing and outputs the result of depression state recognition.

[0150] The fused feature F is input into the classification layer for processing, and the output of the depression state recognition result includes:

[0151] The fused feature F is input into the classification layer, and after mapping by a fully connected layer, the depression probability distribution is output using the Softmax function. ,in, , The probability of depression; The probability of not being depressed;

[0152] When the probability of depression When the threshold is exceeded, the patient is identified as being in a state of depression.

[0153] Further, the second preset threshold needs to be determined according to the actual application scene and model performance, and its conventional value range is between 0.5 and 0.7.

[0154] As shown in Figure 6 , another embodiment of the present application provides a recognition system of the depression gait recognition method based on multi-scale adaptive graph convolution, which comprises:

[0155] A data collection module is configured to acquire a gait skeleton sequence; wherein the skeleton data at each time point comprises N joint nodes and a C-dimensional feature vector of each joint node;

[0156] A spatial feature extraction module is configured to model each skeleton data as a graph structure , and formulate a joint topology partition strategy to extract spatial features in the skeleton data through graph convolution to generate initial spatial features ; wherein, is a set of joint nodes, is the i-th joint node; is an edge set defined based on the physical connectivity of the human skeleton; A spatial feature updating module is configured to dynamically generate a sample-level graph structure based on the initial spatial features

[0157] , fuse a predefined physical connectivity graph, a task-level graph structure and the sample-level graph structure to form a new adjacency matrix, and generate adaptive spatial features through graph convolution operation ; the predefined physical connectivity graph is a topology structure inherent to the skeleton; the task-level graph structure is a trainable parameter matrix, which is updated through back propagation;

[0158] A temporal feature extraction module is configured to input the adaptive spatial features into a multi-scale temporal convolution layer, and perform parallel execution on a plurality of convolution branches with different sampling intervals d, and splice the outputs of the convolution branches to form temporal features ;

[0159] A feature fusion module is configured to fuse the adaptive spatial features and the temporal features through a dual-flow adaptive fusion module to generate fusion features ;

[0160] A state determination module is configured to input the fusion features F into a classification layer for processing, and output a depression state recognition result.

[0161] ​Further, in a certain neurological department of a third-class hospital, researchers used 6 Kinect V2 depth cameras to build a gait collection system, and successfully collected gait data of 194 subjects, including 91 in the study group and 103 in the control group. The effectiveness of the key modules was verified through ablation experiments, and the specific results are shown in Table 2.

[0162] Table 2 Ablation experiment results

[0163]

[0164] The experimental data show that after introducing the spatial feature update module, the accuracy is improved by 2.08%; on this basis, further adding the time sequence feature extraction module, the accuracy is additionally improved by 1.03%.

[0165] In-depth analysis, the application adopts visual means to study the skeleton connection topology graph of different groups. As shown in Figure 7 , the skeleton connection topology graph of the control group is shown. Among them, the red connection represents the inherent skeleton connection, and the blue connection represents the connection formed through learning. From the figure, it can be seen that the skeleton connection distribution of the control group presents symmetry as a whole, mainly concentrated in the hand and foot parts, which is consistent with the coordinated action of hands and feet during normal walking.

[0166] As shown in Figure 8 , the skeleton connection topology graph of the study group is shown. Similarly, the red connection represents the inherent skeleton connection, and the blue connection represents the connection formed through learning. From the figure, it can be obviously observed that the skeleton connection distribution of the study group presents asymmetry, and there are connections between the head, shoulder and shoulder-neck, which is highly related to the gait posture of head forward bending, shoulder inward buckling and trunk depression.

[0167] Through large-scale experiment verification, the model achieves an accuracy of 89.27% in gait-state association recognition. At the same time, visual analysis reveals the strong correlation between the state and the gait topology. Therefore, the technology has significant landing potential in application scenarios such as mental health assessment and rehabilitation monitoring.

[0168] The embodiment also provides a computer device suitable for the depression gait recognition method based on multi-scale adaptive graph convolution, which comprises a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize all or part of the steps of the method described in the embodiment of the application.

[0169] The embodiment further provides a storage medium having a computer program stored thereon, and the computer program is executed by a processor to perform the method in any optional implementation manner of the above-mentioned embodiment. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk.

[0170] The storage medium provided by the embodiment belongs to the same inventive concept as the data storage method provided by the above-mentioned embodiment, and the technical details not described in the embodiment can be referred to the above-mentioned embodiment, and the embodiment has the same beneficial effects as the above-mentioned embodiment.

[0171] In summary, by introducing task-level and sample-level adaptive graph structures, dynamic modeling of high-order spatio-temporal dependencies between skeletal joints is achieved. By fusing the pre-defined physical connectivity graph, task-level graph structure and sample-level graph structure, a new adjacency matrix is formed, which automatically adjusts the skeleton connection mode, thereby effectively extracting deep spatial features such as cross-joint and cross-body part depression-related behavior coupling features. Multi-branch temporal convolution with different sampling intervals or convolution kernel sizes is used to process dynamic features in different time windows in parallel, enhancing the model's ability to model temporal features such as action rhythm changes and behavior periodicity, effectively capturing state-related gait dynamics. Through the dual-flow adaptive fusion module, spatial features and temporal features are optimized collaboratively, effectively suppressing single-modal noise, strengthening state-specific behavior patterns, and generating more discriminative fusion features, thereby improving the accuracy of state recognition. By using graph convolution combined with multi-scale temporal convolution, discriminative features are automatically learned from the skeleton sequence in an end-to-end structure, effectively improving the adaptability and generalization ability of the model.

[0172] It will be apparent to those skilled in the art that the application is not limited to the details of the above-exemplified embodiments and that the present application can be implemented in other particular forms without departing from the spirit or essential characteristics of the present application. The embodiments should therefore be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the above description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

Claims

1. A depression gait recognition method based on multi-scale adaptive graph convolution, characterized in that, Comprise: Obtain a gait skeleton sequence; wherein the skeleton data of each time point comprises a joint node and a C-dimensional feature vector of each joint node; Model each skeleton data as a graph structure. Furthermore, a joint topology partitioning strategy was formulated, and spatial features were extracted from the skeleton data through graph convolution to generate initial spatial features. ;in, For a set of joint nodes, For the first Each joint node; It is an edge set defined based on the physical connectivity of the human skeleton; The spatial features in the skeleton data are extracted by graph convolution to generate initial spatial features comprising: The initial neighborhood joint set of the first joint node The initial spatial feature of the first joint node is generated by weighted average of the features of all joint nodes in the initial neighborhood joint set of the first joint node The initial neighborhood joint set of the first joint node The initial spatial feature of the first joint node is generated by weighted average of the features of all joint nodes in the initial neighborhood joint set of the first joint node The initial spatial feature of the first joint node is generated by weighted average of the features of all joint nodes in the initial neighborhood joint set of the first joint node The initial spatial feature ； wherein is the initial neighborhood joint set is the inner joint node is the input feature; is the node is the class label function with respect to is the first normalization factor; is the weight function; = 1, 2,..., ; ; = 1, 2,..., ; Generating initial spatial features with the formula: ； Based on initial spatial features The dynamic generation of sample-level graph structure, the fusion of pre-defined physical connectivity graph, task-level graph structure and the sample-level graph structure form a new adjacency matrix, and the adaptive spatial features are generated through graph convolution operation The pre-defined physical connectivity graph is the topological structure inherent to the skeleton; the task-level graph structure is a trainable parameter matrix, which is updated through back propagation Adaptive spatial features Input multi-scale temporal convolution layer, M convolution branches with different sampling intervals d are executed in parallel, and the outputs of each convolution branch are spliced to form time sequence features ; The adaptive spatial features and the timing features are fused by a double-flow adaptive fusion module to generate fused features. ;​​ The fused features are input into a classification layer for processing, and a depression state recognition result is output.

2. The multi-scale adaptive graph convolution based depression gait recognition method according to claim 1, characterized in that, The joint topology partition strategy comprises: The i-th joint node Neighborhood joint set Divided into root joint subset Trunk subset and limb subset , ; Joint category tagging function is defined as: ； wherein is a joint node of the neighborhood joint nodes of the joint node = 1, 2,..., ; = 1, 2,..., .

3. The multi-scale adaptive graph convolution based depression gait recognition method according to claim 2, characterized in that, The generating adaptive spatial features comprising: The initial spatial features of the first joint node The initial spatial features of the second joint node The initial spatial features of the third joint node Input two 1x1 convolutional layers to generate a first embedding feature and a second embedding feature ; based on the first embedding feature and the second embedding feature , to generate a sample-level graph structure , which is formulated as: ； predefined physical connectivity graph and task level graph structure normalization to generate a normalized predefined physical connectivity graph and normalized task level graph structure ; Fused normalized pre-defined physical connectivity graph , normalized task-level graph structure and sample-level graph structure , forming a new adjacency matrix , is ； Based on new adjacency matrix Expand the first Joint node The neighborhood range of the formula is: ； In the formula, For the first Joint nodes The extended neighborhood; For the new adjacency matrix The mth row, the mth row Column elements; The first preset threshold; Based on extended neighborhood , for the Joint nodes Perform graph convolution operation to generate the first... Joint nodes Adaptive spatial features The formula is: ； wherein is a joint node of the extended neighborhood ; is a joint node of the extended neighborhood ; and is an input feature of the joint node ; is a class label function of the node relative to is a second normalization factor; = 1, 2,..., ; Generating adaptive spatial features , the formula is: 。 4. The depression gait recognition method based on multi-scale adaptive graph convolution of claim 3, characterized in that, The task-level graph structure is a trainable parameter matrix, updated by backpropagation, with the formula: ； In the formula, is the depression classification loss function, and η is the learning rate.

5. The multi-scale adaptive graph convolution-based depression gait recognition method according to claim 4, characterized in that, The adaptive spatial features are then fused using a dual-stream adaptive fusion module. and time series characteristics The fusion process generates fused features F, including: Adaptive spatial features aligned by channel-wise adaptive convolution and temporal features of a dimensionality generating standard channel number of spatial features and temporal features ; computing the spatial features as a modulating stream, the spatial features relative to the temporal features attention weights , as: ； wherein is a normalization function; is a linear transformation layer; is an activation function; Generating fusion features , the formula is: ； wherein is an element-wise multiplication; is a channel-adaptive convolution.

6. The multi-scale adaptive graph convolution-based depression gait recognition method according to claim 5, characterized in that, The fusion feature F is input into the classification layer for processing, and a depression state recognition result is output, specifically comprising: The fusion feature F is input to a classification layer, and after full connection layer mapping processing, a Softmax function is used to output a depression probability distribution wherein, , is a depression probability; is a non-depression probability; When the probability of depression When the probability of depression exceeds a second predetermined threshold, the depression state is determined.

7. The multi-scale adaptive graph convolution-based depression gait recognition method recognition system according to any one of claims 1 to 6, characterized in that, The system comprises: A data collection module is configured to acquire a gait skeleton sequence; wherein, skeleton data at each time point comprises N joint nodes and a C-dimensional feature vector of each joint node; The spatial feature extraction module is configured to model each skeleton data as a graph structure , and a joint topology partitioning strategy is formulated to extract spatial features in the skeleton data through graph convolution to generate initial spatial features ; wherein, is a set of joint nodes, is a th joint node; is an edge set defined based on physical connectivity of the human skeleton a spatial feature updating module, configured to update the initial spatial feature based on the initial spatial feature dynamically generate a sample-level graph structure, fuse a predefined physical connectivity graph, a task-level graph structure and the sample-level graph structure to form a new adjacency matrix, and generate adaptive spatial features through a graph convolution operation ; the predefined physical connectivity graph is a topology inherent to the skeleton; and the task-level graph structure is a trainable parameter matrix, which is updated through back propagation The timing feature extraction module is configured to extract the adaptive spatial feature The input multi-scale time convolution layer performs M convolution branches with different sampling intervals d in parallel, and splices the outputs of the convolution branches to form the timing feature ; a feature fusion module configured to fuse the adaptive spatial features and the temporal features by a dual-stream adaptive fusion module to generate fused features ;​​ The state determination module is used to determine the fused features. The input is processed by the classification layer, and the output is the result of depression state recognition.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that: The processor executes the computer program to implement the steps of the depression gait recognition method based on multi-scale adaptive graph convolution according to any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to implement the steps of the depression gait recognition method based on multi-scale adaptive graph convolution according to any one of claims 1 to 6.

Images