Continuous scale assessment method and system for parkinsonian gait impairment

Abstract

The disclosure provides a continuous scale evaluation method and system for Parkinson gait disorder, and relates to the technical field of Parkinson gait analysis, which processes the gait skeleton sequence of a subject as follows: first, the gait skeleton sequence is converted into an upper limb feature map sequence and a lower limb feature map sequence, and then local spatio-temporal semantic features of the upper limb and the lower limb are extracted respectively; second, the local spatio-temporal semantic features of the upper limb and the lower limb are fused to obtain gait features; finally, the gait features are classified and regressed to obtain a coarse-grained classification vector and a continuous scale numerical score as a continuous scale evaluation result; the upper limb and the lower limb are separately modeled, and the self-attention mechanism is extended from the spatio-temporal dimension to the channel dimension, which enhances the ability of the model in capturing local spatio-temporal gait features, and finally, the flow type mixing technology and the linear regression method are used to realize the Parkinson gait disorder score in the continuous scale.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of Parkinson gait analysis, in particular to a continuous scale evaluation method and system for Parkinson gait disorder. BACKGROUND

[0002] Parkinson's disease is a common neurodegenerative disease characterized by the gradual loss of dopaminergic neurons. One of the main manifestations of this disease is movement disorders, among which gait disorder is one of the most common and prominent features, which has a greater impact on the daily life of patients.

[0003] Currently, clinicians evaluate the gait disorder of patients through the third part of the MDS-UPDRS scale, and this manual inspection requires regular outpatient visits, which is inefficient and subjective, and the changes in the patient's gait are easily overlooked, so there is an urgent need to develop an objective and efficient method for grading and quantifying the evaluation of Parkinson gait disorder.

[0004] Many studies have explored methods for estimating the MDS-UPDRS score of Parkinson gait based on skeleton data. In 2021, Professor Li Feifei's team at Stanford University first used skeleton data and convolutional networks to predict the MDS-UPDRS score of Parkinson gait disorder. However, convolutional networks cannot effectively capture the spatial dependencies between skeleton joints. Subsequently, Guo et al. combined multi-scale attention mechanisms and ST-GCN to effectively capture the spatio-temporal features in gait skeleton data, improving the accuracy of the model in grading and evaluating Parkinson gait disorder. Overall, these advances have proven the feasibility of using skeleton data for Parkinson gait analysis.

[0005] However, there are still significant limitations in current research, resulting in insufficient accuracy and performance in evaluating Parkinson gait disorder, which can be summarized as follows:

[0006] Firstly, clinical analysis of Parkinson gait disorder usually involves separate evaluation of upper and lower limb movement postures. Current methods for Parkinson gait analysis based on skeleton data mostly rely on classic ST-GCN network structures, which directly model gait features for the whole body skeleton, lacking the ability to model upper and lower limb gait features separately.

[0007] Secondly, in the clinical evaluation of Parkinson gait, medical experts emphasize the importance of local features such as stride, step width, foot lift height, heel strike, and arm swing. Although many methods use attention mechanisms to capture the spatio-temporal features of important joints, there is still a lack of network structures designed to model local gait features.

[0008] Finally, the clinician labels a discrete gait score for the patient according to a scoring scale, and based on this gait score label, the existing method estimates the MDS-UPDRS score of Parkinson gait as a coarse-grained classification problem, ignoring the progressive feature of Parkinson gait disorder; in fact, a continuous scale Parkinson gait disorder score is also expected in clinical diagnosis to analyze the development of the patient's gait disorder in detail, so as to realize more accurate medical scheme formulation; however, medical experts cannot directly give an effective continuous scale gait disorder score label. SUMMARY

[0009] To solve the above problems, the present disclosure proposes a continuous scale evaluation method and system for Parkinson gait disorder, which separately models the upper and lower limbs, and extends the self-attention mechanism from the space-time dimension to the channel dimension, enhancing the model's ability to capture local space-time gait features, and finally realizes the Parkinson gait disorder score on a continuous scale using a flow type mixing technique and linear regression method.

[0010] According to some embodiments, the present disclosure adopts the following technical solutions:

[0011] The continuous scale evaluation method for Parkinson gait disorder comprises:

[0012] obtaining a gait skeleton sequence of a subject;

[0013] inputting the gait skeleton sequence into a trained evaluation network to obtain a coarse-grained classification vector and a continuous scale numerical score as a continuous scale evaluation result;

[0014] The evaluation network first converts the gait skeleton sequence of the subject into an upper limb feature map sequence and a lower limb feature map sequence through gait skeleton graph normalization segmentation, and then extracts local space-time semantic features of the upper and lower limbs from the upper limb feature map sequence and the lower limb feature map sequence based on a full-dimensional self-attention convolution of a double-branch structure; secondly, the local space-time semantic features of the upper and lower limbs are fused by using a self-attention mechanism to obtain gait features; finally, the gait features are classified and regressed to obtain the final continuous scale evaluation result.

[0015] According to some embodiments, the present disclosure adopts the following technical solutions:

[0016] The continuous scale evaluation system for Parkinson gait disorder comprises:

[0017] The acquisition module is configured to obtain a gait skeleton sequence of a subject;

[0018] The evaluation module is configured to input the gait skeleton sequence into a trained evaluation network to obtain a coarse-grained classification vector and a continuous scale numerical score as a continuous scale evaluation result;

[0019] The evaluation network first converts the gait skeleton sequence of the subject into an upper limb feature map sequence and a lower limb feature map sequence through gait skeleton graph normalization segmentation, and then extracts local spatiotemporal semantic features of the upper limb and the lower limb from the upper limb feature map sequence and the lower limb feature map sequence based on a full-dimensional self-attention convolution of a double-branch structure; secondly, the local spatiotemporal semantic features of the upper limb and the lower limb are fused by using a self-attention mechanism to obtain gait features; finally, the gait features are classified and regressed to obtain the final continuous scale evaluation result.

[0020] According to some embodiments, the present disclosure adopts the following technical solution:

[0021] A computer program product comprising a computer program which, when executed by a processor, implements the continuous scale evaluation method for Parkinson gait disorder.

[0022] According to some embodiments, the present disclosure adopts the following technical solution:

[0023] A non-transitory computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the continuous scale evaluation method for Parkinson gait disorder.

[0024] According to some embodiments, the present disclosure adopts the following technical solution:

[0025] An electronic device comprising a processor, a memory and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory; when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the continuous scale evaluation method for Parkinson gait disorder.

[0026] Compared with the prior art, the present disclosure has the following beneficial effects:

[0027] The continuous scale evaluation method for Parkinson gait disorder can effectively model the local features and pedigree characteristics of Parkinson gait by using the full-dimensional self-attention convolution module and the continuous numerical score prediction strategy, thereby improving the performance of Parkinson gait disorder classification.

[0028] The method models the upper limb and the lower limb separately, and extends the self-attention mechanism from the spatial and temporal dimensions to the channel dimension, thereby enhancing the ability of the model in capturing local spatiotemporal gait features; finally, the flow type mixing technology and linear regression method are used to realize the Parkinson gait disorder score in the continuous scale, thereby solving the problems of insufficient precision and performance in Parkinson gait disorder evaluation. BRIEF DESCRIPTION OF DRAWINGS

[0029] The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the present disclosure, illustrate exemplary embodiments of the present disclosure, and to explain the principles of the present disclosure. The drawings should not be considered limiting of the present disclosure.

[0030] Figure 1 Network structure diagram for Example 1.

[0031] Figure 2 Gait skeleton diagram normalized segmentation diagram for Example 1.

[0032] Figure 3 Adaptive channel feature fusion diagram for Example 1.

[0033] Figure 4 Visualization diagram of real score interval and model predicted score for Example 1.

[0034] Figure 5 Correlation diagram between predicted ranking and expert ranking for Example 1. DETAILED DESCRIPTION

[0035] The present disclosure will be further described with reference to the drawings and examples.

[0036] It should be noted that the following detailed description is merely exemplary in nature and is intended to provide further description of the present disclosure. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0037] It should be noted that the terms used herein are merely for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and it should be further understood that the terms "comprise" and / or "comprising," when used in this specification, indicate the presence of the stated features, steps, operations, devices, components, and / or combinations thereof.

[0038] Example 1

[0039] In one embodiment of the present disclosure, a continuous scale evaluation method for Parkinson gait disorder is provided, comprising:

[0040] Obtaining a gait skeleton sequence of a subject;

[0041] Inputting the gait skeleton sequence into a trained evaluation network to obtain a coarse-grained classification vector and a continuous scale numerical score as a continuous scale evaluation result;

[0042] The evaluation network first converts the gait skeleton sequence of the subject into an upper limb feature map sequence and a lower limb feature map sequence through gait skeleton graph normalization segmentation, and then extracts local spatiotemporal semantic features of the upper limb and the lower limb from the upper limb feature map sequence and the lower limb feature map sequence based on an omni-dimensional self-attention convolution of a double-branch structure; secondly, the local spatiotemporal semantic features of the upper limb and the lower limb are fused by using a self-attention mechanism to obtain gait features; finally, the gait features are classified and regressed to obtain the final continuous scale evaluation result.

[0043] As an embodiment, the present disclosure proposes an omni-dimensional self-attention convolutional network (OSConvNet) for predicting the MDS-UPDRS score of Parkinson's gait on a continuous scale, and the functions of the OSConvNet are summarized as follows:

[0044] Firstly, in order to realize the separate modeling of the upper limb and lower limb features, the skeleton graph is divided into two normalized spatiotemporal feature maps: the upper limb feature map and the lower limb feature map. In these normalized feature maps, the root node and the neighboring joint nodes exhibit a grid-like spatiotemporal structure, which enables the convolution kernel to directly model the spatiotemporal features of the skeleton data.

[0045] Secondly, an omni-dimensional self-attention convolution module (OSConv) is designed to model the local spatiotemporal gait features in the upper limb and lower limb feature maps. Based on the standard convolution kernel, this module extends the self-attention mechanism from the spatiotemporal dimension to the channel dimension, enhancing the model's ability to capture local spatiotemporal gait features.

[0046] Then, an adaptive channel feature fusion module (ACFF) is constructed, which effectively fuses the local spatiotemporal gait features from the upper limb and lower limb by using the self-attention mechanism, providing sufficient complementary information for Parkinson's gait analysis.

[0047] Finally, a strategy for predicting the continuous numerical score of Parkinson's gait disorder is proposed, using the MDS-UPDRS score as an anchor point and using flow-type mixing technology to obtain mixed samples and mixed labels on a continuous scale, realizing the prediction of the Parkinson's gait disorder score on a continuous scale. This strategy not only effectively models the spectrum characteristics of Parkinson's gait disorder, but also improves the accuracy of coarse-grained classification of Parkinson's gait patterns under the MDS-UPDRS standard.

[0048] The following section provides a detailed explanation of OSConvNet's network structure, training method, and experimental results analysis:

[0049] I. Network Structure

[0050] like Figure 1 As shown, OSConvNet, in terms of structure, includes a gait skeleton graph normalization and segmentation module, a full-dimensional self-attention convolution module, an adaptive channel feature fusion module, and a classification and regression module.

[0051] 1. Gait skeleton diagram normalization and segmentation module

[0052] In clinical gait analysis, doctors not only observe the patient's stride length, gait speed, foot lift height, heel strike, and other lower limb characteristics, but also take the patient's arm swing as an important reference for gait scoring.

[0053] Inspired by the aforementioned clinical observations, this embodiment segments the skeleton map into upper limb and lower limb feature maps to model gait features in the upper and lower limb skeletons, respectively. Specifically:

[0054] The upper limb joints include: 3 cervical vertebrae, 4 left clavicle, 5 left shoulder, 6 left elbow, 7 left wrist, 11 right clavicle, 12 right shoulder, 13 right elbow, and 14 right wrist.

[0055] The lower limb joints include: 0 pelvis, 18 left hip, 19 left knee, 20 left ankle, 21 left foot, 22 right hip, 23 right knee, 24 right ankle, and 25 right foot.

[0056] like Figure 2 As shown, arranging the joints according to their connectivity yields two normalized feature maps. On these two normalized feature maps, the joints form a grid-like spatiotemporal structure, allowing for direct modeling of the spatiotemporal topology between the joints using traditional convolutional kernels.

[0057] The above skeleton normalization segmentation strategy can be expressed as follows:

[0058]

[0059] Where X represents the patient's gait skeleton sequence. These are upper limb feature map sequences and lower limb feature map sequences, respectively. T is the time length, N is the number of joints, and C0 is the dimension of the joint coordinates.

[0060] 2. Full-dimensional self-attention convolution module

[0061] To model the local spatio-temporal semantic features of Parkinsonian gait, this embodiment proposes an OSConv module, which adopts a double-branch structure to model the local spatio-temporal semantic features within the upper and lower limb feature maps respectively. Each branch of the model is composed of a patch enconding (PE) layer and three OSConv modules, and the output features are where the channel numbers are C0=64, C1=64, C2=128, and C3=256, and the specific steps are as follows:

[0062] (1) The input feature map is divided into a predetermined group along the channel dimension, and each group of feature maps is taken as the input of a single head in the multi-head mechanism.

[0063] Specifically, the input upper limb feature map sequence and lower limb feature map sequence are divided into H groups along the channel dimension, and the hth group of features is taken as the input of the hth single head of the OSConv, where D=C in / H represents the input feature channel number of the single head of the OSConv.

[0064] (2) After calculating the features of each joint on each channel using the single head, a splicing operation is performed along the channel dimension to obtain the features of the single head.

[0065] Specifically, the local spatio-temporal neighborhood N k of the joint n ij is extracted as the center, where j represents the time index, i is the spatial index of the joint, and N k is the coverage area of the convolution kernel with a size of k x k at position (i, j), which is taken as the neighborhood of the joint n ij , and this neighborhood can be represented as In this local neighborhood, the spatio-temporal self-attention weight is calculated as follows:

[0066]

[0067] where, is the feature vector of the joint n ij , and are learnable parameter matrices, is the query vector of the joint n ij , is the key vector of the joint n ij and its neighborhood nodes, and softmax is a normalization exponential function, is the local spatio-temporal self-attention weight between the joint n ij and its neighborhood nodes.

[0068] To further obtain the neighborhood Nk The channel self-attention weight in the neighborhood of the node n is transformed into The channel self-attention weight is then calculated as follows:

[0069]

[0070] wherein, and are learnable feature linear transformation matrices, tanh is a hyperbolic tangent function, is a query matrix, is a key matrix, is the output feature of the node n ij in the neighborhood.

[0071] In the h-th head of the OSConv, the output feature of the node n ij is calculated as follows: i.e., the feature of a single head, as follows:

[0072]

[0073] wherein, is a convolution kernel on the neighborhood of the node n ij , is the d-th row of , is the output feature of the d-th channel of the node n ij , is a Hadamard product, and concat is a concatenation operation along the channel dimension.

[0074] (3) The features of all single heads are fused to obtain local spatio-temporal semantic features.

[0075] Specifically, the local spatio-temporal semantic features output by the multi-head OSConv are represented as:

[0076] X out = FC(concat(X′ 1 ,…,X′ H )),

[0077] wherein, represents the output feature of a single OSConv head, and a fully connected layer (FC) is used to perform a non-linear transformation on the multi-head features to obtain a high-dimensional representation, represents the output feature of the multi-head OSConv.

[0078] The above OSConv module uses a multi-head mechanism to learn diverse feature representations from different subspaces, and obtains local spatio-temporal semantic features of the upper and lower limbs, respectively.

[0079] 3. Adaptive channel feature fusion module

[0080] Inspired by the self-attention mechanism, an adaptive channel feature fusion module (ACFF) is designed to fuse the deep features from the upper and lower limbs, as shown in Figure 3 Specifically, as shown in

[0081] First, a depth separable convolution is used to obtain the query vector and key vector of each channel, and then the vector product of the query vector and the key vector is calculated as the channel weight. The calculation method is represented by the formula:

[0082]

[0083] wherein, is the local spatiotemporal semantic feature of the upper and lower limbs, referred to as the upper and lower limb feature map, DC k and DC q is a depth separable convolution layer, ° is a Hadamard product, and GSP is a global sum pooling operation, are the channel weights of the upper and lower limb feature maps, respectively.

[0084] Then, the channel weight is normalized to obtain the final channel attention weight, as shown in the following formula:

[0085]

[0086] wherein, is the normalized channel attention weight.

[0087] Finally, the normalized channel attention weight is used to fuse the deep features from the upper and lower limb skeleton maps, as follows:

[0088]

[0089] wherein, GAP is a global average pooling, is the final fusion feature, i.e., the gait feature.

[0090] The above ACFF module uses the self-attention mechanism to obtain rich complementary information from the upper and lower limb feature maps, improving the performance of the model in Parkinson gait analysis.

[0091] 4. Classification and regression module

[0092] Based on gait features, coarse-grained classification vectors and numerical scores are predicted using the Softmax classifier CLF and linear regression LR, respectively. Specifically, the gait features are classified according to the MDS-UPDRS standard using the classifier to obtain coarse-grained classification vectors of Parkinson's gait. Linear regression is performed on the gait features to obtain numerical scores on a continuous scale with the MDS-UPDRS score as the anchor.

[0093] Parkinson's gait coarse-grained classification vector based on MDS-UPDRS standard and continuous numerical ratings anchored to MDS-UPDRS scores As the final continuous-scale assessment result of Parkinson's gait disorder.

[0094] II. Training Methods

[0095] Existing methods treat Parkinson's gait MDS-UPDRS score estimation as a coarse-grained classification problem, ignoring the progressive characteristics of Parkinson's gait. To address this, this embodiment employs manifold mixing and linear regression to achieve Parkinson's gait impairment scoring on a continuous scale. This strategy can effectively model the spectrum characteristics of Parkinson's gait impairment. Furthermore, clinicians cannot directly provide effective continuous-scale Parkinson's gait score labels, while manifold mixing provides the model with a mixed label on a continuous scale, solving this key problem.

[0096] 1. Flow pattern mixing

[0097] The full-dimensional self-attention convolutional network OSConvNet is divided into two segments: the neural network segment before layer l and the neural network segment after layer l. OSConvNet can be represented as f(X) = f l (g l (X)), where g l This represents the neural network segment preceding layer l, which maps the input data to the hidden features of layer l. l This represents the neural network segment after layer l, which hides the feature g. l (X) is mapped to the output f(X).

[0098] To train f using a manifold hybrid strategy, firstly, a layer l is randomly selected, and two randomly selected sample data (X,y,s) and (X′,y′,s′) are mapped to the hidden features of the l-th layer, i.e., g. l (X)=X l and g l (X′)=X′ lwhere (y, y') represents the one-hot label vector corresponding to the true MDS-UPDRS score, i.e., the classification label, and (s, s') represents the numerical MDS-UPDRS score provided by medical experts; then, the hidden features, classification labels, and numerical scores of the two samples are mixed to obtain mixed data and mixed labels, which are represented by the following formulas:

[0099]

[0100] wherein, is the mixed data of hidden features, is the mixed label, and the mixing coefficient β is randomly sampled from the Beta distribution.

[0101] Finally, the mixed data is obtained by continuing forward propagation from the lth layer to obtain the final output:

[0102]

[0103] wherein, is the predicted output corresponding to the mixed data, which is used to calculate the loss value and gradient of the mixed output and update the parameters of the neural network.

[0104] 2. Classification and regression

[0105] As mentioned above, the proposed OSConvNet has two outputs, one is the classification vector corresponding to the MDS-UPDRS score, i.e., the coarse-grained classification vector, and the other is the numerical score on a continuous scale; for the classification vector, the cross-entropy loss is used to minimize the classification error, and the classification loss function is constructed as shown in the following formula:

[0106]

[0107] wherein, N c is the number of sample categories, y c and y' c are the classification labels of the two sample data in the cth category.

[0108] When predicting the continuous scale Parkinson's gait score, the mean square error loss is used to minimize the error of the predicted score. The mean square error loss is a commonly used index for evaluating regression performance, and a lower mean square error loss value indicates that the model's prediction is closer to the actual observation value, representing better model performance. The regression loss function is constructed as shown in the following formula:

[0109]

[0110] The loss function optimized by the OSConvNet includes the above classification loss and regression loss, and the final loss function is shown in the following formula:

[0111] L = L c + γL r ,

[0112] where γ is a balance parameter to balance the importance of the two loss terms.

[0113] III. Experimental Results and Analysis

[0114] To verify the effectiveness of the proposed method, a sufficient ablation experiment was conducted on the gait dataset, and the proposed method was compared with the advanced method to verify the effectiveness of the proposed continuous numerical score prediction strategy for Parkinson's gait disorder.

[0115] 1. The effectiveness of the proposed method was verified by ablation experiments, as shown in Table 1:

[0116] Table 1 Ablation experiment

[0117]

[0118] (1) Gait skeleton normalization segmentation strategy and double branch model structure

[0119] The effectiveness of the gait skeleton normalization segmentation strategy and the double branch structure was verified by testing the upper limb branch and the lower limb branch respectively; compared with the OSConvNet of the double branch structure, the upper limb branch and the lower limb branch significantly decreased in all evaluation indicators; specifically, the accuracy of the lower limb branch decreased by 2.46%, the F1 score decreased by 3.56%, and the AUC value decreased by 4.23%. The accuracy of the upper limb branch decreased by 6.39%, the F1 score decreased by 6.19%, and the AUC value decreased by 7.4%.

[0120] The above experimental results show that modeling and fusing the gait features in the upper limb and lower limb skeletons using the double branch structure can effectively improve the accuracy of the model in analyzing Parkinson's gait disorder.

[0121] (2) Full-dimensional self-attention convolution (OSConv)

[0122] To evaluate the effectiveness of the full-dimensional self-attention convolution module (OSConv), four ablation models were designed based on the OSConv module, including: removing the convolution kernel (W d h ) and the channel self-attention weight (Ac) of the local self-attention mechanism (LSA), removing the self-attention weight (A s and A c ) of the original CNN model, removing the channel self-attention weight (A cSpatial Attention CNN (SA-CNN) and Channel Attention CNN (CA-CNN) without the spatial-temporal attention weights (A s ).

[0123] The experimental results show that the performance of the OSConv module is significantly better than the original LSA and CNN, which proves that the OSConv module can effectively model the local spatial-temporal features of Parkinsonian gait. In addition, among the four ablation models, CA-CNN achieves the best performance, which further proves the effectiveness of the proposed channel self-attention A c the effectiveness in modeling the local spatial-temporal dependencies between joints.

[0124] (3) Adaptive Channel Feature Fusion (ACFF)

[0125] To verify the effectiveness of the adaptive channel feature fusion module (ACFF), the ACFF module in the OSConvNet network is replaced with additive fusion, i.e., additive fusion (w / o ACFF).

[0126] The experimental results show that, compared with ACFF, additive fusion (w / o ACFF) leads to a significant decrease in the classification performance of the model; specifically, the accuracy decreases by 1.98%, the F1 score decreases by 2.62%, and the AUC value decreases by 2.4%; these experimental results fully demonstrate the effectiveness of the proposed ACFF module in feature fusion.

[0127] (4) Parkinsonian gait disorder continuous numerical score prediction strategy

[0128] To verify the effectiveness of the proposed Parkinsonian gait disorder continuous numerical score prediction strategy, three ablation models of OSConvNet are set up, which are: a model without flow type mixing technology (w / o flow type mixing), a model without linear regression (w / o Lr), and a model without both flow type mixing and linear regression (w / o flow type mixing and L r ).

[0129] The experimental results show that removing flow type mixing and linear regression (L r ) significantly reduces the classification performance of the model for Parkinsonian gait disorder, which proves the effectiveness of the proposed continuous numerical score prediction strategy in modeling the Parkinsonian gait spectrum characteristics; among the three ablation models, the model without linear regression (w / o L r) achieved the lowest classification accuracy; this experimental result shows that the use of flow pattern mixing alone cannot improve the classification accuracy of Parkinson gait, because Parkinson gait disorder is a continuous spectrum rather than a discrete category, and the use of flow pattern mixing alone makes the boundaries between different gait categories more blurred, thereby reducing the model's ability to distinguish different Parkinson gait disorders.

[0130] 2. Performance comparison with advanced algorithms

[0131] To verify the superiority of the proposed model in Parkinson gait disorder classification, a comparative analysis was conducted on the gait dataset with 7 advanced methods, including 3 abnormal gait analysis methods: OFDDNet, GaitGraph2, CST-GCN and 4 advanced human skeleton behavior recognition methods: 2s-AGCN, CTR-GCN, ST-GCN, Ms-G3D, and the comparison results are shown in Table 2:

[0132] Table 2 Comparison with advanced algorithms

[0133]

[0134] From Table 2, it can be seen that the proposed OSConvNet is superior to other advanced models in various evaluation indicators, with an accuracy of 62.78%, an F1-score of 62.57%, and an AUC of 79.34%; the experimental results show that the proposed method can effectively model the local features and spectrum characteristics of Parkinson gait through the full-dimensional self-attention convolution module and continuous numerical score prediction strategy, and improve the performance of Parkinson gait disorder classification.

[0135] 3. Verification of the continuous numerical score prediction strategy for Parkinson gait disorder

[0136] In the experiment to verify the effectiveness of the continuous numerical score prediction strategy for Parkinson gait disorder, the gait dataset was divided into training and test sets. In the testing process, the multi-view gait data of each subject was input into the model to obtain the corresponding gait impairment prediction scores. Then, the average of these scores was taken as the numerical score of each patient's gait disorder.

[0137] The effectiveness of the Parkinson gait impairment continuous numerical score prediction strategy is evaluated using classification accuracy and correlation analysis; the classification accuracy includes fine-grained classification accuracy and coarse-grained classification accuracy; the fine-grained classification accuracy refers to the accuracy of the predicted score matching the fine-grained score interval provided by experts; the coarse-grained classification accuracy refers to the accuracy of the predicted score matching the original MDS-UPDRS score interval. According to the predicted score, a predicted ranking of the severity of gait impairment of 37 Parkinson patients is obtained, and the correlation analysis includes Spearman correlation analysis and Kendall correlation analysis of the predicted ranking and the expert ranking.

[0138] In Figure 4 The real score interval and the predicted score of 37 patients are visualized, and the predicted score of 23 patients (blue dots) matches the score interval marked by experts, with a fine-grained accuracy of 62.16%. At the same time, the coarse-grained classification accuracy reaches 86.5%, and only 5 patients are misclassified (red circles); Figure 5 The correlation between the predicted ranking and the expert ranking is shown, and the scatter plot shows that the data points are distributed from the lower left corner to the upper right corner, indicating a positive correlation between the predicted ranking and the expert ranking. The Spearman correlation coefficient r = 0.88, and the p-value p = 4.1 x 10 -13 , proving that there is a significant positive correlation between the predicted ranking and the expert ranking; thereby verifying the effectiveness of the Parkinson gait impairment continuous numerical score prediction strategy.

[0139] Embodiment 2

[0140] An embodiment of the present disclosure provides a continuous scale evaluation system for Parkinson gait impairment, comprising:

[0141] An acquisition module configured to acquire a gait skeleton sequence of a subject;

[0142] An evaluation module configured to input the gait skeleton sequence into a trained evaluation network to obtain a coarse-grained classification vector and a continuous scale numerical score as a continuous scale evaluation result;

[0143] The evaluation network first converts the gait skeleton sequence of the subject into an upper limb feature map sequence and a lower limb feature map sequence through gait skeleton graph normalization segmentation, and then extracts local spatio-temporal semantic features of the upper limb and the lower limb from the upper limb feature map sequence and the lower limb feature map sequence based on a full-dimensional self-attention convolution of a double-branch structure; secondly, the local spatio-temporal semantic features of the upper limb and the lower limb are fused using a self-attention mechanism to obtain gait features; finally, the gait features are classified and regressed to obtain the final continuous scale evaluation result.

[0144] Embodiment 3

[0145] In an embodiment of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method for continuous scale assessment of Parkinson gait disorder.

[0146] Embodiment 4

[0147] In an embodiment of the present disclosure, a non-transitory computer readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the method for continuous scale assessment of Parkinson gait disorder.

[0148] Embodiment 5

[0149] In an embodiment of the present disclosure, an electronic device is provided, comprising a processor, a memory, and a computer program; wherein the processor is connected with the memory, and the computer program is stored in the memory; when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device implements the method for continuous scale assessment of Parkinson gait disorder.

[0150] The present disclosure is described with reference to the flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The means for implementing the functions specified in one or more flows and / or blocks.

[0151] These computer program instructions can also be loaded onto a computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate a computer-implemented process, so that the instructions executed on the computer or other programmable devices provide a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The steps for implementing the functions specified in one or more flows and / or blocks.

[0152] The specific embodiments of the present disclosure are described above with reference to the accompanying drawings, but are not intended to limit the protection scope of the present disclosure, and those skilled in the art should understand that various modifications or changes made on the basis of the technical solutions of the present disclosure without creative labor are still within the protection scope of the present disclosure.

Claims

1. A method for the continuous scale assessment of parkinsonian gait impairment, characterized in that, The method comprises the following steps: obtaining a gait skeleton sequence of a subject; inputting the gait skeleton sequence into a trained evaluation network to obtain a coarse-grained classification vector and a numerical score in a continuous scale as a continuous scale evaluation result; wherein the evaluation network first converts the gait skeleton sequence of the subject into an upper limb feature map sequence and a lower limb feature map sequence through gait skeleton graph normalization segmentation, and then extracts local spatiotemporal semantic features of the upper limb and the lower limb from the upper limb feature map sequence and the lower limb feature map sequence based on a full-dimensional self-attention convolution of a double-branch structure; secondly, the local spatiotemporal semantic features of the upper limb and the lower limb are fused by using a self-attention mechanism to obtain gait features; finally, the gait features are classified and regressed to obtain the final continuous scale evaluation result; the evaluation network is trained for continuous scale evaluation of Parkinson's gait disorder by using a flow type mixing technology to obtain mixed samples and mixed labels in a continuous scale.

2. The continuous scale assessment method for parkinson gait disorder according to claim 1, wherein, The gait skeleton sequence is a group of skeleton graphs arranged in chronological order, which are composed of a plurality of joints and joint coordinates and used for representing the gait of a patient at a time index; the gait skeleton graph normalization segmentation is to divide the joints in the skeleton graph into two categories of upper limb joints and lower limb joints, and arrange the joints of the same category according to the connection relationship to obtain two standardized upper limb feature maps and lower limb feature maps.

3. The continuous scale assessment method for parkinson gait disorder according to claim 1, wherein, The full-dimensional self-attention convolution is to learn diverse feature expressions from different subspaces by using a multi-head mechanism, specifically as follows: divide the input feature map into a predetermined group along the channel dimension, and use each group of feature maps as the input of a single head in the multi-head mechanism; after calculating the features of each joint on each channel by using a single head, perform a splicing operation along the channel dimension to obtain the features of the single head; fuse the features of all single heads to obtain local spatiotemporal semantic features.

4. The continuous scale assessment method for parkinson gait disorder according to claim 1, wherein, The fusion of the local spatiotemporal semantic features of the upper limb and the lower limb is specifically as follows: use a depth separable convolution to obtain a query vector and a key vector for each channel, and calculate the vector product of the query vector and the key vector as a channel weight; normalize the channel weight to obtain the final channel attention weight; fuse the local spatiotemporal semantic features of the upper limb and the lower limb by using the channel attention weight.

5. The continuous scale assessment method for parkinson gait disorder according to claim 1, wherein, The classification and regression of the gait features include: perform linear regression on the gait features to obtain a numerical score in a continuous scale with MDS-UPDRS score as an anchor point; perform classification on the gait features based on the MDS-UPDRS standard to obtain a Parkinson's gait coarse-grained classification vector.

6. A continuous scale assessment system for Parkinsonian gait impairment, characterized by, The method comprises the following steps: an obtaining module configured to obtain a gait skeleton sequence of a subject; an evaluation module configured to input the gait skeleton sequence into a trained evaluation network to obtain a coarse-grained classification vector and a numerical score in a continuous scale as a continuous scale evaluation result; The evaluation network first converts the gait skeleton sequence of the subject into an upper limb feature map sequence and a lower limb feature map sequence through gait skeleton graph normalization segmentation, and then extracts local spatiotemporal semantic features of the upper limb and the lower limb from the upper limb feature map sequence and the lower limb feature map sequence based on a full-dimensional self-attention convolution of a double-branch structure; secondly, the local spatiotemporal semantic features of the upper limb and the lower limb are fused by using a self-attention mechanism to obtain gait features; finally, the gait features are classified and regressed to obtain a final continuous scale evaluation result. The evaluation network is trained for the continuous scale evaluation of Parkinson's gait disorder by using a flow type mixing technology to obtain mixed samples and mixed labels of the continuous scale.

7. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the continuous scale evaluation method for Parkinson's gait disorder according to any one of claims 1-5.

8. A non-transitory computer-readable storage medium, comprising: The non-transitory computer readable storage medium is used to store computer instructions, and the computer instructions, when executed by a processor, implement the continuous scale evaluation method for Parkinson's gait disorder according to any one of claims 1-5.

9. An electronic device, comprising: Comprise: A processor, a memory and a computer program; wherein the processor is connected with the memory, and the computer program is stored in the memory; when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the continuous scale evaluation method for Parkinson's gait disorder according to any one of claims 1-5.

Images