Establishing parkinsonian gait impairment assessment model, method, device and medium
By preprocessing gait videos taken from the side and training them with a two-stream neural network, a gait impairment assessment model for Parkinson's disease was established. This solved the problems of time-consuming and subjective manual assessment, and enabled a rapid and objective assessment of gait impairment.
Patent Information
- Application Number
- CN202310387454.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-03
- Publication Date
- 2026-03-03
- Estimated Expiration
- 2043-04-03
AI Technical Summary
Current technologies for gait assessment in Parkinson's disease patients rely on manual assessment methods, which are time-consuming, highly subjective, and cannot be automatically assessed at home.
By preprocessing gait videos captured from the side, extracting skeleton map sequences and multi-cycle gait energy maps, and training them with a two-stream neural network, a gait impairment assessment model for Parkinson's disease is established, and the neural network is used to automatically assess gait videos.
It enables rapid and objective assessment of gait impairment in Parkinson's disease, improving the accuracy and convenience of the assessment, and is suitable for home use.
Smart Images
Figure CN116543455B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of medical image processing, and in particular relates to a method, device and medium for establishing a gait impairment assessment model for Parkinson's disease. Background Technology
[0002] Parkinson's disease (PD) is a common neurodegenerative disease, characterized by resting tremor, bradykinesia, rigidity, and postural instability, which severely impacts patients' quality of life. Early and timely diagnosis is crucial for initiating neuroprotective treatment and subsequent management.
[0003] Currently, the main basis for assessing motor function in Parkinson's disease is the MDS-UPDRS assessment scale. The third part of this scale evaluates the motor symptoms of PD, requiring raters to score the severity of 18 items, including gait, upper limb flexibility, and leg flexibility, on a scale of 0-4, based on their observations of the patient. A score of 0 indicates normal, and a score of 4 indicates severe.
[0004] Clinically, experienced neurologists use this scale to quantitatively assess patients' motor and non-motor symptoms. However, this manual assessment method presents two challenges: firstly, it is time-consuming and subjective; secondly, it requires patients to visit the hospital regularly, which is not conducive to long-term disease management and control. China has the largest number of Parkinson's patients in the world, but due to the lack of obvious early symptoms and high medical costs, the consultation rate is less than 40%.
[0005] Deep learning learns the inherent patterns and hierarchical representations of sample data. The information gained during this learning process greatly aids in interpreting data such as text, images, and sound. Its ultimate goal is to enable machines to possess analytical and learning capabilities like humans, allowing them to recognize data such as text, images, and sound. Therefore, how to utilize deep learning models to assess gait impairment in Parkinson's disease is a pressing research topic. Summary of the Invention
[0006] To address the problems of time-consuming and subjective manual assessments of Parkinson's disease symptoms in existing technologies, and the inability of patients to conduct automated assessments at home, this application proposes a method, equipment, and medium for establishing a gait impairment assessment model for Parkinson's disease. The technical solution is as follows:
[0007] On the one hand, this application provides a method for establishing a gait impairment assessment model for Parkinson's disease, including the following steps:
[0008] S10: Preprocess the gait video to extract the skeleton map sequence and multi-cycle gait energy map, where the gait video is a walking video of a Parkinson's patient taken from the side.
[0009] S20: Using skeleton graph sequences and multi-cycle gait energy maps, combined with gait impairment rating labels obtained by doctors based on gait videos, a neural network is trained to obtain a gait impairment assessment model for Parkinson's disease, where the gait impairment rating labels are used to classify the degree of gait impairment.
[0010] Furthermore, the gait impairment assessment model for Parkinson's disease is a two-stream neural network, including skeleton flow, contour flow, vector splicing unit, fully connected layer, and softmax layer;
[0011] The skeleton stream contains multiple ST-GCN units to process the input skeleton map sequence and obtain the skeleton stream output vector. The contour stream contains multiple VGG units to process the input multi-period gait energy map and obtain the contour stream output vector. The skeleton stream output vector and the contour stream output vector are concatenated by the input vector concatenation unit, and then the two streams are fused through a fully connected layer. Finally, the softmax layer is passed to obtain the predicted probability value of each score. The score with the highest predicted probability value is the gait score estimate of the gait video.
[0012] Furthermore, step S10, which involves preprocessing the gait video and extracting multi-cycle gait energy maps, specifically includes:
[0013] S101: Extract contour maps from each frame of the gait video, arrange the contour maps in chronological order, and obtain a contour map sequence;
[0014] S102: Determine the gait energy cycle based on the contour map sequence;
[0015] The gait energy cycle is the time interval between adjacent peaks and troughs in the curve of the distance between the two feet over time in the profile sequence. The starting point of the gait energy cycle is the time point corresponding to the peak or trough.
[0016] S103: Divide the contour map sequence into a groups according to the gait energy cycle, and superimpose the contour maps in each group to form a gait energy map, thereby obtaining a gait energy maps.
[0017] S104: Connect the a gait energy maps in chronological order in the horizontal direction to obtain a multi-cycle gait energy map.
[0018] Furthermore, the calculation of the superposition of the contour maps within each group to form a gait energy map in step S103 is as follows:
[0019]
[0020] Among them, I c(x,y,n) is the contour map extracted from the nth frame of the gait video, N is the number of video frames contained in one gait energy cycle, x is the horizontal coordinate of the pixel in the contour map, y is the vertical coordinate of the pixel in the contour map, and GEI(x,y) takes the value of pixels.
[0021] Furthermore, step S10 involves preprocessing the gait video and extracting the skeleton map sequence, specifically including:
[0022] S111: Extract the two-dimensional coordinates of the joints from each frame of the gait video;
[0023] S112: Calculate the skeleton sequence corresponding to the gait video based on the two-dimensional coordinates of the joints and the spatial connection relationship between the joints;
[0024] Furthermore, the calculation of the skeleton sequence corresponding to the gait video in step S112 based on the two-dimensional coordinates of the joints and the spatial connection relationship between the joints is as follows:
[0025] J = (V, E)
[0026] The node set V includes the two-dimensional coordinates of the joints in all frames of the gait video; the edge set E includes two types of spatial connections: proximal and distal; the proximal connections include the left and right shoulders, spine, left and right hips, left and right upper arms, and left and right thighs, while the distal connections include the left and right lower arms and left and right calves.
[0027] The spatial connectivity of edge set E is represented by adjacency matrix A, where the number a in the i-th row and j-th column of matrix A is... ij This represents the spatial connection between joint i and joint j. If the spatial connection is proximal, then a ij =x, if the spatial connection is farthest, then a ij = y, where x and y are positive integers, and 1 <x<y。
[0028] Furthermore, the edge set E also includes a temporal connection relationship: temporal connection; the temporal connection includes the connection between the current node of each joint and its next node;
[0029] Temporal connections in edge set E are achieved through one-dimensional convolutions of joints along the time direction, as shown below:
[0030]
[0031] Among them, X t The sequence of two-dimensional coordinates of a certain joint along the time direction is a t x 2 matrix, where t is the number of frames in the gait video, N is the number of convolution kernels in the temporal convolution, and W... k Let K be the k-th temporal convolution kernel, which is a one-dimensional vector, and B be a learnable linear bias.
[0032] Furthermore, the calculation of the ST-GCN cell is as follows:
[0033]
[0034] Among them, X in As input features, X out For output features, A is the adjacency matrix, and D is the adjacency matrix. ii =∑ j (A ij +I ij ) is the degree matrix used to normalize the adjacency matrix A, W is the learnable weight coefficient matrix, and B is the learnable bias coefficient.
[0035] Furthermore, the VGG unit includes a two-dimensional convolutional layer and a max pooling layer.
[0036] Furthermore, training the neural network in step S20 also includes:
[0037] The loss value is calculated using the loss function, and the loss value is backpropagated. Training stops when the error converges.
[0038] The loss function used is cross-entropy, and the specific calculation is as follows:
[0039]
[0040] Where N is the number of samples, K is the number of categories, and p ic Let represent the probability that the class of the i-th sample is c, i.e.
[0041] Furthermore, y ic ∈{0,1}, when the class of the i-th sample is c, the corresponding y ic =1, otherwise y ic =0.
[0042] On the other hand, this application provides a method for assessing gait impairment in Parkinson's disease, including the following steps:
[0043] S1: Preprocess the gait video to extract the skeleton map sequence and multi-cycle gait energy map, where the gait video is a walking video of the person to be evaluated taken from the side.
[0044] S2: The Parkinson's disease gait impairment assessment model is obtained by inputting the skeleton diagram sequence and multi-cycle gait energy map into any of the above methods to establish a gait impairment assessment model, and the gait score of the person to be assessed is obtained.
[0045] Furthermore, step S2 includes using a voting mechanism to obtain the gait score optimization result, specifically:
[0046] The gait videos of the same person to be evaluated are grouped, and multiple gait scores are obtained for each group. The score that appears most frequently in the multiple gait scores is selected as the gait score optimization result for the person to be evaluated.
[0047] Furthermore, following step S2, the following steps are also included:
[0048] S3: Calculate the joint response values of the skeleton diagram sequence in the gait impairment assessment model for Parkinson's disease;
[0049] S4: Classify and calculate the joint response values to obtain the average response value for each body part;
[0050] S5: Based on joint response values and average response values, obtain refined gait assessment results.
[0051] Furthermore, the joint response values are calculated as follows:
[0052]
[0053] Where S is the joint response value vector, N is the number of joints, each value in the joint response value vector represents the response value of the Parkinson's disease gait impairment assessment model at that joint, T is the length of the input skeleton diagram sequence, and C... out O is the number of output channels for the skeleton stream. it X is the output matrix of the skeleton flow. out The elements in.
[0054] Furthermore, the joints are divided into 6 groups, representing 6 body parts: neck, torso, left arm, right arm, left leg, and right leg. In step S3, the joint response values are classified and calculated to obtain the average response value for each body part, specifically including:
[0055] Calculate the average response value for each group of joints to obtain the average joint response value for each body part.
[0056] On the other hand, this application provides a computer device, which includes a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein the processor executes the program to implement the method for establishing a gait impairment assessment model for Parkinson's disease as described in any of the above claims, or to implement the method for assessing gait impairment for Parkinson's disease as described in any of the above claims.
[0057] In another aspect, this application provides a readable storage medium storing a computer program, which, when executed by a processor, implements the method for establishing a gait impairment assessment model for Parkinson's disease, or the method for assessing gait impairment for Parkinson's disease as described above.
[0058] The beneficial effects of this application are as follows: This application's solution utilizes the contours and joints extracted from gait videos to generate two different data streams: gait energy cycle diagrams and skeleton diagrams. Combined with neural network training, a corresponding neural network model is generated. This neural network model can analyze the gait videos of Parkinson's disease patients to be evaluated to obtain gait scores. Based on the gait scores, the patient's condition is assessed. This neural network model can conveniently and quickly assess the degree of gait impairment in Parkinson's patients with high accuracy. Attached Figure Description
[0059] Figure 1A This is a schematic diagram of the framework of an embodiment of the method for establishing a gait impairment assessment model for Parkinson's disease according to this application;
[0060] Figure 1B This is a schematic diagram of the framework of an embodiment of the method for assessing gait impairment in Parkinson's disease according to this application;
[0061] Figure 2A This is a flowchart of an embodiment of the method for establishing a gait impairment assessment model for Parkinson's disease according to this application;
[0062] Figure 2B yes Figure 2A A sub-flowchart of one embodiment of step S10;
[0063] Figure 2C yes Figure 2A A sub-flowchart of one embodiment of step S10;
[0064] Figure 3 This is a flowchart of an embodiment of the method for assessing gait impairment in Parkinson's disease according to this application;
[0065] Figure 4 This is a flowchart of an embodiment of the method for refined assessment of gait impairment in Parkinson's disease according to this application;
[0066] Figure 5 This is a flowchart of an embodiment of the method for quantifying gait impairment in Parkinson's disease according to this application;
[0067] Figure 6 This is a schematic diagram of the framework of the Parkinson's disease gait impairment assessment model in this application;
[0068] Figure 7 This is a schematic diagram of an embodiment of gait video acquisition according to this application;
[0069] Figure 8 This is a schematic diagram of the entire process of an embodiment of the method for assessing gait impairment in Parkinson's disease according to this application;
[0070] Figure 9 This is a schematic diagram of one embodiment of the skeleton diagram of this application;
[0071] Figure 10 This is a waveform diagram of an embodiment of the change in the distance between the two feet over time in the gait video of this application;
[0072] Figure 11 This is a schematic diagram of an embodiment of the multi-cycle gait energy map of this application;
[0073] Figure 12 This is a distribution diagram of the average response values of each joint in one embodiment of this application;
[0074] Figure 13 This is a distribution diagram of the average response values of various body parts in one embodiment of this application;
[0075] Figure 14 This is a ROC curve diagram corresponding to different scores in one embodiment of this application;
[0076] Figure 15 This is a confusion matrix in one embodiment of this application;
[0077] Figure 16 This is a schematic diagram of the hardware operating environment of an embodiment of the method for establishing a gait impairment assessment model for Parkinson's disease or a method for assessing gait impairment for Parkinson's disease. Detailed Implementation
[0078] To facilitate understanding of this application, a more detailed description is provided below with reference to the accompanying drawings and specific embodiments. Preferred embodiments of this application are shown in the drawings. However, this application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application.
[0079] It should be noted that, unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. For example, the term "a plurality of" includes two or more.
[0080] The following are explanations of some terms used in this application:
[0081] ST-GCN: Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition.
[0082] VGG: VGG is an image recognition network proposed by the Visual Geometry Group in Oxford in 2014. It uses a deeper network structure and has proven that increasing network depth can affect network performance to some extent.
[0083] Gait is a useful biomarker for early identification of Parkinson's disease. Many studies currently attempt to collect gait information using wearable devices such as wristbands to achieve early screening for Parkinson's disease. However, because these wearable devices can only measure movement in a specific area, they cannot fully characterize overall movement characteristics, and key factors such as lateralization and coordination are difficult to detect.
[0084] In view of this, this application provides a method, device and medium for establishing and using a gait impairment assessment model for Parkinson's disease. Only a smartphone is needed to collect video samples, and gait analysis is performed based on the collected video samples to obtain assessment results that better reflect the overall movement characteristics.
[0085] refer to Figure 1A-1B This demonstrates how to build and use a model.
[0086] Specifically, refer to Figure 1A This method is used to establish a gait impairment assessment model for Parkinson's disease. After collecting gait videos, the videos need to be manually evaluated and labeled. For example, a doctor might watch each collected gait video and assign an MDS-UPDRS score. The label file X corresponding to that gait video contains this score information. The gait videos are then preprocessed, and the preprocessed data is fed into a neural network for model training to obtain a prediction result Y. Training stops when the difference between the prediction result Y and the label file X meets a pre-set threshold. At this point, the neural network is the successfully trained gait impairment assessment model for Parkinson's disease.
[0087] Further, refer to Figure 1B This method uses a pre-trained Parkinson's disease gait impairment assessment model for evaluation. Instead of manual evaluation and labeling of the collected gait videos, the pre-processed videos are directly fed into the Parkinson's disease gait impairment assessment model. The model calculates and outputs a prediction result Z, which is the model's score for the gait video. This process can be automated and requires no doctor's intervention.
[0088] Furthermore, the process of building a model (training a model) and the process of using the model are separate. Once a model is trained, it can be used an unlimited number of times (e.g., tens of thousands of times) and can be run on different hardware environments.
[0089] Further, refer to Figure 7 This is one embodiment of a gait video acquisition method. As shown in the figure, a mobile phone (or camera) is mounted on a phone holder with its long side parallel to the ground, approximately 5 meters away from the subject's sagittal plane. This ensures that the mobile phone (or camera) has a field of view of approximately 5 meters when the subject walks in a direction parallel to their sagittal plane. The subject is required to walk from the left side of the video frame to the right side, then turn around and return, repeating this three times. The recorded video is then cut into 6 segments, removing the turning parts and retaining only the straight-line walking segments, thus obtaining 6 gait video segments.
[0090] The gait videos in this application are not limited to the above-mentioned acquisition methods. As long as the acquired gait video is shot from the side, remains horizontal and stable during shooting, and the subject's entire body is always fully presented in the frame, it is a usable gait video. This is because gait videos shot from a side view are more conducive to extracting human body contour features, thereby enabling gait feature analysis based on changes in the human body contour map.
[0091] The gait videos in this application are not limited to the ranges mentioned above, and multiple videos of different durations and distances can be recorded for the same subject.
[0092] Furthermore, the subjects in the gait videos included patients with varying degrees of Parkinson's disease and healthy individuals. Figure 1A In the training process shown, the subjects were referred to as Parkinson's patients, and doctors manually assigned scores to them. Figure 1B In the assessment process shown, the subjects are referred to as the assessees, and the trained model provides scores.
[0093] On the one hand, refer to Figure 2A-2C This application provides a method for establishing a gait impairment assessment model for Parkinson's disease, comprising the following steps:
[0094] S10: Preprocess the gait video to extract the skeleton map sequence and multi-cycle gait energy map, where the gait video is a walking video of a Parkinson's patient taken from the side.
[0095] S20: Using skeleton graph sequences and multi-cycle gait energy maps, combined with gait impairment rating labels obtained by doctors based on gait videos, a neural network is trained to obtain a gait impairment assessment model for Parkinson's disease, where the gait impairment rating labels are used to classify the degree of gait impairment.
[0096] Among them, the skeleton map sequence is mainly derived from the joint features in the gait video, and the multi-period gait energy map is mainly derived from the contour features in the gait video.
[0097] By capturing gait videos from the side, it's possible to extract not only skeletal features based on joints but also body contour features, particularly changes in the body contour during gait. This allows for gait feature analysis based on these contour changes. In contrast, videos captured from the front cannot accurately perceive changes in the body contour, such as stride length, stride speed, arm swing angle, and body tilt angle. These features are much better extracted from side views.
[0098] Therefore, this application extracts skeleton map sequences and multi-cycle gait energy maps based on joints and contours from gait videos taken from the side. It combines these two data streams that represent gait characteristics from different angles for neural network training, thereby obtaining a dual-stream network model that combines contours and joints. Compared with existing technologies, this model has the advantages of convenient and efficient sampling and accurate screening, providing a feasible method for early screening and assessment of Parkinson's disease.
[0099] Further, refer to Figure 2B As shown, step S10, which involves preprocessing the gait video and extracting the multi-cycle gait energy map, specifically includes:
[0100] S101: Extract contour maps from each frame of the gait video, arrange the contour maps in chronological order, and obtain a contour map sequence;
[0101] S102: Determine the gait energy cycle based on the contour map sequence;
[0102] The gait energy cycle is the time interval between adjacent peaks and troughs in the curve of the distance between the two feet over time in the profile sequence. The starting point of the gait energy cycle is the time point corresponding to the peak or trough.
[0103] S103: Divide the contour map sequence into a groups according to the gait energy cycle, and superimpose the contour maps in each group to form a gait energy map, thereby obtaining a gait energy maps.
[0104] S104: Connect the a gait energy maps in chronological order in the horizontal direction to obtain a multi-cycle gait energy map.
[0105] Specifically, refer to Figure 8 As shown, the contour map is a black and white human body contour map, with one contour map corresponding to each frame. All the contour maps extracted in the gait video in chronological order are finally arranged into a contour map sequence (i.e., the contour sequence in the figure).
[0106] Further, refer to Figure 10As shown, a gait energy cycle is one step, which is the time interval between adjacent peaks and troughs. The starting point of a gait energy cycle is the time point corresponding to either a peak or a trough. For example, when the starting point of a gait energy cycle is a trough of 0.0, the gait energy cycle is the time interval between the adjacent peak of 0.5 and the trough of 0.0. By overlaying the contour maps within this gait energy cycle, a gait energy map is obtained, corresponding to... Figure 11 The first of six consecutive human silhouette changes. The overlay method involves superimposing all silhouettes within the gait energy cycle using the midpoint of the head as a reference.
[0107] Furthermore, Figure 11 In this context, a = 6, and b is the number of contour maps within one gait energy cycle. Therefore, Figure 11 The diagram shows six consecutive gait energy maps, which are then stitched together horizontally to obtain a multi-cycle gait energy map.
[0108] Furthermore, the calculation of the superposition of the contour maps within each group to form a gait energy map in step S103 is as follows:
[0109]
[0110] Among them, I c (x,y,n) is the contour map extracted from the nth frame of the gait video, N is the number of video frames contained in one gait energy cycle, x is the horizontal coordinate of the pixel in the contour map, y is the vertical coordinate of the pixel in the contour map, and GEI(x,y) takes the value of pixels.
[0111] Further, refer to Figure 2C Step S10 involves preprocessing the gait video and extracting the skeleton map sequence, specifically including:
[0112] S111: Extract the two-dimensional coordinates of the joints from each frame of the gait video;
[0113] S112: Calculate the skeleton sequence corresponding to the gait video based on the two-dimensional coordinates of the joints and the spatial connection relationship between the joints.
[0114] Specifically, refer to Figure 8 As shown, each frame of the gait video corresponds to a joint diagram, and one gait video corresponds to a set of joint sequences. There are certain spatial connections between the joint sequences. Based on these spatial connections, a skeleton diagram corresponding to each frame of the gait video can be calculated. Arranging the skeleton diagrams in chronological order yields... Figure 8 The skeleton diagram sequence shown.
[0115] Further, the calculation of the skeleton graph sequence corresponding to the gait video according to the two-dimensional coordinates of the joints and the spatial connection relationship in step S112 is specifically as follows:
[0116] J = (V, E),
[0117] Reference Figure 9 As shown, the joint classification provided by an embodiment of the present application has a total of 15 joints, and the serial numbers of the joints are respectively represented as: 0 upper neck, 1 lower neck, 2 left shoulder, 3 left elbow, 4 left wrist, 5 right shoulder, 6 right elbow, 7 right wrist, 8 coccyx, 9 left hip joint, 10 left knee, 11 left ankle, 12 right hip joint, 13 right knee, 14 right ankle.
[0118] Among them, the node set V includes the two-dimensional coordinates of the joints in all frames of the gait video; the edge set E includes two types of spatial connection relationships: proximal and distal; and one type of temporal connection relationship: sequential connection; the proximal includes the left and right shoulders, the spine, the left and right hips, the left and right upper arms, and the left and right thighs, and the distal includes the left and right forearms and the left and right calves; the sequential connection includes the connection between the node of each joint at the current moment and its next moment node;
[0119] In other embodiments, the edge set E may also be selected based on other spatial or temporal conditions.
[0120] In an embodiment of the present application, the spatial connection relationship of the edge set E is represented by an adjacency matrix A in actual operations. The number a at the i-th row and j-th column in the matrix A ij represents the spatial connection relationship between joint i and joint j. If the spatial connection relationship is proximal, then a ij = x. If the spatial connection relationship is distal, then a ij = y, where x and y are positive integers, and 1 < x < y. For example: x = 2, y = 3.
[0121] The sequential connection of the edge set E is realized by one-dimensional convolution of the joints along the time direction, as follows:
[0122]
[0123] Among them, X t is the sequence of the two-dimensional coordinates of a certain joint along the time direction, which is a matrix with t rows and 2 columns. t is the number of frames included in the video, N is the number of convolution kernels for sequential convolution, W k is the k-th sequential convolution kernel, which is a one-dimensional vector, and its length can be freely adjusted as needed. B is a learnable linear bias.
[0124] Further, reference Figure 6The neural network used for training in this application, and the gait impairment assessment model for Parkinson's disease obtained after training, are two-stream neural networks, including skeleton flow, contour flow, vector splicing unit, fully connected layer, and softmax layer.
[0125] The skeleton stream contains multiple ST-GCN units to process the input skeleton map sequence and obtain the skeleton stream output vector. The contour stream contains multiple VGG units to process the input multi-period gait energy map and obtain the contour stream output vector. The skeleton stream output vector and the contour stream output vector are concatenated by the input vector concatenation unit, and then the two streams are fused through a fully connected layer. Finally, the softmax layer is passed to obtain the predicted probability value of each score. The score with the highest predicted probability value is the gait score estimate of the gait video.
[0126] ST-GCN stands for Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition, which is a spatiotemporal graph convolutional network for human action recognition.
[0127] VGG is an image recognition network proposed by the Visual Geometry Group in Oxford in 2014. It uses a deeper network structure and has demonstrated that increasing network depth can affect network performance to some extent.
[0128] Furthermore, the calculation of the ST-GCN unit in this application is as follows:
[0129]
[0130] Among them, X in As input features, X out For output features, A is the adjacency matrix, and D is the adjacency matrix. ii =∑ j (A ij +I ij ) is the degree matrix used to normalize the adjacency matrix A, W is the learnable weight coefficient matrix, and B is the learnable bias coefficient.
[0131] Furthermore, the VGG unit includes a two-dimensional convolutional layer and a max pooling layer.
[0132] Specifically, refer to Figure 8In one embodiment of this application, the skeleton flow includes 10 ST-GCN network units, and the contour flow includes 5 VGG network units. Each VGG unit consists of 2 or 3 two-dimensional convolutional layers and 1 max pooling layer. The number of two-dimensional convolutional layers and the number of channels in the five VGG units are shown in Table 1. All convolutional layers use 3*3 convolutional kernels with a stride of 1.
[0133] Table 1
[0134] VGG block number kernel size Number of channels Number of convolutional layers 1 3 64 2 2 3 128 2 3 3 256 3 4 3 512 3 5 3 512 3
[0135] In one embodiment of this application, the dual-stream neural network achieves the fusion of dual-stream information through vector concatenation and fully connected layers. The length of the skeleton stream output vector is 139264, and the length of the joint stream output vector is 25088. The vector concatenation unit concatenates the two into a long vector of length 164362. Then, through two fully connected layers and one softmax layer, the neural network model obtains the predicted probability values of different preset score categories. The one with the highest probability is the gait score estimate of the patient to which the sample belongs.
[0136] For example, refer to Figure 8 As shown, when the preset score categories include 0, 1, and 2, the predicted probability values of the three scores 0, 1, and 2 will be obtained by softmax respectively, and the one with the highest probability is the final gait score estimate.
[0137] Furthermore, training the neural network in step S20 also includes:
[0138] The loss value is calculated using the loss function, and the loss value is backpropagated. Training stops when the error converges.
[0139] The loss function used is cross-entropy, and the specific calculation is as follows:
[0140]
[0141] Where N is the number of samples, K is the number of categories, and p ic Let represent the probability that the class of the i-th sample is c, i.e.
[0142] Furthermore, y ic ∈{0,1}, when the class of the i-th sample is c, the corresponding y ic =1, otherwise y ic =0.
[0143] By introducing a loss function, error convergence can be accelerated more quickly, thus improving training efficiency.
[0144] On the other hand, reference Figure 3 This application provides a method for assessing gait impairment in Parkinson's disease, comprising the following steps:
[0145] S1: Preprocess the gait video to extract the skeleton map sequence and multi-cycle gait energy map, where the gait video is a walking video of the person to be evaluated taken from the side.
[0146] S2: The Parkinson's disease gait impairment assessment model is obtained by inputting the skeleton diagram sequence and multi-cycle gait energy map into any of the above methods to establish a gait impairment assessment model, and the gait score of the person to be assessed is obtained.
[0147] This method is applied to the established gait impairment assessment model for Parkinson's disease. Based on a unified concept, the specific procedures for extracting the skeleton diagram sequence and multi-cycle gait energy map are as described above and will not be repeated here.
[0148] Furthermore, step S2 includes using a voting mechanism to obtain the gait score optimization result, specifically:
[0149] The gait videos of the same person to be evaluated are grouped, and multiple gait scores are obtained for each group. The score that appears most frequently in the multiple gait scores is selected as the gait score optimization result for the person to be evaluated.
[0150] For example, when six gait videos of the same person being evaluated are input, six rating results are obtained: 1, 1, 1, 2, 0, 0. The most frequent result is 1, so the optimal gait rating for the person being evaluated should be 1.
[0151] Furthermore, when there are multiple scores that appear most frequently, the one with the highest value is selected as the optimized gait score for the person being evaluated.
[0152] For example, when six gait videos of the same person being evaluated are input, six rating results are obtained: 1, 1, 1, 0, 0, 0. The most frequent values are 1 and 0, and 1 is greater than 0. Therefore, the optimized gait rating result for the person being evaluated should be 1.
[0153] Specifically, in one embodiment of this application, an experimental dataset was constructed based on the Tianjin Huanhu Hospital platform, containing 54 PD patients and 26 age-matched healthy individuals as controls. During the evaluation process, accuracy, precision, recall, F1 score, receiver operating characteristic (ROC) curve, and area under the curve (AUC) were used as performance indicators to evaluate the classification results of the model. The specific analysis of the experimental results is as follows:
[0154] 1. Classification Result Analysis
[0155] Five-fold cross-validation was used on the experimental dataset to evaluate the Parkinson's disease gait impairment assessment model obtained by the method proposed in this application. Experimental results show that the method of this application can achieve an overall accuracy of 71.3%. As shown in Table 2, the classification performance of the Parkinson's disease gait impairment assessment model obtained by the method proposed in this application is presented in each category. The accuracy, recall, F1 score, and AUC value were calculated for each score category, and satisfactory accuracy was achieved for each score category. Figure 15 The confusion matrix shown also illustrates this. Figure 14 The ROC curves for each score category are shown in the figure. Among them, the healthy person category (0 points) has the best classification performance, while the performance of 1 points and 2 points is similar. Therefore, the gait impairment assessment model for Parkinson's disease obtained by the method of this application can effectively distinguish Parkinson's patients from healthy people, and has the classification ability for early screening of the disease.
[0156] Table 2
[0157] accuracy Recall rate F1 score AUC 0 0.840 0.808 0.824 0.892 1 0.586 0.654 0.618 0.745 2 0.731 0.679 0.704 0.733
[0158] 2. Comparison with advanced Parkinson's gait assessment methods
[0159] Table 3 compares this application with world-leading work in related fields. The results show that the method proposed in this application achieves better classification performance on a larger sample set, significantly outperforming existing methods and works on this classification task.
[0160] Table 3
[0161]
[0162] Further, refer to Figure 4 After obtaining the gait score, this application also proposes a refined evaluation method to obtain the damage status of each joint by calculating the neural network response value, as follows:
[0163] Following step S2, the following steps are also included:
[0164] S3: Calculate the joint response values of the skeleton diagram sequence in the gait impairment assessment model for Parkinson's disease;
[0165] S4: Classify and calculate the joint response values to obtain the average response value for each body part;
[0166] S5: Based on joint response values and average response values, obtain refined gait assessment results.
[0167] Furthermore, the joint response values are calculated as follows:
[0168]
[0169] Where S is the joint response value vector, N is the number of joints, each value in the joint response value vector represents the response value of the Parkinson's disease gait impairment assessment model at that joint, T is the length of the input skeleton diagram sequence, and C... out O is the number of output channels for the skeleton stream. it X is the output matrix of the skeleton flow. out The elements in.
[0170] Furthermore, the joints are divided into 6 groups, representing 6 body parts: neck (joint 0, joint 1), trunk (joint 1, joint 8), left arm (joints 5-7), right arm (joints 2-4), left leg (joints 12-14), and right leg (joints 9-11). Step S3 involves classifying and calculating the joint response values to obtain the average response value for each body part, specifically including:
[0171] The average response value of each group of joints was calculated separately to obtain the average joint response value for each body part. This further provides a characterization of the damage status of each body part of the person being evaluated.
[0172] Specifically, refer to Figure 12 , Figure 13 In one embodiment of this application, statistical analysis was performed on the response value vectors of 54 PD patients in the experimental dataset to calculate the average response value of each joint and the average response value of each body part. The results are as follows: Figure 12 and 13 As shown.
[0173] Figure 12 The distribution of the average response values for each joint is displayed. The average response value for each joint is a normalized value, specifically obtained by dividing the average response value of each joint by the maximum value. For example, the average response value of joint 0 is 20, which is the maximum value. Therefore, the normalized average response value of joint 0 is 1. Correspondingly, the normalized average response values of other joints are 0.8, 0.6, 0.4, 0.2, etc.
[0174] from Figure 13 The results show that for PD patients, the neural network has higher response values for the neck, trunk, and arm, indicating that the neural network pays more attention to the movement of the patient's upper body. Among them, the three body parts with the highest response values are the neck, trunk, and left arm.
[0175] Using the above methods, after obtaining a score, it is possible to further determine the specific damaged body parts of the patient and the degree of damage to each body part.
[0176] Further, refer to Figure 5 This application also proposes another refined method for quantifying gait impairment, which obtains more granular information on movement impairment through gait quantification parameters, as detailed below:
[0177] S6: Preprocess the gait video and extract the skeleton sequence, where the gait video is a walking video of the person to be evaluated taken from the side;
[0178] S7: Calculate and obtain quantization parameters based on the skeleton diagram sequence;
[0179] S8: Evaluate the motion characteristics of the person being evaluated based on quantitative parameters.
[0180] The method for obtaining the skeleton diagram sequence in step S6 is as described above and will not be repeated here.
[0181] On the one hand, this gait impairment quantification method can be a supplementary quantification assessment method after obtaining gait scores using a model. In this case, it can be executed after step S1, after step S2, or after step S5.
[0182] On the other hand, this gait impairment quantification method can also be used alone.
[0183] Furthermore, the motion characteristics, corresponding quantization parameters, and calculation methods are as follows:
[0184] When the movement characteristic is stride length, the quantification parameter is stride length, which is calculated as the value corresponding to the peak in the curve of the distance between the left ankle joint 11 and the right ankle joint 14 over time.
[0185] When the movement characteristic is walking speed, the quantification parameter is trunk speed, which is calculated as the horizontal speed of the lower neck joint 1.
[0186] When the motion characteristic is arm swing, the quantification parameters include the arm swing angle and the bilateral asymmetry coefficient of the arm swing.
[0187] The arm swing angle is calculated as the maximum angle of the upper arm (including both left and right sides) swinging back and forth.
[0188] The calculation method for the bilateral asymmetry coefficient of arm swing is as follows:
[0189] The larger side has a swing angle of θ max The smaller side is θ min .
[0190] When the movement characteristic is cervical flexion, the quantitative parameter is the cervical flexion angle, which is calculated as the angle between the line connecting the upper cervical joint 0 and the lower cervical joint 1 and the vertical direction.
[0191] When the movement characteristic is trunk flexion, the quantitative parameter is the trunk flexion angle, which is calculated as the angle between the line connecting the lower cervical joint 1 and the coccygeal joint 8 and the vertical direction.
[0192] When the movement characteristic is gait cycle, the quantification parameter is the gait cycle, which is calculated as the time interval between two adjacent peaks or between two adjacent troughs in the curve showing the change in the distance between the left ankle joint (11) and the right ankle joint (14) over time. It is important to note that the gait cycle here differs from the gait energy cycle described earlier; the gait cycle consists of two steps, while the gait energy cycle consists of one step.
[0193] Specifically, refer to Figure 10 As shown, one gait cycle is defined as the time point from 0.5 corresponding to the first peak to 1.75 corresponding to the second peak. Alternatively, one gait cycle is defined as the time point from 0.0 corresponding to the first trough to 1.25 corresponding to the second trough.
[0194] Furthermore, step S7 specifically includes:
[0195] S71: Determine whether the value of the quantization parameter exceeds the preset threshold range. If the result is that it exceeds the threshold range, then it is determined that there is an anomaly in the motion characteristics.
[0196] The values of these physically meaningful quantitative parameters can characterize the state of the corresponding movement characteristics, thus providing an effective reference for fine-grained gait impairment and personalized gait assessment. Based on the MDS-UPDRS gait score, it is possible to further identify which specific movement characteristics are problematic, thereby further determining which body parts are affected.
[0197] Furthermore, after determining that the motion characteristics are abnormal, step S7 also includes the following steps:
[0198] S72: Perform Spearman correlation analysis on multiple quantitative parameters and the gait score of the Parkinson's disease patient to obtain the correlation coefficient, where the gait score is the MDS-UPDRS score corresponding to the gait video;
[0199] S73: Sort the correlation coefficients to obtain the ranking of the degree of abnormality of the motion features.
[0200] Specifically, in one embodiment of this application, the seven quantitative parameters mentioned above were statistically analyzed in 54 PD patients, and the Spearman correlation coefficients between these quantitative values and MDS-UPDRS gait scores were calculated. Let X be a set of quantitative values for the 54 patients, and Y be their set of gait scores. Arranging these two sets in ascending order yields R(X) and R(Y). The Spearman correlation coefficient ρ is calculated as follows:
[0201]
[0202] Where cov is the covariance of the two variables, and σ is the standard deviation of the variables.
[0203] The results of Spearman correlation analysis between the seven quantitative values and gait scores are shown in Table 4 below:
[0204] Table 4
[0205]
[0206] As shown in the table above, the three indicators most correlated with gait scores are trunk speed, neck flexion angle, and maximum arm swing angle. This aligns with the three body parts with the highest response values mentioned earlier, further validating the effectiveness of the neural network response value indicator, i.e., its ability to reflect gait impairment.
[0207] In another aspect, this application provides a computer device, characterized in that the computer device includes a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein when the processor executes the computer program, it implements the above-described method for establishing a gait impairment assessment model for Parkinson's disease, or a method for assessing gait impairment in Parkinson's disease, or a method for quantifying gait impairment in Parkinson's disease.
[0208] In another aspect, this application provides a readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method for establishing a gait impairment assessment model for Parkinson's disease, or a method for assessing gait impairment in Parkinson's disease, or a method for quantifying gait impairment in Parkinson's disease.
[0209] For details, please refer to the following: Figure 16 In practical applications, Figure 16 A schematic diagram of the hardware operating environment involved in the method for establishing a gait impairment assessment model for Parkinson's disease, or the method for assessing gait impairment for Parkinson's disease, or the method for quantifying gait impairment for Parkinson's disease.
[0210] like Figure 16As shown, the hardware operating environment may include: a processor 1001, such as a CPU; a communication bus 1002; a user interface 1003; a network interface 1004; and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or stable non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0211] Those skilled in the art will understand that Figure 16 The hardware structure for operating the method described in this application shown in the figure does not constitute a limitation on the device for operating the method. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0212] like Figure 16 As shown, the memory 1005, which serves as a readable storage medium, may include an operating system, a network communication module, a user interface module, and computer programs. The operating system is a management and control program that supports the operation of the network communication module, the user interface module, the computer programs, and other programs or software. The network communication module manages and controls the network interface 1004, and the user interface module manages and controls the user interface 1003.
[0213] exist Figure 16 In the hardware structure shown, the network interface 1004 is mainly used to connect to the backend server and communicate with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and communicate with the client; the processor 1001 can call the computer program stored in the memory 1005 and execute the steps of the aforementioned method for establishing a gait impairment assessment model for Parkinson's disease, or the method for assessing gait impairment for Parkinson's disease, or the method for quantifying gait impairment for Parkinson's disease.
[0214] In summary, this application has the following beneficial effects:
[0215] By extracting contours and joints from gait videos, two different data streams—gait energy cycle maps and skeleton maps—are generated. These data streams are then combined with neural network training to produce a corresponding neural network model. This neural network model can analyze the gait videos of Parkinson's disease patients to be evaluated and obtain gait scores. Based on these gait scores, the degree of gait impairment in Parkinson's patients can be assessed conveniently and quickly with high accuracy.
[0216] Furthermore, gait videos filmed from the side not only allow for the extraction of skeletal features based on joints, but also for the extraction of body contour features, particularly the gait variations during walking. This allows for gait feature analysis based on changes in the body contour. In contrast, videos filmed from the front cannot accurately perceive changes in the body contour; for example, changes in stride length, stride speed, arm swing angle, and body tilt angle cannot be observed. These features are much better extracted from side views.
[0217] Furthermore, a voting mechanism is used to obtain gait score optimization results, resulting in a more accurate gait score.
[0218] Furthermore, by calculating the network response value, after obtaining the score, the damage status of each joint can be further obtained.
[0219] Furthermore, by classifying joints to correspond to different body parts, after obtaining a score, it is possible to further determine the specific damaged body parts of the patient, as well as the degree of damage to each body part.
[0220] Furthermore, by using gait quantification parameters, we can obtain more granular information about motor impairment. Based on the gait score, we can further identify which specific motor characteristics are problematic, thereby further determining which body parts have issues.
[0221] Furthermore, by performing correlation analysis between multiple quantitative parameters and the gait score of the Parkinson's disease patient to obtain correlation coefficients, and ranking the correlation coefficients, the degree of abnormality of the motor characteristics is obtained, thereby further understanding the relative degree of abnormality of the problematic motor characteristics and thus providing directions that should be focused on.
Claims
1. A method of establishing a model for assessing gait impairment in Parkinson's disease, characterized by, The method comprises the steps of: S10: preprocessing the gait video to extract a skeleton graph sequence and a multi-cycle gait energy graph, wherein the gait video is a walking video of a Parkinson's disease patient taken from the side; S20: training a neural network using the skeleton graph sequence and the multi-cycle gait energy graph in combination with a gait impairment score label diagnosed by a doctor according to the gait video to obtain a Parkinson's disease gait impairment evaluation model, wherein the gait impairment score label is used to divide the degree of gait impairment; The Parkinson's disease gait impairment evaluation model is a double-flow neural network, which comprises a skeleton flow, a contour flow, a vector splicing unit, a fully connected layer, and a softmax layer; The skeleton flow comprises a plurality of ST-GCN units for processing the input skeleton graph sequence to obtain a skeleton flow output vector, and the contour flow comprises a plurality of VGG units for processing the input multi-cycle gait energy graph to obtain a contour flow output vector; the skeleton flow output vector and the contour flow output vector are input into the vector splicing unit for splicing, then the double-flow information is fused through the fully connected layer, and finally the prediction probability value of each score is obtained through the softmax layer, wherein the score with the highest prediction probability value is the estimated value of the gait score of the gait video; The step S10 of preprocessing the gait video to extract a multi-cycle gait energy graph specifically comprises: S101: extracting a contour graph from each frame of the gait video, arranging the contour graphs in time sequence to obtain a contour graph sequence; S102: determining a gait energy cycle according to the contour graph sequence; The gait energy cycle is the time interval between adjacent peak values and valley values in the curve of the distance between the two feet in the contour graph sequence changing with time, and the starting point of the gait energy cycle is the time point corresponding to the peak value or the valley value; S103: dividing the contour graph sequence into a groups according to the gait energy cycle, and superimposing the contour graphs in each group to form a gait energy graph, thereby obtaining a gait energy graph; S104: connecting the a gait energy graphs in the horizontal direction in time sequence to obtain a multi-cycle gait energy graph; The calculation of superimposing the contour graphs in each group in the step S103 to form a gait energy graph is specifically as follows: ， wherein, is a contour map extracted from the nth frame of the gait video, is a number of video frames contained in the gait energy image cycle, x is a horizontal direction coordinate of a pixel point in the contour map, y is a vertical direction coordinate of a pixel point in the contour map, and GEI(x, y) takes a pixel value. The step S10 of preprocessing the gait video to extract a skeleton graph sequence specifically comprises: S111: extracting two-dimensional coordinates of joints from each frame of the gait video; S112: calculating a skeleton graph sequence corresponding to the gait video according to the two-dimensional coordinates of the joints and the spatial connection relationship between the joints.
2. The method of establishing a model for assessing gait impairment in Parkinson's disease according to claim 1, wherein, The calculation of calculating a skeleton graph sequence corresponding to the gait video according to the two-dimensional coordinates of the joints and the spatial connection relationship between the joints in the step S112 is specifically as follows: J = (V, E), wherein the node set V includes two-dimensional coordinates of joints in all frames of the gait video; the edge set E includes two types of spatial connection relationships: proximal and distal; the proximal includes left and right shoulders, spine, left and right hips, left and right upper arms, and left and right upper thighs; and the distal includes left and right lower arms and left and right lower legs. The edge set Spatial connectivity is based on an adjacency matrix. Representation, matrix The first in Line number Number of columns Indicates joint With joints If the spatial connection relationship is proximal, then =x, if the spatial connection is at the far end, then =y, where x and y are positive integers, and 1 <x<y。 3. The method of establishing a model for assessing gait impairment in Parkinson's disease according to claim 2, wherein, The edge set E further comprises a time connection relationship: time connection; the time connection comprises a connection between a current time node of each joint and a next time node thereof; The edge set The timing connection of the joint is achieved by one-dimensional convolution in the time direction as follows: wherein, is a t row 2 column matrix, t is the number of frames contained in the gait video, N is the number of convolution kernels of the time sequence convolution, is a one-dimensional vector, and B is a learnable linear bias.
4. The method of establishing a model for assessing gait impairment in Parkinson's disease according to claim 3, wherein, The calculation of the ST-GCN unit is as follows: ， wherein, is an input feature, is an output feature, is the adjacency matrix, D is the degree matrix, which is a diagonal matrix, and its ith row, ith column element is defined as: is used to normalize the adjacency matrix wherein, is the identity matrix, whose ith row, ith column element is , is a learnable weight coefficient matrix, is a learnable bias coefficient.
5. The method of establishing a model for assessing gait impairment in Parkinson's disease according to claim 1, wherein, The VGG unit comprises a two-dimensional convolution layer and a maximum pooling layer.
6. The method of establishing a model for assessing gait impairment in Parkinson's disease according to claim 1, wherein, The training of the neural network in the step S20 further comprises: loss value is calculated by a loss function, the loss value is back propagated, and the training is stopped when error converges; The loss function adopts cross entropy, and the specific calculation is as follows: ， wherein, is the number of samples, is the number of classes, denotes the class of the th sample is , i.e. ; Further, When the category of the first sample is , the corresponding , otherwise .
7. A method of assessing gait impairment in Parkinson's disease, characterized by, The method comprises the steps of: S1: pre-processing a gait video to extract a skeleton graph sequence and a multi-cycle gait energy graph, wherein the gait video is a walking video of an evaluator taken from the side; S2: inputting the skeleton graph sequence and the multi-cycle gait energy graph into the Parkinson's disease gait impairment evaluation model obtained by the method for establishing a Parkinson's disease gait impairment evaluation model according to any one of claims 1 to 6 to obtain a gait score of the evaluator.
8. The method of assessing gait impairment in Parkinson's disease according to claim 7, wherein, The step S2 comprises obtaining a gait score optimization result by using a voting mechanism, specifically: The gait videos of the same evaluator are grouped, and a plurality of groups of gait scores are obtained respectively, and the score with the highest occurrence frequency is selected from the plurality of groups of gait scores, that is, the gait score optimization result of the evaluator.
9. The method of assessing gait impairment in Parkinson's disease according to claim 7, wherein, After the step S2, the following steps are further included: S3: calculating joint response values of the skeleton graph sequence in the Parkinson's disease gait impairment evaluation model; S4: classifying and calculating the joint response values to obtain average response values of each body part; S5: obtaining a refined gait evaluation result according to the joint response values and the average response values.
10. The method of assessing gait impairment in Parkinson's disease according to claim 9, wherein, The calculation of the joint response values is as follows: ， ， wherein S is a joint response value vector, N is the number of the joints, each value in the joint response value vector represents the response value of the Parkinson's gait impairment evaluation model at the joint, is the length of the input skeleton graph sequence, is the output channel number of the skeleton flow, is the output matrix of the skeleton flow element in the output matrix of the skeleton flow.
11. The method of assessing gait impairment in Parkinson's disease according to claim 10, wherein, The joints are divided into six groups, representing six body parts: neck, torso, left arm, right arm, left leg, and right leg. In step S3, the joint response values are classified and calculated to obtain average response values of each body part, specifically comprising: The average response values of each group of joints are calculated respectively to obtain average joint response values of each body part.
12. A computer device, comprising: The computer device comprises a computer readable storage medium, a processor, and a computer program stored on the computer readable storage medium and executable on the processor, wherein the processor implements the method for establishing a Parkinson's disease gait impairment evaluation model according to any one of claims 1 to 6 or the method for evaluating Parkinson's disease gait impairment according to any one of claims 7 to 11 when executing the program.
13. A readable storage medium, characterized by, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method for establishing a Parkinson's disease gait impairment evaluation model according to any one of claims 1 to 6 or the method for evaluating Parkinson's disease gait impairment according to any one of claims 7 to 11.