Freezing gait assessment method, device and system

Abstract

The present invention provides a frozen gait evaluation method, device and system. The frozen gait evaluation method includes: generating a kinematic feature set, a dynamic feature set and a spatiotemporal parameter feature set of the gait of the object under test; for each feature set, constructing an attribute graph for characterizing the feature set and extracting features from the attribute graph to obtain corresponding graph features and the importance score of each gait feature in the feature set; based on the weight values ​​corresponding to each feature set, fusing and extracting features from the graph features corresponding to each feature set to obtain global features; inputting the global features into a prediction module to obtain the frozen gait severity score of the object under test; for each gait feature, determining the gait score of the gait feature based on the normalized feature value, the importance score and the weight value corresponding to the feature set of the gait feature, and generating a radar chart for characterizing the gait pattern of the object under test based on the gait scores of all gait features.

Description

Technical Field

[0001] The present disclosure generally relates to the technical field of multimodal data processing, and more specifically, to a method, device, and system for evaluating freezing gait. Background Art

[0002] Freezing of Gait (FOG) is mainly manifested as an inability to move forward suddenly during walking, feeling that the feet are "sucked" to the ground, unable to lift or step forward freely, and difficult to move. This symptom usually appears in the middle and late stages of Parkinson's Disease (PD), which will increase the risk of falls for patients and further reduce their mobility.

[0003] With the progress of computer technology, the application of Instrumented Gait Analysis (IGA) technology in the task of evaluating freezing gait is becoming more and more extensive. However, in the prior art, the evaluation of freezing gait is usually based on single-modal data, and the accuracy needs to be improved. Summary of the Invention

[0004] An exemplary embodiment of the present disclosure is to provide a method, device, and system for evaluating freezing gait, which can quickly and accurately evaluate the severity of freezing gait and can provide an interpretable gait pattern analysis result.

[0005] According to an embodiment of the present disclosure, a method for evaluating freezing gait is provided, including: obtaining lower limb joint motion data and plantar pressure data of a subject under multiple gait cycles during walking; generating a kinematic feature set, a dynamic feature set, and a spatio-temporal parameter feature set of the subject's gait based on the obtained lower limb joint motion data and plantar pressure data; respectively constructing an attribute graph for characterizing each feature set in the kinematic feature set, the dynamic feature set, and the spatio-temporal parameter feature set, and using a graph attention module to extract features from the attribute graph to obtain graph features corresponding to the feature set and importance scores of each gait feature in the feature set; based on the weight values corresponding to each feature set, performing fusion processing on the graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatio-temporal parameter feature set to obtain fused features, and performing feature extraction on the fused features to obtain global features; inputting the global features into a prediction module to obtain a freezing gait severity score of the subject; for each gait feature in the kinematic feature set, the dynamic feature set, and the spatio-temporal parameter feature set, determining a gait score of the gait feature based on the normalized feature value of the gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set where the gait feature is located, and generating a radar chart for characterizing the gait pattern of the subject based on the gait scores of all gait features.

[0006] Optionally, the steps of generating a kinematic feature set, a dynamic feature set, and a spatio-temporal parameter feature set of the gait of the subject based on the acquired lower limb joint movement data and plantar pressure data include: constructing a hip-knee joint angle cycle diagram and a knee-ankle joint angle cycle diagram based on the acquired lower limb joint movement data, and obtaining the kinematic feature set of the gait of the subject by extracting the following features of the hip-knee joint angle cycle diagram and the knee-ankle joint angle cycle diagram: area, number of harmonic square waves, average value of coupling angle, variability of coupling angle, mean absolute relative phase, and deviation phase; generating a plantar pressure curve and a centroid trajectory of plantar pressure based on the acquired plantar pressure data, and obtaining the dynamic feature set of the gait of the subject by extracting multiple features of the plantar pressure curve and the centroid trajectory of plantar pressure, where the multiple features include: maximum value of the vertical ground reaction force in the early support phase, maximum value of the vertical ground reaction force in the late support phase, minimum value of the vertical ground reaction force in the mid-support phase, growth rate of the vertical ground reaction force in the early support phase, decline rate of the vertical ground reaction force in the late support phase, average value, coefficient of variation, and asymmetry index of the intersection points of the forward centroid trajectory and the lateral centroid trajectory in different gait cycles; generating a spatio-temporal parameter feature set of the gait of the subject based on the acquired lower limb joint movement data, where the spatio-temporal parameter feature set includes: swing phase time, stance phase time, single-limb support time, double-limb support time, step frequency, walking ratio, stance phase ratio, swing phase ratio, single-limb support phase ratio, double-limb support phase ratio, walking speed, step length, stride, and velocity.

[0007] Optionally, the steps of determining the gait score of each gait feature based on the normalized eigenvalue of each gait feature, the importance score of this gait feature, and the weight value corresponding to the feature set where this gait feature is located include: taking the product of the normalized eigenvalue of each gait feature, the importance score of this gait feature, and the weight value corresponding to the feature set where this gait feature is located as the gait score of this gait feature.

[0008] Optionally, each graph attention module after the graph attention module for feature extraction of the attribute graph of the kinematic feature set, the graph attention module for feature extraction of the attribute graph of the dynamic feature set, and the graph attention module for feature extraction of the attribute graph of the spatio-temporal parameter feature set includes: a graph attention neural network composed of q graph attention network layers and a pooling layer; where the input of the l th graph attention network layer in the q graph attention network layers is: , l is 1 and q is an integer, represents the output of the l -1st graph attention network layer, represents the adjacency matrix of the attribute graph; wherein, the input of the first graph attention network layer in the q graph attention network layers is the node feature matrix of the attribute graph; wherein, the input of the pooling layer is: , and the output of the pooling layer is the graph feature.

[0009] Optionally, the step of fusing the graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatio-temporal parameter feature set based on the weight values corresponding to each feature set to obtain the fused features includes: mapping each graph feature to a common subspace by maximizing the correlation between the projection matrices of different graph features; splicing after assigning corresponding weight values to the representations of each graph feature in the common subspace to obtain the fused features; wherein, the weight value corresponding to the representation of the graph feature in the common subspace is: the weight value corresponding to the feature set to which the graph feature belongs; the weight values corresponding to each feature set are obtained by learning from multiple training samples.

[0010] Optionally, the step of extracting features from the fused features to obtain global features includes: using p encoders to extract features from the fused features to obtain global features; wherein, the output of the l th encoder is: , , represents the multi-head self-attention function, represents the normalization function, represents the output of the feed-forward network layer in the l th encoder; wherein, is the fused feature; wherein, the output of the pth encoder is the global feature.

[0011] Optionally, the prediction module includes: a first prediction module and a second prediction module; wherein, the step of inputting the global feature into the prediction module to obtain the frozen gait severity score of the object to be measured includes: inputting the global feature into the first prediction module to obtain a prediction result for characterizing whether the object to be measured has a frozen gait; if the first prediction result characterizes that the object to be measured has a frozen gait, then inputting the global feature into the second prediction module to obtain a score for characterizing the severity of the frozen gait of the object to be measured; if the first prediction result characterizes that the object to be measured does not have a frozen gait, then determining that the frozen gait severity score of the object to be measured is 0.

[0012] According to an embodiment of the present disclosure, a freezing gait assessment device is provided, including: a data acquisition unit configured to acquire lower limb joint motion data and plantar pressure data of a subject under multiple gait cycles during walking; a gait feature extraction unit configured to generate a kinematic feature set, a dynamic feature set, and a spatio-temporal parameter feature set of the subject's gait based on the acquired lower limb joint motion data and plantar pressure data; a graph construction and representation learning unit configured to respectively construct an attribute graph for characterizing each of the kinematic feature set, the dynamic feature set, and the spatio-temporal parameter feature set, and use a graph attention module to extract features from the attribute graph to obtain graph features corresponding to the feature set and importance scores of each gait feature in the feature set; a feature fusion and representation learning unit configured to perform fusion processing on the graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatio-temporal parameter feature set based on the weight values corresponding to each feature set to obtain fused features, and perform feature extraction on the fused features to obtain global features; a prediction unit configured to input the global features into a prediction module to obtain a freezing gait severity score of the subject; and a gait pattern analysis unit configured to determine a gait score of each gait feature based on the normalized feature value of the gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set where the gait feature is located in the kinematic feature set, the dynamic feature set, and the spatio-temporal parameter feature set, and generate a radar chart for characterizing the gait pattern of the subject based on the gait scores of all gait features.

[0013] According to an embodiment of the present disclosure, a freezing gait assessment system is provided, including: a lower limb motion capture device and a data processing device; wherein, the lower limb motion capture device includes: a binocular camera for capturing lower limb joint motion data of a subject under multiple gait cycles during walking; a plantar pressure acquisition device for acquiring plantar pressure data of the subject under the multiple gait cycles; wherein, the data processing device includes: at least one processor; at least one memory storing computer-executable instructions, wherein when the computer-executable instructions are run by the at least one processor, the at least one processor is caused to execute the freezing gait assessment method as described above.

[0014] Optionally, the lower limb motion capture device further includes: a plurality of reflective marker balls for wearing on the hip joint, knee joint, ankle joint, heel, and toe tip of the subject; an infrared light source array for emitting infrared light to the plurality of reflective marker balls; wherein, the binocular camera is a near-infrared binocular camera for capturing the infrared light reflected by the plurality of reflective marker balls under the multiple gait cycles.

[0015] The freezing gait assessment method, device, and system according to an exemplary embodiment of the present disclosure use multi-modal data for freezing gait assessment, improving the accuracy of the task of assessing the severity of freezing gait. In addition, it can also provide an interpretable gait pattern analysis result of the subject to intuitively understand which gait characteristics of the subject are severely affected by the freezing gait symptoms of the subject.

[0016] In the following description, some aspects and / or advantages of the general concept of the present disclosure will be set forth, and some aspects and / or advantages will be learned from the following description or the implementation of the general concept of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] These and / or other aspects and advantages of the present application will become clearer and easier to understand from the following detailed description of the embodiments of the present application in conjunction with the accompanying drawings, in which:

[0018] Figure 1 A flowchart showing a freezing gait assessment method according to an exemplary embodiment of the present disclosure;

[0019] Figure 2 An example of a lower limb motion capture device according to an exemplary embodiment of the present disclosure;

[0020] Figure 3 A schematic diagram of joint angles on the sagittal plane of a human body in a scenario of wearing a reflective marker ball according to an exemplary embodiment of the present disclosure;

[0021] Figure 4 An example of steps of a freezing gait assessment method according to an exemplary embodiment of the present disclosure;

[0022] Figure 5 An example of a radar chart for characterizing the gait pattern of a subject according to an exemplary embodiment of the present disclosure;

[0023] Figure 6 A structural block diagram showing a freezing gait assessment device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] Reference will now be made in detail to the embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings, wherein the same reference numerals always refer to the same components. The following embodiments will be described with reference to the accompanying drawings to explain the present disclosure.

[0025] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of the present disclosure are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described here can be implemented in an order other than those illustrated or described here. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0026] It should be noted here that "at least one of several items" in the present disclosure all represents the inclusion of three parallel situations: "any one of the several items", "a combination of any multiple of the several items", and "the whole of the several items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example, "performing at least one of step one and step two" means the following three parallel situations: (1) performing step one; (2) performing step two; (3) performing step one and step two.

[0027] Figure 1 The flowchart showing the freezing gait assessment method according to an exemplary embodiment of the present disclosure.

[0028] Referring to Figure 1 , in step S100, lower limb joint movement data and plantar pressure data of the object to be measured during walking are acquired under multiple gait cycles.

[0029] As an exemplary embodiment, the lower limb joints may include but are not limited to: hip joint, knee joint, ankle joint.

[0030] As an exemplary embodiment, the lower limb joint movement data and plantar pressure data of the object to be measured collected by a lower limb motion capture device may be acquired. The lower limb motion capture device may include a binocular camera and a plantar pressure acquisition device.

[0031] As an example, as Figure 2 shown, the binocular camera is specifically a near-infrared binocular camera, and the plantar pressure acquisition device is specifically a plantar pressure insole. In addition, the lower limb motion capture device may further include: a plurality of reflective marker balls and an infrared light source array. The plurality of reflective marker balls are used to be worn at the hip joint, knee joint, ankle joint, heel and toe tip of the object to be measured; the infrared light source array is used to emit infrared light to the plurality of reflective marker balls; the near-infrared binocular camera is used to capture the infrared light reflected by the plurality of reflective marker balls under multiple gait cycles, that is, to capture the movement data of the plurality of reflective marker balls worn on the lower limbs of the object to be measured as the lower limb joint movement data of the object to be measured.

[0032] As an example, the infrared light source array may specifically include two circular infrared light source arrays of 850 nm, and the infrared light source array may be arranged around the near-infrared binocular camera. As an example, the brightness of the infrared light source array can be manually adjusted according to the actual environment, so as to ensure that the near-infrared binocular camera can collect the lower limb joint movement data of the object to be measured. As an example, the near-infrared binocular camera can collect the lower limb joint movement data of the object to be measured at a speed of 60 frames per second and a resolution of 720p, ensuring the continuity and smoothness of the collected lower limb joint movement data.

[0033] As an example, the plantar pressure insole may be provided with a plurality of (for example, 99) independent sensing areas. For example, a sensing area layout of 15 rows and 7 columns can be adopted. The output resistance of the sensing area will change correspondingly with the change of the external pressure. Therefore, the plantar pressure insole can measure the pressure distribution of each area of the sole in real time.

[0034] As an example, first, wear a plurality of reflective marker balls and plantar pressure insoles on the object to be measured; then, require the object to be measured to stand still in a T posture for 1 second, and the data collected during this process can be used for sensor calibration; then, require the object to be measured to walk along a path of a certain length (for example, 20 meters long) at a comfortable speed of their own choice, so that the near-infrared binocular camera can capture the lower limb joint movement data of the object to be measured under multiple gait cycles during walking, and the plantar pressure insole can collect the plantar pressure data of the object to be measured under these multiple gait cycles.

[0035] In addition, the lower limb motion capture device may further include: an RGB camera, which is used to shoot a color video of the movement process of the object to be measured.

[0036] In step S200, based on the obtained lower limb joint movement data and plantar pressure data, a kinematic feature set, a dynamic feature set, and a spatio-temporal parameter feature set of the gait of the object to be measured are generated.

[0037] As an exemplary embodiment, step S200 may include: constructing a hip-knee joint angle cycle diagram and a knee-ankle joint angle cycle diagram based on the obtained lower limb joint movement data, and obtaining a kinematic feature set of the gait of the object to be measured by extracting a plurality of features of the hip-knee joint angle cycle diagram and the knee-ankle joint angle cycle diagram. That is, the kinematic feature set of the gait of the object to be measured includes: a plurality of features of the hip-knee joint angle cycle diagram and the knee-ankle joint angle cycle diagram.

[0038] As an example, Figure 3The black circles therein represent the wearing positions of multiple reflective marker balls. 1 represents the hip joint, 2 represents the knee joint, 3 represents the ankle joint, 4 represents the heel, and 5 represents the toe tip. As an example, first, based on the lower limb joint movement data of the object to be measured (i.e., the movement data of multiple reflective marker balls), the joint angle data of the hip, knee, and ankle in the sagittal plane within different gait cycles can be calculated ( , , ); then, the hip-knee joint angle cycle diagram can be formed by combining the hip joint angle data and the knee joint angle data, and the knee-ankle joint angle cycle diagram can be formed by combining the knee joint angle data and the ankle joint angle data; next, non-linear techniques can be used to calculate the area (Area), the number of harmonic square waves (num of harmonics), the average value of the coupling angle (average value of coupling angle, MCA), the variability of the coupling angle (coupling angle variability, CAV), the mean absolute relative phase (mean absolute relative phase, MARP), and the deviation phase (deviation phase, DP), etc. of each cycle diagram, and add them to the kinematic feature set.

[0039] As an exemplary embodiment, step S200 may include: generating a plantar pressure curve and a plantar pressure centroid trajectory based on the acquired plantar pressure data, and obtaining the dynamic feature set of the gait of the object to be measured by extracting multiple features of the plantar pressure curve and the plantar pressure centroid trajectory. That is, the dynamic feature set of the gait of the object to be measured includes: multiple features of the plantar pressure curve and the plantar pressure centroid trajectory.

[0040] As an example, the plantar pressure curve may be a curve of the sum of the pressure values corresponding to all pressure sensing areas on the sole (i.e., the total pressure value) changing with time. As an example, the following gait features can be extracted from the plantar pressure curve: GRF1, GRF2, GRF3, Bn, and En, etc., and added to the dynamic feature set. GRF1 and GRF3 respectively represent the maximum values of the vertical ground reaction force (vGRF) in the early support stage and the late support stage. GRF2 reflects the minimum value of the vGRF in the mid-support stage. Bn represents the growth rate of the vGRF in the early support stage, while En represents the decline rate of the vGRF in the late support stage. In addition, the average value (Mean), the coefficient of variation (CV), and the asymmetry index (USI), etc. of these gait features can be further calculated and added to the dynamic feature set.

[0041] As an example, the plantar pressure centroid trajectory may specifically include the anterior-posterior (AP direction) centroid trajectory and the medial-lateral (ML direction) centroid trajectory. As an example, the following gait features may be extracted from the anterior-posterior centroid trajectory and the medial-lateral centroid trajectory: the mean, coefficient of variation (CV), and asymmetry index (USI) of the intersection points of the anterior-posterior centroid trajectory and the medial-lateral centroid trajectory in different gait cycles, etc., and added to the kinetic feature set.

[0042] As an exemplary embodiment, step S200 may include generating a spatio-temporal parameter feature set of the gait of the subject to be measured based on the acquired lower limb joint movement data.

[0043] As an exemplary embodiment, the spatio-temporal parameter feature set may include: swing phase time (s); stance phase time (s); single-limb support time (s); double-limb support time (s); step frequency (step / min); walking ratio; stance phase percentage (%); swing phase percentage (%); single-limb support phase percentage (%); double-limb support phase percentage (%); walking speed (m / s); step length (m); stride (m); speed (m / s), etc.

[0044] According to the exemplary embodiments of the present disclosure, these multimodal gait feature sets with rich physical meanings not only provide an interpretable data source for the task of quantitatively evaluating the severity of frozen gait based on the graph-based multimodal fusion framework, but also help to improve the evaluation accuracy.

[0045] In step S300, for each of the kinematic feature set, the kinetic feature set, and the spatio-temporal parameter feature set, an attribute graph for characterizing the feature set is constructed, and the graph attention module is used to extract features from the attribute graph to obtain the graph features corresponding to the feature set and the importance scores (e.g., attention scores) of each gait feature in the feature set.

[0046] It should be understood that each subject to be measured corresponds to a kinematic feature set, a kinetic feature set, and a spatio-temporal parameter feature set respectively, each kinematic feature set corresponds to an attribute graph respectively, each kinetic feature set corresponds to an attribute graph respectively, and each spatio-temporal parameter feature set corresponds to an attribute graph respectively.

[0047] As an example, all gait feature vectors of the i-th subject to be measured , represents the total number of gait features, where represents the kinematic feature set of the gait of the i-th subject to be measured, with a total of gait features; represents the kinetic feature set of the gait of the i-th subject to be measured, with a total of gait features; Denote the spatio-temporal parameter feature set of the gait of the i-th measured object, with a total of gait features.

[0048] As Figure 4 shown, the kinematic feature set of the gaits of N measured objects is denoted as , the dynamic feature set of the gaits of N measured objects is denoted as , and the spatio-temporal parameter feature set of the gaits of N measured objects is denoted as .

[0049] As Figure 4 shown in (a), as an example, each feature set of the gait of the i-th measured object can be represented using a graph structure, and the feature attribute graph corresponding to the i-th measured object serves as the graphical representation of the i-th measured object. For the attribute graph of each feature set of the gait of the i-th measured object (i.e., the feature attribute graph), the nodes in this attribute graph correspond one-to-one with the gait features in this feature set, and the edges represent the correlations between these gait features. Specifically: The attribute graph of the kinematic feature set of the gait of the i-th measured object , represents the vertex set, represents the node feature matrix, represents the edge set, and the matrix representation of the edge set is the adjacency matrix ; The attribute graph of the dynamic feature set of the gait of the i-th measured object , represents the vertex set, represents the node feature matrix, represents the edge set, and the matrix representation of the edge set is the adjacency matrix ; The attribute graph of the spatio-temporal parameter feature set of the gait of the i-th measured object , represents the vertex set, represents the node feature matrix, represents the edge set, and the matrix representation of the edge set is the adjacency matrix .

[0050] As an example, the adjacency matrix can be constructed using a fully connected method, which can maximize the potential of information flow and capture global characteristics. In addition, as another example, the construction method of the attribute graph can be obtained by learning from multiple training samples. For example, an edge is established between any two nodes only when the correlation between them satisfies a learned specific condition (e.g., greater than a specific threshold).

[0051] As an exemplary embodiment, the graph attention module for feature extraction of the attribute graph of the m-th type of feature set in the kinematic feature set, dynamic feature set, and spatio-temporal parameter feature set includes: a graph attention neural network composed of q (for example, q can be 3) graph attention network layers and a pooling layer, which is used to learn features from graph data, such as Figure 4 shown in (a). Specifically, when m = 1, the m-th type of feature set represents the kinematic feature set; when m = 2, the m-th type of feature set represents the dynamic feature set; when m = 3, the m-th type of feature set represents the spatio-temporal parameter feature set.

[0052] As an example, the input of the l -th graph attention network layer is: , l where is 1 and is an integer of q, represents the output of the l -1-th graph attention network layer, and represents the adjacency matrix of the attribute graph of the m-th type of feature set. The input of the first graph attention network layer is the node feature matrix of the attribute graph of the m-th type of feature set. The input of the pooling layer is: , and the output of the pooling layer is the graph feature corresponding to the m-th type of feature set. As an example, represents a graph attention network layer (for example, with a hidden dimension of 64).

[0053] According to the exemplary embodiment of the present disclosure, the graph attention neural network effectively captures the complex relationships between nodes and their neighbors through its self-attention mechanism, and at the same time assigns importance scores to each node in different modalities, thereby improving the interpretability of the graph-based multi-modal fusion framework. Subsequently, the pooling layer generates a compact and representative graph representation by averaging the node features.

[0054] In step S400, based on the weight values corresponding to each feature set, the graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatio-temporal parameter feature set are fused to obtain the fused features, and the fused features are subjected to feature extraction to obtain the global features.

[0055] As an exemplary embodiment, the graph features can be mapped to a common subspace by maximizing the correlation between the projection matrices of different graph features; then, by assigning corresponding weight values to the representations of each graph feature in the common subspace and splicing them, the fused features are obtained; where the weight value corresponding to the representation of the graph feature in the common subspace is: the weight value corresponding to the feature set to which the graph feature belongs.

[0056] As an example, through generalized canonical correlation analysis, each graph feature can be mapped to a common subspace. In multi-modal data processing, different modalities often differ in dimension and distribution. Direct feature fusion may lead to information loss or poor alignment. Generalized canonical correlation analysis solves this problem by mapping each graph feature to a low-dimensional shared subspace. Generalized canonical correlation analysis not only enhances the interaction information between different modalities but also effectively reduces redundant information, thus providing a more consistent and compact feature representation.

[0057] Specifically, the graph features corresponding to the kinematic feature sets of the gaits of N measured objects can be labeled as , where represents the feature dimension of the kinematic feature set; the graph features corresponding to the dynamic feature sets of the gaits of N measured objects can be labeled as , where represents the feature dimension of the dynamic feature set; the graph features corresponding to the spatio-temporal parameter feature sets of the gaits of N measured objects can be labeled as , where represents the feature dimension of the spatio-temporal parameter feature set.

[0058] As an example, after being projected into the shared subspace through the generalized canonical correlation analysis algorithm, can be represented in the shared subspace as , , is 's projection matrix; can be represented in the shared subspace as , , is 's projection matrix; can be represented in the shared subspace as , , is 's projection matrix; is the dimension of the shared subspace. For example, it can be set to {2, 3, 4, 5, 6, 7, 8, 9}.

[0059] Generalized canonical correlation analysis calculates the projection matrix of each modality by maximizing the correlation between the projected features of different modalities. The formula is as follows:

[0060]

[0061] Where represents the trace function, , .

[0062] The importance of different modalities for downstream tasks may vary. Therefore, during the fusion process, corresponding weights are introduced respectively (i.e., the weight value corresponding to the kinematic feature set), (i.e., the weight value corresponding to the dynamic feature set), (i.e., the weight value corresponding to the spatio-temporal parameter feature set), so that the differences between modalities can be handled more flexibly and the discriminative ability of the fused features can be enhanced. The fused feature Z can be expressed as:

[0063]

[0064] where are learnable weight parameters, and the sum of the three is 1, which can be obtained by learning from multiple training samples. The fused feature representation not only fuses the effective information of each modality, removes redundant information, but also reflects the importance weights of each modality.

[0065] According to an exemplary embodiment of the present disclosure, the Generalized Canonical Correlation Analysis (GCCA) algorithm is combined with a weighted fusion layer for aligning and fusing multi-modal features, as shown in Figure 4 (b).

[0066] As an exemplary embodiment, p (for example, p can be 3) encoders (such as transformer encoders) can be used to perform deep feature extraction on the fused features, so as to enhance the ability to capture global information.

[0067] As an example, the output l of the th encoder is: , l where is 1 and is an integer of p, , represents the multi-head self-attention function, represents the normalization function, represents the output of the feed-forward network layer in the l th encoder; the input of the first encoder is the fused feature Z , , representing the feature dimension of each measured object; the output of the pth encoder is the global feature. represents the feed-forward network layer.

[0068] As shown in Figure 4As shown in (b), each Transformer encoder may include: multi-head self-attention (MHSA), a feed-forward neural network (FFN), and a normalization layer. Each Transformer encoder first uses the multi-head self-attention mechanism to calculate the global dependencies between features: Next, a feed-forward neural network is used to obtain the final output: .

[0069] In step S500, the global features are input into the prediction module to obtain the frozen gait severity score of the object under test.

[0070] As an exemplary embodiment, the prediction module includes: a first prediction module and a second prediction module; step S500 may include: inputting the global features into the first prediction module (i.e., the first fence module) to obtain a prediction result for characterizing whether the object under test has a frozen gait; if the first prediction result characterizes that the object under test has a frozen gait, then input the global features into the second prediction module (i.e., the second fence module) to obtain a frozen gait severity score for characterizing the object under test; if the first prediction result characterizes that the object under test does not have a frozen gait, then determine that the frozen gait severity score of the object under test is 0.

[0071] In fact, the prediction module is used to perform two tasks, including identifying whether the object under test has a frozen gait and evaluating the frozen gait severity score of the object under test. The former is a binary classification task, and the latter is a regression task. The present disclosure takes into account that there are a large number of samples in the sample database with a frozen gait severity score of 0, and this zero-inflation problem will have a negative impact on the performance of the graph-based multi-modal fusion framework in accurately evaluating the severity of frozen gait. To address the zero-inflation problem, the present disclosure designs a dual fence module, as Figure 4 shown in (c). By decomposing the complex task into two simpler sub-tasks, the zero-inflation problem is solved, thereby improving the performance of the graph-based multi-modal fusion framework. As an example, the types of the first prediction module and the second prediction module may include but are not limited to one of the following: Adaptive Boosting (Adboost), Random Forest (RF), Support Vector Machine (SVM), and Extreme Gradient Boosting (Xgboost).

[0072] In step S600, for each gait feature in the kinematic feature set, the kinetic feature set, and the spatio-temporal parameter feature set, based on the normalized feature value of the gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set where the gait feature is located, determine the gait score of the gait feature, and based on the gait scores of all gait features, generate a radar chart for characterizing the gait pattern of the object under test.

[0073] As an exemplary embodiment, the product of the normalized feature value of each gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set to which the gait feature belongs can be used as the gait score of the gait feature.

[0074] About normalizing the characteristic values ​​of gait characteristics to obtain normalized characteristic values, the normalized characteristic values , among which, represents the eigenvalue before normalization, represents the upper limit of the characteristic value of the gait feature, Indicates the lower limit of the characteristic value of the gait feature.

[0075] It should be understood that at least one radar chart may be generated, for example, multiple radar charts may be generated corresponding to multiple feature sets one by one. Figure 5 An example of a radar chart for characterizing a gait pattern of a measured object according to an exemplary embodiment of the present disclosure is shown, Figure 5 The 10 axes in correspond to the 10 gait features one by one.

[0076] By comparing the radar chart used to characterize the gait pattern of the subject under test with the standard radar chart (a radar chart used to characterize the normal gait pattern), the higher the similarity between the two, the more normal the gait pattern of the subject under test; and, based on the difference between the two, it is possible to intuitively know which gait features of the subject under test are seriously affected by the freezing gait symptoms of the subject under test, thereby realizing the visualization of the gait pattern analysis results of the subject under test.

[0077] According to an exemplary embodiment of the present disclosure, in a graph-based multimodal fusion framework, the graph attention neural network can provide the importance score of each gait feature relative to its modality, and the feature fusion module can provide the weight value corresponding to each modality. On this basis, the gait score of each gait feature can be calculated to obtain a radar chart for characterizing the gait pattern of the subject under test. In this way, it is possible to intuitively determine which gait features of the subject under test are most seriously affected by the freezing gait symptom, and this information can provide an effective reference for other subsequent operations (e.g., management, supervision, and care of the subject under test).

[0078] According to the exemplary embodiments of the present disclosure, various parameters in the graph-based multimodal fusion framework (e.g., parameters of the graph construction method, parameters of various graph attention modules, parameters of various encoders, weight parameters used in fusion, parameters of the prediction module, etc.) can be obtained through training based on training samples, and each training sample may include: corresponding kinematic feature sets, dynamic feature sets, spatiotemporal parameter feature sets, and label vectors.

[0079] As an example, the label vector of the i-th training sample can be expressed as , For the The freezing gait severity score of a training sample is a continuous variable with a value range of [0, 28]; is the classification variable of the -th training sample, and its value is , where 1 indicates that the

[0080] -th training sample has a freezing gait.

[0081] Figure 6 FIG. shows a structural block diagram of a freezing gait assessment device according to an exemplary embodiment of the present disclosure.

[0082] As Figure 6 shown, the freezing gait assessment device according to an exemplary embodiment of the present disclosure includes: a data acquisition unit 101, a gait feature extraction unit 102, a graph construction and representation learning unit 103, a feature fusion and representation learning unit 104, a prediction unit 105, and a gait pattern analysis unit 106.

[0083] Specifically, the data acquisition unit 101 is configured to acquire lower limb joint motion data and plantar pressure data of the object to be measured during multiple gait cycles while walking.

[0084] The gait feature extraction unit 102 is configured to generate a kinematic feature set, a kinetic feature set, and a spatio-temporal parameter feature set of the gait of the object to be measured based on the acquired lower limb joint motion data and plantar pressure data.

[0085] The graph construction and representation learning unit 103 is configured to construct an attribute graph for characterizing each feature set in the kinematic feature set, the kinetic feature set, and the spatio-temporal parameter feature set, and use a graph attention module to extract features from the attribute graph to obtain graph features corresponding to the feature set and importance scores of each gait feature in the feature set.

[0086] The feature fusion and representation learning unit 104 is configured to perform fusion processing on the graph features corresponding to the kinematic feature set, the graph features corresponding to the kinetic feature set, and the graph features corresponding to the spatio-temporal parameter feature set based on the weight values corresponding to each feature set to obtain fused features, and perform feature extraction on the fused features to obtain global features.

[0087] The prediction unit 105 is configured to input the global features into a prediction module to obtain the freezing gait severity score of the object to be measured.

[0088] The gait pattern analysis unit 106 is configured to determine the gait score of each gait feature in the kinematic feature set, the kinetic feature set, and the spatio-temporal parameter feature set based on the normalized eigenvalue of the gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set where the gait feature is located, and generate a radar chart for characterizing the gait pattern of the subject under test based on the gait scores of all gait features.

[0089] As an exemplary embodiment, the gait feature extraction unit 102 may be configured to: construct a hip-knee joint angle cycle diagram and a knee-ankle joint angle cycle diagram based on the acquired lower limb joint movement data, and obtain the kinematic feature set of the gait of the subject under test by extracting the following features of the hip-knee joint angle cycle diagram and the knee-ankle joint angle cycle diagram: area, number of harmonic square waves, average value of the coupling angle, variability of the coupling angle, mean absolute relative phase, and deviation phase; generate a plantar pressure curve and a plantar pressure centroid trajectory based on the acquired plantar pressure data, and obtain the kinetic feature set of the gait of the subject under test by extracting a plurality of features of the plantar pressure curve and the plantar pressure centroid trajectory, where the plurality of features include: the maximum value of the vertical ground reaction force in the early support phase, the maximum value of the vertical ground reaction force in the late support phase, the minimum value of the vertical ground reaction force in the mid-support phase, the growth rate of the vertical ground reaction force in the early support phase, the decline rate of the vertical ground reaction force in the late support phase, the average value, coefficient of variation, and asymmetry index of the intersection points of the forward centroid trajectory and the lateral centroid trajectory in different gait cycles; generate the spatio-temporal parameter feature set of the gait of the subject under test based on the acquired lower limb joint movement data, where the spatio-temporal parameter feature set includes: swing phase time, stance phase time, single-limb support time, double-limb support time, step frequency, walking ratio, stance phase ratio, swing phase ratio, single-limb support phase ratio, double-limb support phase ratio, walking speed, step length, stride, speed.

[0090] As an exemplary embodiment, the gait pattern analysis unit 106 may be configured to: take the product of the normalized eigenvalue of each gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set where the gait feature is located as the gait score of the gait feature.

[0091] As an exemplary embodiment, each graph attention module after the graph attention module for feature extraction of the attribute graph of the kinematic feature set, the graph attention module for feature extraction of the attribute graph of the kinetic feature set, and the graph attention module for feature extraction of the attribute graph of the spatio-temporal parameter feature set includes: a graph attention neural network composed of q graph attention network layers and a pooling layer; where the input of the l th graph attention network layer in the q graph attention network layers is: , l For 1 and q is an integer, indicates the first l -1 graph attention network layer output, represents the adjacency matrix of the attribute graph; wherein the input of the first graph attention network layer among the q graph attention network layers is the node feature matrix of the attribute graph; the input of the pooling layer is: , the output of the pooling layer is the graph feature.

[0092] As an exemplary embodiment, the feature fusion and representation learning unit 104 can be configured to: map each graph feature to a common subspace by maximizing the correlation between the projection matrices of different graph features; obtain fused features by assigning corresponding weight values ​​to the representations of each graph feature in the common subspace and then splicing them; wherein the weight value corresponding to the representation of the graph feature in the common subspace is: the weight value corresponding to the feature set corresponding to the graph feature; the weight value corresponding to each feature set is obtained by learning multiple training samples.

[0093] As an exemplary embodiment, the feature fusion and representation learning unit 104 may be configured to: use p encoders to extract features from the fused features to obtain global features; wherein the first l Output of encoders is: , , represents the multi-head self-attention function, represents the normalization function, indicates the first l The output of the feedforward network layer in the encoder; where is the fused feature; the output of the pth encoder is the global feature.

[0094] As an exemplary embodiment, the prediction module includes: a first prediction module and a second prediction module; wherein the prediction unit 105 can be configured to: input the global feature into the first prediction module to obtain a prediction result for characterizing whether the subject under test has a freezing gait; if the first prediction result indicates that the subject under test has a freezing gait, then input the global feature into the second prediction module to obtain a freezing gait severity score for characterizing the subject under test; if the first prediction result indicates that the subject under test has not a freezing gait, then determine that the freezing gait severity score of the subject under test is 0.

[0095] It should be understood that the specific processing performed by the freezing gait evaluation device according to the exemplary embodiment of the present disclosure has been referred to Figures 1 - 5A detailed description has been given, and relevant details will not be elaborated here.

[0096] It should be understood that each unit and module in the freezing gait assessment device according to the exemplary embodiments of the present disclosure can be implemented as a hardware component and / or a software component.

[0097] According to an exemplary embodiment of the present disclosure, there is also provided a freezing gait assessment system, including the lower limb motion capture device and the data processing device as described in the above exemplary embodiment.

[0098] Specifically, the lower limb motion capture device includes: a binocular camera for capturing lower limb joint motion data of the subject during multiple gait cycles while walking; a plantar pressure acquisition device for acquiring plantar pressure data of the subject during the multiple gait cycles. In addition, the lower limb motion capture device may further include: a plurality of reflective marker balls for being worn at the hip joint, knee joint, ankle joint, heel, and toe tip of the subject; an infrared light source array for emitting infrared light to the plurality of reflective marker balls; wherein, the binocular camera can be a near-infrared binocular camera for capturing the infrared light reflected by the plurality of reflective marker balls during the multiple gait cycles.

[0099] The data processing device includes: at least one memory and at least one processor, and a set of computer-executable instructions is stored in the at least one memory. When the set of computer-executable instructions is executed by the at least one processor, the freezing gait assessment method as described in the above exemplary embodiment is executed.

[0100] As an example, the data processing device can be a PC computer, a tablet device, a personal digital assistant, a smart phone, or other devices capable of executing the above set of instructions. Here, the data processing device does not have to be a single electronic device, and can also be an aggregate of any devices or circuits capable of executing the above instructions (or instruction sets) alone or jointly. The data processing device can also be a part of an integrated control system or a system manager, or can be configured as a portable electronic device that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0101] In the data processing device, the processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. As an example and not a limitation, the processor may further include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0102] The processor can run the instructions or code stored in the memory. Among them, the memory 600 can also store data. The instructions and data can also be sent and received via a network interface device through a network, where the network interface device can adopt any known transmission protocol.

[0103] The memory can be integrated with the processor. For example, RAM or flash memory can be arranged within an integrated circuit microprocessor or the like. In addition, the memory can include separate devices, such as external disk drives, storage arrays, or other storage devices that can be used by any database system. The memory and the processor can be operatively coupled or can communicate with each other, for example, via I / O ports, network connections, etc., such that the processor can read files stored in the memory.

[0104] In addition, the data processing device can also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.). All components of the data processing device can be connected to each other via a bus and / or a network.

[0105] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions can also be provided, wherein when the instructions are run by at least one processor, the at least one processor is caused to execute the freeze gait assessment method as described in the above exemplary embodiment. Examples of the computer-readable storage medium herein include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc memory, hard disk drive (HDD), solid state drive (SSD), cartridge memory (such as a multimedia card, a secure digital (SD) card, or an extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, and any other device that is configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and provide the computer program and any associated data, data files, and data structures to a processor or a computer such that the processor or the computer can execute the computer program. The computer program in the above computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, proxy devices, servers, etc. In addition, in one example, the computer program and any associated data, data files, and data structures are distributed on a networked computer system such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner by one or more processors or computers.

[0106] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided, and instructions in the computer program product may be executed by at least one processor to complete the freezing gait assessment method as described in the above exemplary embodiment.

[0107] Other embodiments of the present disclosure will be readily apparent to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and examples are only to be considered exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0108] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes may be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A freezing gait assessment method, characterized in that: include: Acquire lower limb joint motion data and plantar pressure data of the subject during multiple gait cycles during walking; Based on the acquired lower limb joint motion data and plantar pressure data, a kinematic feature set, a dynamic feature set, and a spatiotemporal parameter feature set of the gait of the subject are generated; For each feature set in the kinematic feature set, dynamic feature set, and spatiotemporal parameter feature set, an attribute graph is constructed to characterize the feature set, and a graph attention module is used to extract features from the attribute graph to obtain the graph features corresponding to the feature set and the importance score of each gait feature in the feature set; Based on the weight values ​​corresponding to each feature set, the graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatiotemporal parameter feature set are fused to obtain fused features, and feature extraction is performed on the fused features to obtain global features; The global features are input into the prediction module to obtain the freezing gait severity score of the tested subject; For each gait feature in the kinematic feature set, the dynamic feature set, and the spatiotemporal parameter feature set, the gait score of the gait feature is determined based on the normalized feature value of the gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set in which the gait feature is located. Based on the gait scores of all gait features, a radar chart for characterizing the gait pattern of the subject is generated.

2. The method according to claim 1, characterized in that Based on the acquired lower limb joint motion data and plantar pressure data, the steps of generating a kinematic feature set, a dynamic feature set, and a spatiotemporal parameter feature set of the gait of the measured object include: Based on the acquired lower limb joint motion data, hip and knee joint angle cycle diagrams and knee and ankle joint angle cycle diagrams are constructed, and the following features of the hip and knee joint angle cycle diagrams and knee and ankle joint angle cycle diagrams are extracted: area, harmonic square wave number, average value of coupling angle, variability of coupling angle, average absolute relative phase and deviation phase, so as to obtain the kinematic feature set of the gait of the subject; Generate a plantar pressure curve and a plantar pressure centroid trajectory based on the acquired plantar pressure data, and obtain a set of dynamic characteristics of the gait of the measured object by extracting multiple features of the plantar pressure curve and the plantar pressure centroid trajectory, wherein the multiple features include: the maximum value of the vertical ground reaction force in the early support stage, the maximum value of the vertical ground reaction force in the late support stage, the minimum value of the vertical ground reaction force in the middle support stage, the growth rate of the vertical ground reaction force in the early support stage, the decrease rate of the vertical ground reaction force in the late support stage, the average value, the coefficient of variation and the asymmetry index of the intersection of the forward centroid trajectory and the lateral centroid trajectory in different gait cycles; Based on the acquired lower limb joint motion data, a spatiotemporal parameter feature set of the gait of the subject is generated, wherein the spatiotemporal parameter feature set includes: swing phase time, stance phase time, single-limb support time, double-limb support time, step frequency, walking ratio, stance phase proportion, swing phase proportion, single-limb support phase proportion, double-limb support phase proportion, step speed, step length, stride, and speed.

3. The method according to claim 1, characterized in that The step of determining a gait score of each gait feature based on the normalized feature value of each gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set to which the gait feature belongs comprises: The product of the normalized feature value of each gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set to which the gait feature belongs is taken as the gait score of the gait feature.

4. The method according to claim 1, characterized in that: Each of the graph attention modules after the graph attention module for extracting features from the attribute graph of the kinematic feature set, the graph attention module for extracting features from the attribute graph of the dynamic feature set, and the graph attention module for extracting features from the attribute graph of the spatiotemporal parameter feature set comprises: a graph attention neural network consisting of q graph attention network layers and a pooling layer; Among them, the first l The input of the graph attention network layer for: , l for 1 and An integer of q, Indicates l -1 output of the graph attention network layer, Represents the adjacency matrix of the attribute graph; Among them, the input of the first graph attention network layer among the q graph attention network layers is is the node feature matrix of the attribute graph; Among them, the input of the pooling layer is: , the output of the pooling layer is the graph feature.

5. The method according to claim 1, characterized in that Based on the weight values ​​corresponding to the feature sets, the step of fusing the graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatiotemporal parameter feature set to obtain fused features includes: By maximizing the correlation between the projection matrices of different graph features, each graph feature is mapped to a common subspace; By assigning corresponding weight values ​​to the representations of each graph feature in the common subspace and then concatenating them, the fused features are obtained; Among them, the weight value corresponding to the representation of the graph feature in the common subspace is: the weight value corresponding to the feature set corresponding to the graph feature; the weight value corresponding to each feature set is obtained by learning multiple training samples.

6. The method according to claim 1, characterized in that The step of extracting features from the fused features to obtain global features includes: extracting features from the fused features using p encoders to obtain global features; Among them, l The output of the encoder for: , , represents the multi-head self-attention function, represents the normalization function, Indicates l The output of the feed-forward network layer in the encoder; in, is the post-fusion feature; Among them, the output of the p-th encoder is the global feature.

7. The method according to claim 1, characterized in that The prediction module includes: a first prediction module and a second prediction module; The step of inputting the global features into the prediction module to obtain the freezing gait severity score of the subject includes: Inputting the global feature into the first prediction module to obtain a prediction result for characterizing whether the subject has a freezing gait; If the first prediction result indicates that the subject has a freezing gait, the global feature is input into a second prediction module to obtain a freezing gait severity score for indicating the subject; If the first prediction result indicates that the subject does not have a freezing gait, then the freezing gait severity score of the subject is determined to be 0.

8. A freezing gait assessment device, characterized in that: include: A data acquisition unit is configured to acquire lower limb joint motion data and plantar pressure data of the subject during multiple gait cycles during walking; A gait feature extraction unit is configured to generate a kinematic feature set, a dynamic feature set, and a spatiotemporal parameter feature set of the gait of the subject under test based on the acquired lower limb joint motion data and plantar pressure data; A graph construction and representation learning unit is configured to construct an attribute graph for representing each feature set in the kinematic feature set, the dynamic feature set, and the spatiotemporal parameter feature set, respectively, and use a graph attention module to perform feature extraction on the attribute graph to obtain a graph feature corresponding to the feature set and an importance score of each gait feature in the feature set; The feature fusion and representation learning unit is configured to fuse the graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatiotemporal parameter feature set based on the weight values ​​corresponding to each feature set to obtain fused features, and perform feature extraction on the fused features to obtain global features; A prediction unit, configured to input the global features into the prediction module to obtain a freezing gait severity score of the subject; The gait pattern analysis unit is configured to determine the gait score of each gait feature in the kinematic feature set, the dynamic feature set and the spatiotemporal parameter feature set based on the normalized feature value of the gait feature, the importance score of the gait feature and the weight value corresponding to the feature set in which the gait feature is located, and generate a radar chart for characterizing the gait pattern of the object under test based on the gait scores of all gait features.

9. A freezing gait assessment system, characterized in that: include: Lower limb motion capture device and data processing device; Wherein, the lower limb motion capture device comprises: A binocular camera is used to capture the lower limb joint motion data of the subject during multiple gait cycles during walking; A plantar pressure collection device, used to collect plantar pressure data of the subject under test during the multiple gait cycles; Wherein, the data processing device comprises: at least one processor; at least one memory storing computer executable instructions, Wherein, when the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to perform the freezing gait assessment method according to any one of claims 1 to 7.

10. The system according to claim 9, characterized in that The lower limb motion capture device also includes: Multiple reflective marker balls for wearing on the hip, knee, ankle, heel and toes of the subject; An infrared light source array, used for emitting infrared light to the plurality of reflective marker balls; The binocular camera is a near-infrared binocular camera, which is used to capture infrared light reflected by the multiple reflective marker balls during the multiple gait cycles.

Images