Freezing gait assessment method, device and system

Abstract

The invention provides a freezing gait assessment method, device and system. The frozen gait evaluation method comprises the following steps: generating a kinematics feature set, a dynamics feature set and a space-time parameter feature set of gaits of a detected object; for each feature set, constructing an attribute graph for representing the feature set, and performing feature extraction on the attribute graph to obtain a corresponding graph feature and an importance score of each gait feature in the feature set; based on the weight value corresponding to each feature set, carrying out fusion and feature extraction on the graph features corresponding to each feature set to obtain global features; inputting the global features into a prediction module to obtain a frozen gait severity score of the detected object; and for each gait feature, determining the gait score of the gait feature based on the normalized feature value of the gait feature, the importance score and the weight value corresponding to the feature set where the gait feature is located, and generating a radar map for representing the gait mode of the detected object based on the gait scores of all the gait features.

Description

Technical Field

[0001] The present disclosure generally relates to the technical field of multimodal data processing, and more specifically, to a freezing gait assessment method, device and system. Background Art

[0002] Freezing of Gait (FOG) is mainly manifested by the sudden inability to move forward during walking, feeling that the feet are "sucked" to the ground, unable to lift or step freely, and difficult to move. This symptom usually occurs in the middle and late stages of Parkinson's Disease (PD), which increases the risk of falling and further reduces the patient's ability to move.

[0003] With the advancement of computer technology, instrumented gait analysis (IGA) technology is increasingly used in freezing gait assessment tasks. However, in existing technologies, freezing gait assessment is usually performed based only on single-modal data, and the accuracy needs to be improved. Summary of the invention

[0004] An exemplary embodiment of the present disclosure is to provide a freezing gait assessment method, device and system, which can quickly and accurately assess the severity of freezing gait and provide interpretable gait pattern analysis results.

[0005] According to an embodiment of the present disclosure, a method for evaluating frozen gait is provided, comprising: obtaining lower limb joint motion data and plantar pressure data of a subject under test in multiple gait cycles during walking; generating a kinematic feature set, a dynamic feature set, and a spatiotemporal parameter feature set of the subject's gait based on the obtained lower limb joint motion data and plantar pressure data; constructing an attribute graph for characterizing each feature set in the kinematic feature set, the dynamic feature set, and the spatiotemporal parameter feature set, respectively, and performing feature extraction on the attribute graph using a graph attention module to obtain graph features corresponding to the feature set and an importance score of each gait feature in the feature set; and evaluating the gait based on the weight values ​​corresponding to the respective feature sets. The graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatiotemporal parameter feature set are fused to obtain fused features, and the fused features are extracted to obtain global features; the global features are input into the prediction module to obtain the freezing gait severity score of the object under test; for each gait feature in the kinematic feature set, the dynamic feature set, and the spatiotemporal parameter feature set, the gait score of the gait feature is determined based on the normalized feature value of the gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set of the gait feature, and based on the gait scores of all gait features, a radar chart for characterizing the gait pattern of the object under test is generated.

[0006] Optionally, the steps of generating a kinematic feature set, a dynamic feature set, and a spatiotemporal parameter feature set of the gait of the object under test based on the acquired lower limb joint motion data and plantar pressure data include: constructing a hip-knee joint angle cycle diagram and a knee-ankle joint angle cycle diagram based on the acquired lower limb joint motion data, and obtaining a kinematic feature set of the gait of the object under test by extracting the following features of the hip-knee joint angle cycle diagram and the knee-ankle joint angle cycle diagram: area, harmonic square wave number, average value of coupling angle, variability of coupling angle, average absolute relative phase, and deviation phase; generating a plantar pressure curve and a plantar pressure centroid trajectory based on the acquired plantar pressure data, and obtaining a dynamic feature set of the gait of the object under test by extracting multiple features of the plantar pressure curve and the plantar pressure centroid trajectory, wherein the multiple features include: The maximum value of the vertical ground reaction force in the early support stage, the maximum value of the vertical ground reaction force in the late support stage, the minimum value of the vertical ground reaction force in the middle support stage, the growth rate of the vertical ground reaction force in the early support stage, the decrease rate of the vertical ground reaction force in the late support stage, the average value, coefficient of variation and asymmetry index of the intersection of the forward center of mass trajectory and the lateral center of mass trajectory in different gait cycles; based on the acquired lower limb joint motion data, the spatiotemporal parameter feature set of the gait of the subject is generated, wherein the spatiotemporal parameter feature set includes: swing phase time, stance phase time, single-limb support time, double-limb support time, step frequency, walking ratio, stance phase proportion, swing phase proportion, single-limb support phase proportion, double-limb support phase proportion, step speed, step length, stride, and speed.

[0007] Optionally, based on the normalized feature value of each gait feature, the importance score of the gait feature and the weight value corresponding to the feature set in which the gait feature is located, the step of determining the gait score of the gait feature includes: taking the product of the normalized feature value of each gait feature, the importance score of the gait feature and the weight value corresponding to the feature set in which the gait feature is located as the gait score of the gait feature.

[0008] Optionally, each of the graph attention modules after the graph attention module for extracting features from the attribute graph of the kinematic feature set, the graph attention module for extracting features from the attribute graph of the dynamic feature set, and the graph attention module for extracting features from the attribute graph of the spatiotemporal parameter feature set comprises: a graph attention neural network consisting of q graph attention network layers and a pooling layer; wherein the first of the q graph attention network layers l The input of the graph attention network layer for: , l for 1 and An integer of q, Indicates l -1 output of the graph attention network layer, Represents the adjacency matrix of the attribute graph; wherein the input of the first graph attention network layer in the q graph attention network layers is is the node feature matrix of the attribute graph; the input of the pooling layer is: , the output of the pooling layer is the graph feature.

[0009] Optionally, based on the weight values ​​corresponding to each feature set, the step of fusing the graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatiotemporal parameter feature set to obtain fused features includes: mapping each graph feature to a common subspace by maximizing the correlation between the projection matrices of different graph features; obtaining the fused features by assigning corresponding weight values ​​to the representations of each graph feature in the common subspace and then splicing them; wherein the weight value corresponding to the representation of the graph feature in the common subspace is: the weight value corresponding to the feature set corresponding to the graph feature; the weight values ​​corresponding to each feature set are obtained by learning from multiple training samples.

[0010] Optionally, the step of extracting features from the fused features to obtain global features comprises: extracting features from the fused features using p encoders to obtain global features; wherein the first l The output of the encoder for: , , represents the multi-head self-attention function, represents the normalization function, Indicates l The output of the feedforward network layer in the encoder; where is the fused feature; the output of the p-th encoder is the global feature.

[0011] Optionally, the prediction module includes: a first prediction module and a second prediction module; wherein, the step of inputting the global feature into the prediction module to obtain the freezing gait severity score of the object under test includes: inputting the global feature into the first prediction module to obtain a prediction result used to characterize whether the object under test has a freezing gait; if the first prediction result characterizes that the object under test has a freezing gait, inputting the global feature into the second prediction module to obtain a freezing gait severity score used to characterize the object under test; if the first prediction result characterizes that the object under test has not had a freezing gait, determining that the freezing gait severity score of the object under test is 0.

[0012] According to an embodiment of the present disclosure, a freezing gait assessment device is provided, comprising: a data acquisition unit, configured to acquire lower limb joint motion data and plantar pressure data of a subject under test in multiple gait cycles during walking; a gait feature extraction unit, configured to generate a kinematic feature set, a dynamic feature set, and a spatiotemporal parameter feature set of the subject's gait based on the acquired lower limb joint motion data and plantar pressure data; a graph construction and representation learning unit, configured to construct an attribute graph for representing each feature set in the kinematic feature set, the dynamic feature set, and the spatiotemporal parameter feature set, respectively, and perform feature extraction on the attribute graph using a graph attention module to obtain graph features corresponding to the feature set and an importance score of each gait feature in the feature set; a feature fusion and representation learning unit, configured to Based on the weight values ​​corresponding to each feature set, the graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatiotemporal parameter feature set are fused to obtain fused features, and the fused features are extracted to obtain global features; the prediction unit is configured to input the global features into the prediction module to obtain the freezing gait severity score of the object under test; the gait pattern analysis unit is configured to determine the gait score of each gait feature in the kinematic feature set, the dynamic feature set, and the spatiotemporal parameter feature set based on the normalized feature value of the gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set of the gait feature, and generate a radar chart for characterizing the gait pattern of the object under test based on the gait scores of all gait features.

[0013] According to an embodiment of the present disclosure, a freezing gait assessment system is provided, comprising: a lower limb motion capture device and a data processing device; wherein the lower limb motion capture device comprises: a binocular camera for capturing lower limb joint motion data of a subject under test in multiple gait cycles during walking; a plantar pressure acquisition device for acquiring plantar pressure data of the subject under test in the multiple gait cycles; wherein the data processing device comprises: at least one processor; at least one memory for storing computer executable instructions, wherein the computer executable instructions, when executed by the at least one processor, prompt the at least one processor to execute the freezing gait assessment method as described above.

[0014] Optionally, the lower limb motion capture device also includes: a plurality of reflective marker balls, which are used to be worn on the hip joints, knee joints, ankle joints, heels and toes of the subject; an infrared light source array, which is used to emit infrared light to the plurality of reflective marker balls; wherein the binocular camera is a near-infrared binocular camera, which is used to capture the infrared light reflected by the plurality of reflective marker balls during the plurality of gait cycles.

[0015] According to the freezing gait assessment method, device and system of the exemplary embodiments of the present disclosure, multimodal data is used to perform freezing gait assessment, which improves the accuracy of the task of assessing the severity of freezing gait. In addition, it can also provide interpretable gait pattern analysis results of the subject under test, so as to intuitively know which gait characteristics of the subject under test are seriously affected by the freezing gait symptoms of the subject under test.

[0016] In the following description, some aspects and / or advantages of the general inventive concept will be set forth, and some other aspects and / or advantages will be known through the following description or the implementation of the general inventive concept. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] These and / or other aspects and advantages of the present application will become clearer and easier to understand from the following detailed description of the embodiments of the present application in conjunction with the accompanying drawings, in which: Figure 1 A flowchart showing a freezing gait assessment method according to an exemplary embodiment of the present disclosure; Figure 2 An example of a lower limb motion capture device according to an exemplary embodiment of the present disclosure is shown; Figure 3 A schematic diagram showing joint angles on the sagittal plane of a human body in a scenario of wearing a reflective marker ball according to an exemplary embodiment of the present disclosure; Figure 4 An example of steps of a freezing gait assessment method according to an exemplary embodiment of the present disclosure is shown; Figure 5 An example of a radar chart for characterizing a gait pattern of a measured subject according to an exemplary embodiment of the present disclosure is shown; Figure 6 A structural block diagram of a freezing gait assessment device according to an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0018] Reference will now be made in detail to the embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like parts throughout. The embodiments will be described below with reference to the drawings in order to explain the present disclosure.

[0019] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0020] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three types of parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three types of parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0021] Figure 1 A flow chart of a freezing gait assessment method according to an exemplary embodiment of the present disclosure is shown.

[0022] Reference Figure 1 In step S100, the lower limb joint motion data and plantar pressure data of the subject under test in multiple gait cycles during walking are obtained.

[0023] As an exemplary embodiment, the lower limb joints may include but are not limited to: hip joints, knee joints, and ankle joints.

[0024] As an exemplary embodiment, lower limb joint motion data and plantar pressure data of the subject collected by a lower limb motion capture device can be obtained. The lower limb motion capture device can include a binocular camera and a plantar pressure collection device.

[0025] As an example, Figure 2 As shown, the binocular camera is specifically a near-infrared binocular camera, and the plantar pressure collection device is specifically a plantar pressure insole. In addition, the lower limb motion capture device may further include: multiple reflective marker balls and an infrared light source array. The multiple reflective marker balls are used to be worn on the hip joints, knee joints, ankle joints, heels and toes of the object being measured; the infrared light source array is used to emit infrared light to the multiple reflective marker balls; the near-infrared binocular camera is used to capture the infrared light reflected by the multiple reflective marker balls in multiple gait cycles, that is, to capture the motion data of the multiple reflective marker balls worn on the lower limbs of the object being measured as the lower limb joint motion data of the object being measured.

[0026] As an example, the infrared light source array may specifically include two 850nm circular infrared light source arrays, and the infrared light source array may be arranged around the near-infrared binocular camera. As an example, the brightness of the infrared light source array may be manually adjusted according to the actual environment, so as to ensure that the near-infrared binocular camera can collect the lower limb joint motion data of the subject under test. As an example, the near-infrared binocular camera may collect the lower limb joint motion data of the subject under test at a speed of 60 frames per second and a resolution of 720p, thereby ensuring the continuity and fluency of the collected lower limb joint motion data.

[0027] As an example, a plantar pressure insole may be provided with a plurality of (e.g., 99) independent sensing areas, for example, a 15-row, 7-column sensing area layout may be adopted. The output resistance of the sensing area will change accordingly with the change of the external pressure, so the plantar pressure insole can measure the pressure distribution of each area of ​​the sole of the foot in real time.

[0028] As an example, first, the subject is given a number of reflective marker balls and plantar pressure insoles; then, the subject is required to stand still in a T-position for 1 second, and the data collected during this process can be used to calibrate the sensor; then, the subject is required to walk along a trail of a certain length (for example, 20 meters long) at a self-selected comfortable speed, so that the near-infrared binocular camera can capture the lower limb joint motion data of the subject in multiple gait cycles during walking, and the plantar pressure insoles can collect the plantar pressure data of the subject in these multiple gait cycles.

[0029] In addition, the lower limb motion capture device may also include: an RGB camera, which is used to capture color video of the motion process of the object being measured.

[0030] In step S200, a kinematic feature set, a dynamic feature set, and a spatiotemporal parameter feature set of the gait of the subject are generated based on the acquired lower limb joint motion data and plantar pressure data.

[0031] As an exemplary embodiment, step S200 may include: constructing a hip-knee joint angle cycle diagram and a knee-ankle joint angle cycle diagram based on the acquired lower limb joint motion data, and obtaining a kinematic feature set of the gait of the subject being measured by extracting multiple features of the hip-knee joint angle cycle diagram and the knee-ankle joint angle cycle diagram. That is, the kinematic feature set of the gait of the subject being measured includes: multiple features of the hip-knee joint angle cycle diagram and the knee-ankle joint angle cycle diagram.

[0032] As an example, Figure 3The black circles in the figure represent the wearing positions of multiple reflective marker balls, 1 represents the hip joint, 2 represents the knee joint, 3 represents the ankle joint, 4 represents the heel, and 5 represents the toe. As an example, the joint angle data of the hip, knee, and ankle in the sagittal plane in different gait cycles can be calculated based on the lower limb joint motion data of the subject (i.e., the motion data of multiple reflective marker balls). , , ); then the hip joint angle data and the knee joint angle data are combined into a hip-knee joint angle cycle diagram, and the knee joint angle data and the ankle joint angle data are combined into a knee-ankle joint angle cycle diagram; next, the nonlinear technology can be used to calculate the area (Area), number of harmonics (num of harmonics), average value of coupling angle (MCA), coupling angle variability (CAV), mean absolute relative phase (MARP) and deviation phase (DP) of each cycle diagram and add them to the kinematic feature set.

[0033] As an exemplary embodiment, step S200 may include: generating a plantar pressure curve and a plantar pressure centroid trajectory based on the acquired plantar pressure data, and obtaining a dynamic feature set of the gait of the object under test by extracting multiple features of the plantar pressure curve and the plantar pressure centroid trajectory. That is, the dynamic feature set of the gait of the object under test includes: multiple features of the plantar pressure curve and the plantar pressure centroid trajectory.

[0034] As an example, the plantar pressure curve may be a curve showing the sum of the pressure values ​​corresponding to all pressure sensing areas of the plantar (i.e., the total pressure value) over time. As an example, the following gait features may be extracted from the plantar pressure curve: GRF1, GRF2, GRF3, Bn, and En, etc., and added to the dynamic feature set. GRF1 and GRF3 represent the maximum values ​​of the vertical ground reaction force (vGRF) in the early support stage and the late support stage, respectively. GRF2 reflects the minimum value of the vGRF in the mid-support stage. Bn represents the growth rate of the vGRF in the early support stage, while En represents the decrease rate of the vGRF in the late support stage. In addition, the mean (Mean), coefficient of variation (CV), and asymmetry index (USI) of these gait features may be further calculated and added to the dynamic feature set.

[0035] As an example, the plantar pressure centroid trajectory may specifically include a forward (AP direction) centroid trajectory and a lateral (ML direction) centroid trajectory. As an example, the following gait features may be extracted from the forward centroid trajectory and the lateral centroid trajectory: the mean value (Mean), coefficient of variation (CV), and unsymmetry index (USI) of the intersection of the forward centroid trajectory and the lateral centroid trajectory in different gait cycles, etc., and added to the dynamic feature set.

[0036] As an exemplary embodiment, step S200 may include generating a spatiotemporal parameter feature set of the gait of the subject based on the acquired lower limb joint motion data.

[0037] As an exemplary embodiment, the spatiotemporal parameter feature set may include: swing phase time (s); stance phase time (s); single-limb support time (s); double-limb support time (s); step frequency (step / min); walking ratio; stance phase proportion (%); swing phase proportion (%); single-limb support phase proportion (%); double-limb support phase proportion (%); walking speed (m / s); step length (m); stride (m); speed (m / s), etc.

[0038] According to an exemplary embodiment of the present disclosure, these multimodal gait feature sets with rich physical meanings not only provide an interpretable data source for the task of quantitatively assessing the severity of freezing gait in a graph-based multimodal fusion framework, but also help to improve the assessment accuracy.

[0039] In step S300, for each feature set in the kinematic feature set, the dynamic feature set, and the spatiotemporal parameter feature set, an attribute graph for characterizing the feature set is constructed, and a graph attention module is used to extract features from the attribute graph to obtain graph features corresponding to the feature set and an importance score (e.g., attention score) for each gait feature in the feature set.

[0040] It should be understood that each measured object corresponds to a kinematic feature set, a dynamic feature set, and a spatiotemporal parameter feature set, each kinematic feature set corresponds to an attribute graph, each dynamic feature set corresponds to an attribute graph, and each spatiotemporal parameter feature set corresponds to an attribute graph.

[0041] As an example, all gait feature vectors of the i-th subject , represents the total number of gait features, where Represents the kinematic feature set of the gait of the i-th measured object, with a total of gait characteristics; Represents the dynamic feature set of the gait of the i-th measured object, with a total of gait characteristics; Represents the spatiotemporal parameter feature set of the gait of the i-th measured object, with a total of gait characteristics.

[0042] like Figure 4 As shown, the kinematic feature set of the gait of N measured subjects is recorded as , the dynamic feature set of the gait of N measured subjects is recorded as , the spatiotemporal parameter feature set of the gait of N measured subjects is recorded as .

[0043] like Figure 4 As shown in (a), as an example, each feature set of the gait of the i-th measured object can be represented by a graph structure, and the feature attribute graph corresponding to the i-th measured object is , as the graph representation of the i-th measured object. For the attribute graph of each feature set of the i-th measured object's gait (i.e., feature attribute graph), the nodes in the attribute graph correspond one-to-one to the gait features in the feature set, and the edges represent the correlation between these gait features. Specifically: the attribute graph of the kinematic feature set of the i-th measured object's gait , represents a vertex set, represents the node feature matrix, Represents an edge set, and the matrix of the edge set is represented as an adjacency matrix ; Attribute graph of the dynamic feature set of the gait of the i-th measured object , represents a vertex set, represents the node feature matrix, Represents an edge set, and the matrix of the edge set is represented as an adjacency matrix ; Attribute graph of the empty parameter feature set of the gait of the i-th measured object , represents a vertex set, represents the node feature matrix, Represents an edge set, and the matrix of the edge set is represented as an adjacency matrix .

[0044] As an example, a fully connected method can be used to construct an adjacency matrix, which can maximize the potential of information flow and capture global characteristics. In addition, as another example, the construction method of the attribute graph can be obtained by learning multiple training samples, for example, when the correlation between any two nodes meets a specific learned condition (for example, greater than a specific threshold), an edge is established between the two nodes.

[0045] As an exemplary embodiment, a graph attention module for extracting features from an attribute graph of an m-th feature set in a kinematic feature set, a dynamic feature set, and a spatiotemporal parameter feature set includes: a graph attention neural network consisting of q (for example, q may be 3) graph attention network layers and a pooling layer for learning features from graph data, such as Figure 4 Specifically, when m is 1, the mth feature set represents the kinematic feature set; when m is 2, the mth feature set represents the dynamic feature set; when m is 3, the mth feature set represents the spatiotemporal parameter feature set.

[0046] As an example, l The input of the graph attention network layer for: , l for 1 and An integer of q, Indicates l -1 output of the graph attention network layer, The adjacency matrix of the attribute graph representing the mth feature set. The input of the first graph attention network layer is the node feature matrix of the attribute graph of the mth feature set. The input of the pooling layer is: , the output of the pooling layer is the graph feature corresponding to the mth feature set. As an example, Represents a graph attention network layer (e.g., with hidden dimension 64).

[0047] According to an exemplary embodiment of the present disclosure, the graph attention neural network effectively captures the complex relationship between a node and its neighbors through its self-attention mechanism, while assigning importance scores to each node in different modalities, thereby improving the interpretability of the graph-based multimodal fusion framework. Subsequently, the pooling layer generates a compact and representative graph representation by averaging the node features.

[0048] In step S400, based on the weight values ​​corresponding to each feature set, the graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatiotemporal parameter feature set are fused to obtain fused features, and feature extraction is performed on the fused features to obtain global features.

[0049] As an exemplary embodiment, each graph feature can be mapped to a common subspace by maximizing the correlation between the projection matrices of different graph features; then, the representations of each graph feature in the common subspace are assigned corresponding weight values ​​and then spliced ​​to obtain a fused feature; wherein the weight value corresponding to the representation of the graph feature in the common subspace is: the weight value corresponding to the feature set corresponding to the graph feature.

[0050] As an example, generalized canonical correlation analysis can be used to map each graph feature to a common subspace. In multimodal data processing, different modalities often differ in dimension and distribution. Direct feature fusion may lead to information loss or poor alignment. Generalized canonical correlation analysis solves this problem by mapping each graph feature to a low-dimensional shared subspace. Generalized canonical correlation analysis not only enhances the interactive information between different modalities, but also effectively reduces redundant information, thereby providing a more consistent and compact feature representation.

[0051] Specifically, the graph features corresponding to the kinematic feature set of the gait of N measured objects can be marked as ,in, Represents the feature dimension of the kinematic feature set; the graph features corresponding to the dynamic feature set of the gait of N measured objects can be marked as ,in, Represents the feature dimension of the dynamic feature set; the graph features corresponding to the spatiotemporal parameter feature set of the gait of N measured objects can be marked as ,in, Represents the feature dimension of the spatiotemporal parameter feature set.

[0052] As an example, after projection to the shared subspace by the generalized canonical correlation analysis algorithm, In the shared subspace, it can be expressed as , , yes The projection matrix; In the shared subspace, it can be expressed as , , yes The projection matrix; In the shared subspace, it can be expressed as , , yes The projection matrix; is the dimension of the shared subspace, for example, it can be set to {2,3,4,5,6,7,8,9}.

[0053] Generalized canonical correlation analysis calculates the projection matrix of each mode by maximizing the correlation between the projection features of different modes. The formula is as follows:

[0054] in, represents the trace function, , .

[0055] Different modalities may have different importance to downstream tasks. Therefore, in the fusion process, Introduce the corresponding weights respectively (i.e., the weight value corresponding to the kinematic feature set), (i.e., the weight value corresponding to the dynamic feature set), (i.e., the weight value corresponding to the spatiotemporal parameter feature set), which makes it possible to more flexibly handle the differences between modalities and enhance the distinguishing ability of fused features. The fused feature Z can be expressed as:

[0056] in, is a learnable weight parameter, the sum of the three is 1, which can be obtained by learning multiple training samples. The fused feature representation It not only integrates the effective information of each modality and removes redundant information, but also reflects the importance weight of each modality.

[0057] According to an exemplary embodiment of the present disclosure, a generalized canonical correlation analysis (GCCA) algorithm is combined with a weighted fusion layer to align and fuse multimodal features, such as Figure 4 (b) shown.

[0058] As an exemplary embodiment, p (for example, p may be 3) encoders (for example, transformer encoders) may be used to perform deep feature extraction on the fused features, thereby enhancing the ability to capture global information.

[0059] As an example, l The output of the encoder for: , l for 1 and p is an integer, , represents the multi-head self-attention function, represents the normalization function, Indicates l The output of the feedforward network layer in the encoder; the input of the first encoder is the fused feature Z , , represents the feature dimension of each object under test; the output of the pth encoder is the global feature. Represents a feed-forward network layer.

[0060] like Figure 4As shown in (b), each transformer encoder may include: multi-head self-attention (MHSA), feedforward neural network (FFN) and normalization layer. Each transformer encoder first uses the multi-head self-attention mechanism to calculate the global dependency between features: ; Next, use a feedforward neural network to get the final output: .

[0061] In step S500, the global features are input into a prediction module to obtain a freezing gait severity score of the subject.

[0062] As an exemplary embodiment, the prediction module includes: a first prediction module and a second prediction module; step S500 may include: inputting the global feature into the first prediction module (i.e., the first barrier module) to obtain a prediction result for characterizing whether the subject under test has a freezing gait; if the first prediction result characterizes that the subject under test has a freezing gait, inputting the global feature into the second prediction module (i.e., the second barrier module) to obtain a freezing gait severity score for characterizing the subject under test; if the first prediction result characterizes that the subject under test has not a freezing gait, determining that the freezing gait severity score of the subject under test is 0.

[0063] In fact, the prediction module is used to perform two tasks, including identifying whether the subject has a freezing gait and evaluating the freezing gait severity score of the subject. The former is a binary classification task and the latter is a regression task. The present disclosure takes into account that a large number of samples in the sample database have a freezing gait severity score of 0. This zero expansion problem will have a negative impact on the performance of the graph-based multimodal fusion framework in accurately evaluating the severity of freezing gait. In order to deal with the zero expansion problem, the present disclosure designs a double fence module, such as Figure 4 As shown in (c), by decomposing a complex task into two simpler subtasks, the zero expansion problem is solved, thereby improving the performance of the graph-based multimodal fusion framework. As an example, the types of the first prediction module and the second prediction module may include but are not limited to one of the following: adaptive boosting (Adboost), random forest (RF), support vector machine (SVM), and extreme gradient boosting (Xgboost).

[0064] In step S600, for each gait feature in the kinematic feature set, the dynamic feature set, and the spatiotemporal parameter feature set, the gait score of the gait feature is determined based on the normalized feature value of the gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set in which the gait feature is located, and based on the gait scores of all gait features, a radar chart for characterizing the gait pattern of the subject is generated.

[0065] As an exemplary embodiment, the product of the normalized feature value of each gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set to which the gait feature belongs can be used as the gait score of the gait feature.

[0066] Regarding normalizing the characteristic values ​​of the gait characteristics to obtain normalized characteristic values, the normalized characteristic values ,in, represents the eigenvalue before normalization, represents the upper limit of the eigenvalue of the gait feature, Indicates the lower limit of the eigenvalue of this gait feature.

[0067] It should be understood that at least one radar chart may be generated, for example, multiple radar charts may be generated in one-to-one correspondence with multiple feature sets. Figure 5 shows an example of a radar chart for characterizing a gait pattern of a measured object according to an exemplary embodiment of the present disclosure, Figure 5 The 10 axes in correspond to the 10 gait features one by one.

[0068] By comparing the radar chart used to characterize the gait pattern of the subject under test with the standard radar chart (a radar chart used to characterize the normal gait pattern), the higher the similarity between the two, the more normal the gait pattern of the subject under test; and, based on the difference between the two, it can be intuitively known which gait characteristics of the subject under test are seriously affected by the freezing gait symptoms of the subject under test, thereby realizing the visualization of the gait pattern analysis results of the subject under test.

[0069] According to an exemplary embodiment of the present disclosure, in a graph-based multimodal fusion framework, the graph attention neural network can provide the importance score of each gait feature relative to the modality to which it belongs, and the feature fusion module can provide the weight value corresponding to each modality. On this basis, the gait score of each gait feature can be calculated to obtain a radar chart for characterizing the gait pattern of the measured object. In this way, it is possible to intuitively determine which gait features of the measured object are most seriously affected by the freezing gait symptom, and this information can provide an effective reference for other subsequent operations (e.g., management, supervision, and care of the measured object, etc.).

[0070] According to an exemplary embodiment of the present disclosure, various parameters in the graph-based multimodal fusion framework (for example, parameters of the graph construction method, parameters of each graph attention module, parameters of each encoder, weight parameters used for fusion, parameters of the prediction module, etc.) can be obtained based on training samples, and each training sample may include: a corresponding kinematic feature set, a dynamic feature set, a spatiotemporal parameter feature set, and a label vector.

[0071] As an example, the label vector of the i-th training sample can be expressed as , For the The freezing gait severity score of the training samples is a continuous variable with a value range of [0, 28]; For the The categorical variables of training samples are , where 1 means the The training samples showed freezing gait.

[0072] According to an exemplary embodiment of the present disclosure, a multimodal gait feature set with rich physical meaning is proposed, and a graph-based multimodal fusion framework capable of fusing multiple modal data is developed to further improve the accuracy of quantitative assessment of the severity of freezing gait.

[0073] Figure 6 A structural block diagram of a freezing gait assessment device according to an exemplary embodiment of the present disclosure is shown.

[0074] like Figure 6 As shown, the freezing gait evaluation device according to an exemplary embodiment of the present disclosure includes: a data acquisition unit 101, a gait feature extraction unit 102, a graph construction and representation learning unit 103, a feature fusion and representation learning unit 104, a prediction unit 105, and a gait pattern analysis unit 106.

[0075] Specifically, the data acquisition unit 101 is configured to acquire lower limb joint motion data and plantar pressure data of the subject in multiple gait cycles during walking.

[0076] The gait feature extraction unit 102 is configured to generate a kinematic feature set, a dynamic feature set, and a spatiotemporal parameter feature set of the gait of the subject based on the acquired lower limb joint motion data and plantar pressure data.

[0077] The graph construction and representation learning unit 103 is configured to construct an attribute graph for representing each feature set in the kinematic feature set, the dynamic feature set, and the spatiotemporal parameter feature set, respectively, and use a graph attention module to extract features from the attribute graph to obtain graph features corresponding to the feature set and an importance score for each gait feature in the feature set.

[0078] The feature fusion and representation learning unit 104 is configured to fuse the graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatiotemporal parameter feature set based on the weight values ​​corresponding to each feature set to obtain fused features, and perform feature extraction on the fused features to obtain global features.

[0079] The prediction unit 105 is configured to input the global features into the prediction module to obtain the freezing gait severity score of the subject.

[0080] The gait pattern analysis unit 106 is configured to determine the gait score of each gait feature in the kinematic feature set, the dynamic feature set, and the spatiotemporal parameter feature set based on the normalized feature value of the gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set in which the gait feature is located, and generate a radar chart for characterizing the gait pattern of the measured object based on the gait scores of all gait features.

[0081] As an exemplary embodiment, the gait feature extraction unit 102 can be configured to: construct hip and knee joint angle cycle diagrams and knee and ankle joint angle cycle diagrams based on the acquired lower limb joint motion data, and obtain a kinematic feature set of the gait of the object under test by extracting the following features of the hip and knee joint angle cycle diagrams and the knee and ankle joint angle cycle diagrams: area, harmonic square wave order, average value of coupling angle, variability of coupling angle, average absolute relative phase, and deviation phase; generate a plantar pressure curve and a plantar pressure center of mass trajectory based on the acquired plantar pressure data, and obtain a dynamic feature set of the gait of the object under test by extracting multiple features of the plantar pressure curve and the plantar pressure center of mass trajectory, wherein the multiple features include: the maximum value of the vertical ground reaction force in the early support stage , the maximum value of the vertical ground reaction force in the late support stage, the minimum value of the vertical ground reaction force in the middle support stage, the growth rate of the vertical ground reaction force in the early support stage, the decrease rate of the vertical ground reaction force in the late support stage, the average value, coefficient of variation and asymmetry index of the intersection of the forward center of mass trajectory and the lateral center of mass trajectory in different gait cycles; based on the acquired lower limb joint motion data, the spatiotemporal parameter feature set of the gait of the subject is generated, wherein the spatiotemporal parameter feature set includes: swing phase time, stance phase time, single-limb support time, double-limb support time, step frequency, walking ratio, stance phase proportion, swing phase proportion, single-limb support phase proportion, double-limb support phase proportion, step speed, step length, stride, speed.

[0082] As an exemplary embodiment, the gait pattern analysis unit 106 may be configured to take the product of the normalized feature value of each gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set to which the gait feature belongs as the gait score of the gait feature.

[0083] As an exemplary embodiment, each of the graph attention modules after the graph attention module for extracting features from the attribute graph of the kinematic feature set, the graph attention module for extracting features from the attribute graph of the dynamic feature set, and the graph attention module for extracting features from the attribute graph of the spatiotemporal parameter feature set comprises: a graph attention neural network composed of q graph attention network layers and a pooling layer; wherein the first of the q graph attention network layers l The input of the graph attention network layer for: , l for 1 and An integer of q, Indicates l -1 output of the graph attention network layer, Represents the adjacency matrix of the attribute graph; wherein the input of the first graph attention network layer in the q graph attention network layers is is the node feature matrix of the attribute graph; the input of the pooling layer is: , the output of the pooling layer is the graph feature.

[0084] As an exemplary embodiment, the feature fusion and representation learning unit 104 can be configured as follows: mapping each graph feature to a common subspace by maximizing the correlation between the projection matrices of different graph features; obtaining a fused feature by assigning corresponding weight values ​​to the representations of each graph feature in the common subspace and then concatenating them; wherein the weight value corresponding to the representation of the graph feature in the common subspace is: the weight value corresponding to the feature set corresponding to the graph feature; the weight values ​​corresponding to each feature set are obtained by learning from multiple training samples.

[0085] As an exemplary embodiment, the feature fusion and representation learning unit 104 may be configured to: use p encoders to extract features from the fused features to obtain global features; wherein the first l The output of the encoder for: , , represents the multi-head self-attention function, represents the normalization function, Indicates l The output of the feedforward network layer in the encoder; where is the fused feature; the output of the p-th encoder is the global feature.

[0086] As an exemplary embodiment, the prediction module includes: a first prediction module and a second prediction module; wherein the prediction unit 105 can be configured to: input the global feature into the first prediction module to obtain a prediction result for characterizing whether the subject under test has a freezing gait; if the first prediction result characterizes that the subject under test has a freezing gait, input the global feature into the second prediction module to obtain a freezing gait severity score for characterizing the subject under test; if the first prediction result characterizes that the subject under test has not a freezing gait, determine that the freezing gait severity score of the subject under test is 0.

[0087] It should be understood that the specific processing performed by the freezing gait evaluation device according to the exemplary embodiment of the present disclosure has been described with reference to Figure 1-5The details have been described in detail and will not be repeated here.

[0088] It should be understood that the various units and modules in the freezing gait assessment device according to the exemplary embodiment of the present disclosure may be implemented as hardware components and / or software components.

[0089] According to an exemplary embodiment of the present disclosure, a freezing gait assessment system is also provided, comprising a lower limb motion capture device and a data processing device as described in the above exemplary embodiment.

[0090] Specifically, the lower limb motion capture device includes: a binocular camera for capturing the lower limb joint motion data of the subject under test in multiple gait cycles during walking; a plantar pressure acquisition device for acquiring the plantar pressure data of the subject under test in the multiple gait cycles. In addition, the lower limb motion capture device may also include: multiple reflective marker balls for wearing on the hip joint, knee joint, ankle joint, heel and toe of the subject under test; an infrared light source array for emitting infrared light to the multiple reflective marker balls; wherein the binocular camera may be a near-infrared binocular camera for capturing the infrared light reflected by the multiple reflective marker balls in the multiple gait cycles.

[0091] The data processing device includes: at least one memory and at least one processor, wherein a set of computer executable instructions is stored in the at least one memory, and when the set of computer executable instructions is executed by the at least one processor, the freezing gait assessment method as described in the above exemplary embodiment is executed.

[0092] As an example, the data processing device may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above instruction set. Here, the data processing device is not necessarily a single electronic device, but may also be any collection of devices or circuits capable of executing the above instructions (or instruction sets) individually or in combination. The data processing device may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that is interconnected with an interface locally or remotely (e.g., via wireless transmission).

[0093] In a data processing device, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, a processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0094] The processor may execute instructions or codes stored in the memory, wherein the memory 600 may also store data. Instructions and data may also be sent and received over a network via a network interface device, wherein the network interface device may employ any known transmission protocol.

[0095] The memory may be integrated with the processor, for example, RAM or flash memory is arranged within an integrated circuit microprocessor or the like. In addition, the memory may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory and the processor may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor can read files stored in the memory.

[0096] In addition, the data processing device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the data processing device may be connected to each other via a bus and / or a network.

[0097] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions may also be provided, wherein when the instructions are executed by at least one processor, the at least one processor is prompted to perform the freezing gait assessment method as described in the above exemplary embodiment. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0098] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided, and instructions in the computer program product may be executed by at least one processor to complete the freezing gait assessment method as described in the above exemplary embodiment.

[0099] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0100] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A freezing gait assessment method, characterized in that: include: Acquire lower limb joint motion data and plantar pressure data of the subject during multiple gait cycles during walking; Based on the acquired lower limb joint motion data and plantar pressure data, a kinematic feature set, a dynamic feature set, and a spatiotemporal parameter feature set of the gait of the subject are generated; For each feature set in the kinematic feature set, dynamic feature set, and spatiotemporal parameter feature set, an attribute graph is constructed to characterize the feature set, and a graph attention module is used to extract features from the attribute graph to obtain the graph features corresponding to the feature set and the importance score of each gait feature in the feature set; Based on the weight values ​​corresponding to each feature set, the graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatiotemporal parameter feature set are fused to obtain fused features, and feature extraction is performed on the fused features to obtain global features; The global features are input into the prediction module to obtain the freezing gait severity score of the tested subject; For each gait feature in the kinematic feature set, the dynamic feature set, and the spatiotemporal parameter feature set, the gait score of the gait feature is determined based on the normalized feature value of the gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set in which the gait feature is located. Based on the gait scores of all gait features, a radar chart for characterizing the gait pattern of the subject is generated.

2. The method according to claim 1, characterized in that Based on the acquired lower limb joint motion data and plantar pressure data, the steps of generating a kinematic feature set, a dynamic feature set, and a spatiotemporal parameter feature set of the gait of the measured object include: Based on the acquired lower limb joint motion data, hip and knee joint angle cycle diagrams and knee and ankle joint angle cycle diagrams are constructed, and the following features of the hip and knee joint angle cycle diagrams and knee and ankle joint angle cycle diagrams are extracted: area, harmonic square wave number, average value of coupling angle, variability of coupling angle, average absolute relative phase and deviation phase, so as to obtain the kinematic feature set of the gait of the subject; Generate a plantar pressure curve and a plantar pressure centroid trajectory based on the acquired plantar pressure data, and obtain a set of dynamic characteristics of the gait of the measured object by extracting multiple features of the plantar pressure curve and the plantar pressure centroid trajectory, wherein the multiple features include: the maximum value of the vertical ground reaction force in the early support stage, the maximum value of the vertical ground reaction force in the late support stage, the minimum value of the vertical ground reaction force in the middle support stage, the growth rate of the vertical ground reaction force in the early support stage, the decrease rate of the vertical ground reaction force in the late support stage, the average value, the coefficient of variation and the asymmetry index of the intersection of the forward centroid trajectory and the lateral centroid trajectory in different gait cycles; Based on the acquired lower limb joint motion data, a spatiotemporal parameter feature set of the gait of the subject is generated, wherein the spatiotemporal parameter feature set includes: swing phase time, stance phase time, single-limb support time, double-limb support time, step frequency, walking ratio, stance phase proportion, swing phase proportion, single-limb support phase proportion, double-limb support phase proportion, step speed, step length, stride, and speed.

3. The method according to claim 1, characterized in that The step of determining a gait score of each gait feature based on the normalized feature value of each gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set to which the gait feature belongs comprises: The product of the normalized feature value of each gait feature, the importance score of the gait feature, and the weight value corresponding to the feature set to which the gait feature belongs is taken as the gait score of the gait feature.

4. The method according to claim 1, characterized in that: Each of the graph attention modules after the graph attention module for extracting features from the attribute graph of the kinematic feature set, the graph attention module for extracting features from the attribute graph of the dynamic feature set, and the graph attention module for extracting features from the attribute graph of the spatiotemporal parameter feature set comprises: a graph attention neural network consisting of q graph attention network layers and a pooling layer; Among them, the first l The input of the graph attention network layer for: , l for 1 and An integer of q, Indicates l -1 output of the graph attention network layer, Represents the adjacency matrix of the attribute graph; Among them, the input of the first graph attention network layer among the q graph attention network layers is is the node feature matrix of the attribute graph; Among them, the input of the pooling layer is: , the output of the pooling layer is the graph feature.

5. The method according to claim 1, characterized in that Based on the weight values ​​corresponding to the feature sets, the step of fusing the graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatiotemporal parameter feature set to obtain fused features includes: By maximizing the correlation between the projection matrices of different graph features, each graph feature is mapped to a common subspace; By assigning corresponding weight values ​​to the representations of each graph feature in the common subspace and then concatenating them, the fused features are obtained; Among them, the weight value corresponding to the representation of the graph feature in the common subspace is: the weight value corresponding to the feature set corresponding to the graph feature; the weight value corresponding to each feature set is obtained by learning multiple training samples.

6. The method according to claim 1, characterized in that The step of extracting features from the fused features to obtain global features includes: extracting features from the fused features using p encoders to obtain global features; Among them, l The output of the encoder for: , , represents the multi-head self-attention function, represents the normalization function, Indicates l The output of the feed-forward network layer in the encoder; in, is the post-fusion feature; Among them, the output of the p-th encoder is the global feature.

7. The method according to claim 1, characterized in that The prediction module includes: a first prediction module and a second prediction module; The step of inputting the global features into the prediction module to obtain the freezing gait severity score of the subject includes: Inputting the global feature into the first prediction module to obtain a prediction result for characterizing whether the subject has a freezing gait; If the first prediction result indicates that the subject has a freezing gait, the global feature is input into a second prediction module to obtain a freezing gait severity score for indicating the subject; If the first prediction result indicates that the subject does not have a freezing gait, then the freezing gait severity score of the subject is determined to be 0.

8. A freezing gait assessment device, characterized in that: include: A data acquisition unit is configured to acquire lower limb joint motion data and plantar pressure data of the subject during multiple gait cycles during walking; A gait feature extraction unit is configured to generate a kinematic feature set, a dynamic feature set, and a spatiotemporal parameter feature set of the gait of the subject under test based on the acquired lower limb joint motion data and plantar pressure data; A graph construction and representation learning unit is configured to construct an attribute graph for representing each feature set in the kinematic feature set, the dynamic feature set, and the spatiotemporal parameter feature set, respectively, and use a graph attention module to perform feature extraction on the attribute graph to obtain a graph feature corresponding to the feature set and an importance score of each gait feature in the feature set; The feature fusion and representation learning unit is configured to fuse the graph features corresponding to the kinematic feature set, the graph features corresponding to the dynamic feature set, and the graph features corresponding to the spatiotemporal parameter feature set based on the weight values ​​corresponding to each feature set to obtain fused features, and perform feature extraction on the fused features to obtain global features; A prediction unit, configured to input the global features into the prediction module to obtain a freezing gait severity score of the subject; The gait pattern analysis unit is configured to determine the gait score of each gait feature in the kinematic feature set, the dynamic feature set and the spatiotemporal parameter feature set based on the normalized feature value of the gait feature, the importance score of the gait feature and the weight value corresponding to the feature set in which the gait feature is located, and generate a radar chart for characterizing the gait pattern of the object under test based on the gait scores of all gait features.

9. A freezing gait assessment system, characterized in that: include: Lower limb motion capture device and data processing device; Wherein, the lower limb motion capture device comprises: A binocular camera is used to capture the lower limb joint motion data of the subject during multiple gait cycles during walking; A plantar pressure collection device, used to collect plantar pressure data of the subject under test during the multiple gait cycles; Wherein, the data processing device comprises: at least one processor; at least one memory storing computer executable instructions, Wherein, when the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to perform the freezing gait assessment method according to any one of claims 1 to 7.

10. The system according to claim 9, characterized in that The lower limb motion capture device also includes: Multiple reflective marker balls for wearing on the hip, knee, ankle, heel and toes of the subject; An infrared light source array, used for emitting infrared light to the plurality of reflective marker balls; The binocular camera is a near-infrared binocular camera, which is used to capture infrared light reflected by the multiple reflective marker balls during the multiple gait cycles.

Images