Joint coordination quantification analysis method and device, and electronic device

Abstract

The application provides a joint coordination quantitative analysis method and device and electronic equipment, and relates to the technical field of artificial intelligence. The method comprises the following steps: analyzing a video to be analyzed based on a BlazePose algorithm to obtain coordinate data; determining a kinematics data matrix based on the coordinate data; inputting the kinematics data matrix into a joint coordination attention model to obtain a joint coordination variability score; determining balance, stability, rhythm, reconstruction evaluation results and a maximum joint movement angle based on the kinematics data matrix; and the joint coordination quantitative analysis result of a user comprises the joint coordination variability score, the balance evaluation result, the stability evaluation result, the rhythm evaluation result, the reconstruction evaluation result and the maximum joint movement angle. Through the above method, the device cost and the operation threshold can be reduced, the application scenarios of the method can be expanded, the problem of limited evaluation dimensions caused by a single index or score can be avoided, and the accuracy of the joint coordination quantitative analysis result is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus and electronic device for quantitative analysis of joint coordination. Background Technology

[0002] With the increasing prevalence of various diseases and sports injuries, motor dysfunction is becoming a prominent issue, directly leading to abnormal changes in human movement patterns. The scientific community has confirmed that the human central nervous system (CNS) can regulate human movement patterns through specific synergistic mechanisms. These mechanisms exhibit significant plasticity, meaning that timely and precise rehabilitation interventions can effectively promote the recovery of neurological function and motor ability. Against this backdrop, establishing a comprehensive and quantifiable assessment system for motor synergy is particularly important.

[0003] Currently, relevant research mainly focuses on the intelligent transformation of traditional rehabilitation assessment scales (such as the Fugl-Meyer Assessment Scale, Range of Motion Scale, Berg Balance Scale, etc.). Some related technologies can collect human motion data through virtual reality systems and wearable devices to complete automated assessments, while simultaneously conducting qualitative analysis based on human kinematic and physiological data.

[0004] In the quantitative analysis of joint coordination, the mainstream methods use eigenvalue decomposition techniques such as principal component analysis (PCA) and nonnegative matrix factorization (NMF) to process and analyze human motion data to obtain quantitative analysis results. However, these methods have significant limitations when handling nonlinear data and related task variables, are prone to overfitting, and struggle to simultaneously capture spatiotemporal features and long-term motion dependencies, resulting in insufficient accuracy of the quantitative analysis results. Furthermore, existing motion function analysis systems generally employ wearable or patch-type optical motion capture devices and inertial sensing units to collect human motion data, combined with data-driven machine learning algorithms to identify abnormal movement patterns. Such systems are not only costly but also have complex operating procedures, severely limiting their application and promotion in non-professional settings such as communities and homes.

[0005] In summary, existing methods for quantitative analysis of joint coordination are inaccurate, costly, and limited in application scenarios. Summary of the Invention

[0006] This invention provides a method, apparatus, and electronic device for joint coordination quantitative analysis, which addresses the shortcomings of existing joint coordination quantitative analysis methods, such as inaccurate analysis results, high cost, and limited application scenarios.

[0007] This invention provides a method for quantitative analysis of joint coordination, comprising: analyzing a video to be analyzed based on the BlazePose algorithm to obtain coordinate data of joint marker points; the video to be analyzed is a video of a user performing a motion assessment action; determining the user's kinematic data matrix based on the coordinate data; inputting the kinematic data matrix into a joint coordination attention model to obtain a joint coordination variability score output by the joint coordination attention model; and determining balance assessment results, stability assessment results, rhythmicity assessment results, reconstructiveness assessment results, and maximum joint motion angle based on the kinematic data matrix; wherein, the user's quantitative analysis results of joint coordination include the joint coordination variability score, balance assessment results, stability assessment results, rhythmicity assessment results, reconstructiveness assessment results, and maximum joint motion angle.

[0008] According to the present invention, a joint coordination quantitative analysis method is provided to determine the user's kinematic data matrix based on coordinate data, including: analyzing the coordinate data using a kinematic analysis algorithm to generate time-series trajectory data and calculating the time-series angle data of joint marker points; and standardizing the time-series trajectory data and time-series angle data based on a second-order B-spline interpolation algorithm to generate a kinematic data matrix.

[0009] According to the present invention, a joint coordination quantitative analysis method is provided, wherein the joint coordination attention model includes a concatenated attention module, a graph convolutional network pre-classification module, and a cyclic masking module; the concatenated attention module is used to concatenate features of the kinematic data matrix to generate concatenated features carrying attention weights; the graph convolutional network pre-classification module is used to generate user pre-classification results based on the concatenated features; and the cyclic masking module is used to generate joint coordination variability scores based on the concatenated features and the user pre-classification results.

[0010] According to the joint coordination quantitative analysis method provided by the present invention, the joint marker points include the left hip joint and the right hip joint; the balance assessment result is determined by the following steps: based on the kinematic data matrix, a first dynamic trajectory of the angle between the left hip joint and the horizontal axis is determined; based on the kinematic data matrix, a second dynamic trajectory of the angle between the right hip joint and the horizontal axis is determined; based on the first dynamic trajectory and the second dynamic trajectory, the balance assessment result is determined.

[0011] According to the joint coordination quantitative analysis method provided by the present invention, the joint marker points include the left hip joint and the right hip joint; the stability assessment result is determined by the following steps: based on the kinematic data matrix, the third dynamic trajectory of the user's center of gravity in the sagittal plane is determined; the user's center of gravity is the midpoint of the line connecting the left hip joint and the right hip joint; the standard deviation of the center of gravity trajectory deviation of the third dynamic trajectory is determined; based on the standard deviation of the center of gravity trajectory deviation, the stability assessment result is determined.

[0012] According to the joint coordination quantitative analysis method provided by the present invention, the rhythmicity assessment result is determined through the following steps: determining the dynamic trajectory of the left foot and the dynamic trajectory of the right foot based on the kinematic data matrix; determining the user's gait cycle based on the dynamic trajectory of the left foot and the dynamic trajectory of the right foot; aligning the dynamic trajectory of the left foot and the dynamic trajectory of the right foot based on the dynamic time warping algorithm, and determining the variance value of the gait cycle; and determining the rhythmicity assessment result based on the variance value.

[0013] According to a joint coordination quantitative analysis method provided by the present invention, the reconstructive assessment result is determined through the following steps: obtaining a spatiotemporal vector matrix through nonnegative matrix decomposition based on the kinematic data matrix; generating a reconstructive kinematic data matrix based on the spatiotemporal vector matrix; determining the user's reconstructive percentage based on the reconstructive kinematic data matrix and the kinematic data matrix; and comparing the reconstructive percentage with the reconstructive percentage of healthy individuals to determine the reconstructive assessment result.

[0014] According to a joint coordination quantitative analysis method provided by the present invention, the joint markers include the left hip joint, right hip joint, left knee joint, and right knee joint; the maximum joint motion angle is determined by the following steps: based on the kinematic data matrix, a first spatial angle between the first joint line and the vertical axis is determined; the first joint line is the line connecting the left hip joint and the left knee joint; based on the kinematic data matrix, a second spatial angle between the second joint line and the vertical axis is determined; the second joint line is the line connecting the right hip joint and the right knee joint; the first spatial angle and the second spatial angle are filtered to determine the maximum joint motion angle.

[0015] This invention also provides a joint coordination quantitative analysis device, comprising: a coordinate data determination module, used to analyze the video to be analyzed based on the BlazePose algorithm to obtain coordinate data of joint marker points; the video to be analyzed is a video of a user performing a motion assessment action; a kinematic data determination module, used to determine the user's kinematic data matrix based on the coordinate data; a coordination variability score determination module, used to input the kinematic data matrix into a joint coordination attention model to obtain the joint coordination variability score output by the joint coordination attention model; and an index evaluation module, used to determine balance evaluation results, stability evaluation results, rhythmicity evaluation results, reconstructiveness evaluation results, and maximum joint motion angle based on the kinematic data matrix; wherein, the user's joint coordination quantitative analysis results include joint coordination variability score, balance evaluation results, stability evaluation results, rhythmicity evaluation results, reconstructiveness evaluation results, and maximum joint motion angle.

[0016] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the joint coordination quantitative analysis methods described above.

[0017] The joint coordination quantitative analysis method, device, and electronic device provided by this invention, based on the BlazePose algorithm, analyze the video to be analyzed to obtain the coordinate data of joint marker points; the video to be analyzed is a video of a user performing a motion assessment action; based on the coordinate data, the user's kinematic data matrix is ​​determined; the kinematic data matrix is ​​input into a joint coordination attention model to obtain the joint coordination variability score output by the joint coordination attention model; based on the kinematic data matrix, the balance assessment result, stability assessment result, rhythmicity assessment result, reconstructiveness assessment result, and maximum joint motion angle are determined; wherein, the user's joint coordination quantitative analysis results include the joint coordination variability score, balance assessment result, stability assessment result, rhythmicity assessment result, reconstructiveness assessment result, and maximum joint motion angle. By employing the above method, only the video of the user performing motion assessment actions needs to be analyzed to obtain the user's joint coordination quantitative analysis results. This effectively reduces equipment costs and operational barriers, expanding the application scenarios of the method. Simultaneously, based on the BlazePose algorithm, the video is analyzed to obtain the coordinate data of joint marker points. After determining the user's kinematic data matrix based on the coordinate data, a joint coordination attention model is used to generate a joint coordination variability score. A multi-dimensional evaluation index system is introduced, and the evaluation results of five evaluation indicators—balance, stability, rhythmicity, reconstructiveness, and maximum joint motion angle—are determined based on the kinematic data matrix. Finally, the user's joint coordination quantitative analysis results are composed of the joint coordination variability score, balance evaluation results, stability evaluation results, rhythmicity evaluation results, reconstructiveness evaluation results, and maximum joint motion angle. This avoids the problem of limited evaluation dimensions caused by a single indicator or score, effectively improving the accuracy of the joint coordination quantitative analysis results. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the joint coordination quantitative analysis method provided by the present invention.

[0020] Figure 2 This is a schematic diagram of the intelligent joint coordination quantitative analysis system provided by the present invention.

[0021] Figure 3 This is a schematic diagram showing the marking positions of the joint markers provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the joint coordination quantitative analysis device provided by the present invention.

[0023] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] Please see Figures 1 to 3 , Figure 1 This is a flowchart illustrating the joint coordination quantitative analysis method provided by the present invention. Figure 2 This is a schematic diagram of the intelligent joint coordination quantitative analysis system provided by the present invention. Figure 3 This is a schematic diagram showing the marking positions of the joint markers provided by the present invention.

[0026] like Figure 2 As shown, the joint coordination quantitative analysis method is applied to an intelligent joint coordination quantitative analysis system. The hardware of this system includes a personal computer and a mobile terminal device with high-definition camera capabilities. The mobile terminal device and the personal computer can establish a stable data transmission channel through a standard USB (Universal Serial Bus) interface. Data transmission between the mobile terminal device and the personal computer can also occur via Bluetooth or a wireless network. The online evaluation platform of the intelligent joint coordination quantitative analysis system can render a three-dimensional virtual human motion model in real time and integrates a multimedia guidance module. Through a combination of a visual interface and voice prompts, it assists users in completing personal information registration and spatial registration of the healthy or affected limb.

[0027] Optionally, mobile terminal devices include, but are not limited to, mobile phones, cameras, tablets, and photographic equipment.

[0028] Preferably, in a home setting, the mobile terminal device is a mobile phone; in a medical setting, the mobile terminal device is a binocular camera.

[0029] like Figure 1 As shown, the joint coordination quantitative analysis method includes steps S110 to S140, and the specific steps are as follows:

[0030] S110: Based on the BlazePose algorithm, it analyzes the video to be analyzed and obtains the coordinate data of the joint marker points.

[0031] The videos to be analyzed are those of users performing motion assessment actions.

[0032] Optionally, the user is a stroke patient group with the following clinical characteristics: (1) first occurrence of unilateral cerebral infarction or cerebral hemorrhage; (2) presence of unilateral or bilateral elbow flexor and / or extensor muscle spasm symptoms; (3) passive range of motion of both elbow joints can reach more than 90°; (4) modified Ashworth scale score between 1 and 3; (5) basic auditory comprehension ability and able to cooperate in completing motor assessment actions; (6) exclusion of other known related diseases that may affect upper limb motor function.

[0033] Specifically, the intelligent joint coordination quantitative analysis system has designed a standardized exercise assessment program. Before assessing the user, the user needs to perform the following exercise assessment movements: for the user's upper limbs, shoulder flexion movements within a range of 90° to 180° in the sagittal plane are required; for the user's lower limbs, knee flexion movements are required while keeping the hip joint fully extended.

[0034] Optionally, all motion assessment actions are performed under the guidance of an intelligent joint coordination quantitative analysis system to ensure the accuracy and consistency of the assessment data.

[0035] While the user performs the motion assessment action, a mobile terminal device can be used to film the user, obtain the video to be analyzed while the user is performing the motion assessment action, and upload the video to the online assessment platform of the intelligent joint coordination quantitative analysis system.

[0036] It should be noted that, as Figure 3 As shown, based on the needs of clinical rehabilitation assessment, the intelligent joint coordination quantitative analysis system established a human kinematic topology containing 14 joint markers. These markers include both hip joints, both acromions, both olecranon processes of the elbows, both thumb metacarpophalangeal joints, both little finger metacarpophalangeal joints, both knee joints, and both ankle joints; the two hip joints include the left hip joint (…). Figure 3 (marked point 1) and right hip joint ( Figure 3 (Point 2 in the text), bilateral acromions including the left acromion ( Figure 3 (marked point 3) and right acromion ( Figure 3 (Point 4 in the text) The olecranon processes of both elbows, including the left olecranon process ( Figure 3 (marked point 5) and the olecranon process of the right elbow ( Figure 3 (Point 6 in the text) The bilateral thumb metacarpophalangeal joints include the left thumb metacarpophalangeal joint ( Figure 3 (marked point 7) and the metacarpophalangeal joint of the right thumb ( Figure 3 (Point 8 in the text) The metacarpophalangeal joints of both little fingers, including the left little finger metacarpophalangeal joint ( Figure 3 (marked point 9) and the metacarpophalangeal joint of the right little finger ( Figure 3 (Point 10 in the text), bilateral knee joints including the left knee joint ( Figure 3 (marked point 11) and the right knee joint ( Figure 3 (Point 12 in the text), bilateral ankle joints including the left ankle joint ( Figure 3 (marked point 13) and right ankle joint ( Figure 3 (Point 14 in the text). The spatial distribution relationship of each joint marker point is as follows: Figure 3 As shown, these joint markers together constitute a complete human motion analysis network.

[0037] Furthermore, the intelligent joint coordination quantitative analysis system can use the optimized BlazePose algorithm to analyze the video to be analyzed. The BlazePose algorithm can accurately extract the three-dimensional spatial coordinates of 14 joint marker points as coordinate data by using the video frame rate of the video to be analyzed as the sampling interval.

[0038] S120: Determine the user's kinematic data matrix based on coordinate data.

[0039] Specifically, the intelligent joint coordination quantitative analysis system can calculate the angle change parameters of 14 joint marker points based on coordinate data and through kinematic analysis algorithms.

[0040] Furthermore, to eliminate differences in individual user movement speeds, the intelligent joint coordination quantitative analysis system can use a second-order B-spline interpolation algorithm with C2 continuity to standardize the angle change parameters of 14 joint marker points, generating a kinematic data matrix with a unified format. .

[0041] Among them, the kinematic data matrix , Representing real numbers, kinematic data matrix It is a three-dimensional matrix, with each dimension corresponding to the number of frames in the video being analyzed. , number of joint marker points and spatial coordinate components .

[0042] It should be noted that, The value can be 14 or 28, when When the value is 14, it indicates that there are 14 joint markers, but the angle changes of the joint markers are not considered. When the value is 28, it means that there are 14 joint markers and the angular changes of the 14 joint markers in the horizontal direction axis (i.e., the x-axis) and the sagittal direction axis (i.e., the y-axis) are considered respectively. The value is 3, which represents the spatial coordinate components of the three axes (i.e., the x-axis, y-axis, and z-axis).

[0043] S130: Input the kinematic data matrix into the joint coordination attention model to obtain the joint coordination variability score output by the joint coordination attention model.

[0044] Specifically, the joint collaborative attention model includes a concatenated attention module, a graph convolutional network pre-classification module, and a cyclic mask module.

[0045] The concatenated attention module is used to concatenate features of the kinematic data matrix to generate concatenated features carrying attention weights.

[0046] The graph convolutional network pre-classification module is used to generate user pre-classification results based on concatenated features.

[0047] The cyclic masking module is used to generate joint covariance scores based on concatenated features and user pre-classification results.

[0048] S140: Based on the kinematic data matrix, determine the balance assessment results, stability assessment results, rhythmic assessment results, remodeling assessment results, and maximum joint motion angle.

[0049] The results of the user's joint coordination quantitative analysis include joint coordination variability score, balance assessment results, stability assessment results, rhythmic assessment results, remodeling assessment results, and maximum joint motion angle.

[0050] The joint coordination quantitative analysis method provided in this embodiment is based on the BlazePose algorithm. It analyzes the video to be analyzed to obtain the coordinate data of joint marker points. The video to be analyzed is a video of a user performing a motion assessment action. Based on the coordinate data, the user's kinematic data matrix is ​​determined. The kinematic data matrix is ​​input into a joint coordination attention model to obtain the joint coordination variability score output by the model. Based on the kinematic data matrix, the balance assessment result, stability assessment result, rhythmicity assessment result, reconstructiveness assessment result, and maximum joint motion angle are determined. The user's joint coordination quantitative analysis results include the joint coordination variability score, balance assessment result, stability assessment result, rhythmicity assessment result, reconstructiveness assessment result, and maximum joint motion angle. By employing the above method, only the video of the user performing motion assessment actions needs to be analyzed to obtain the user's joint coordination quantitative analysis results. This effectively reduces equipment costs and operational barriers, expanding the application scenarios of the method. Simultaneously, based on the BlazePose algorithm, the video is analyzed to obtain the coordinate data of joint marker points. After determining the user's kinematic data matrix based on the coordinate data, a joint coordination attention model is used to generate a joint coordination variability score. A multi-dimensional evaluation index system is introduced, and the evaluation results of five evaluation indicators—balance, stability, rhythmicity, reconstructiveness, and maximum joint motion angle—are determined based on the kinematic data matrix. Finally, the user's joint coordination quantitative analysis results are composed of the joint coordination variability score, balance evaluation results, stability evaluation results, rhythmicity evaluation results, reconstructiveness evaluation results, and maximum joint motion angle. This avoids the problem of limited evaluation dimensions caused by a single indicator or score, effectively improving the accuracy of the joint coordination quantitative analysis results.

[0051] In some embodiments, determining the user's kinematic data matrix based on coordinate data includes: analyzing the coordinate data using a kinematic analysis algorithm to generate temporal trajectory data and calculating temporal angle data of joint markers; and standardizing the temporal trajectory data and temporal angle data using a second-order B-spline interpolation algorithm to generate a kinematic data matrix.

[0052] Specifically, after obtaining the coordinate data, the intelligent joint coordination quantitative analysis system can analyze the coordinate data through kinematic analysis algorithms to generate time-series trajectory data and calculate the time-series angle data of 14 joint marker points.

[0053] Furthermore, to eliminate differences in individual user movement speeds, the intelligent joint coordination quantitative analysis system can use a second-order B-spline interpolation algorithm with C2 continuity to standardize the temporal trajectory data and temporal angle data of 14 joint markers, generating a kinematic data matrix with a unified format. .

[0054] Among them, the kinematic data matrix , Representing real numbers, kinematic data matrix It is a three-dimensional matrix, with each dimension corresponding to the number of frames in the video being analyzed. , number of joint marker points and spatial coordinate components .

[0055] It should be noted that, The value can be 14 or 28, when When the value is 14, it indicates that there are 14 joint markers, but the angle changes of the joint markers are not considered. When the value is 28, it means that there are 14 joint markers and the angular changes of the 14 joint markers in the horizontal direction axis (i.e., the x-axis) and the sagittal direction axis (i.e., the y-axis) are considered respectively. The value is 3, which represents the spatial coordinate components of the three axes (i.e., the x-axis, y-axis, and z-axis).

[0056] In some embodiments, the joint coordination attention model includes a concatenated attention module, a graph convolutional network pre-classification module, and a cyclic masking module; the concatenated attention module is used to concatenate features of the kinematic data matrix to generate concatenated features carrying attention weights; the graph convolutional network pre-classification module is used to generate user pre-classification results based on the concatenated features; and the cyclic masking module is used to generate joint coordination variability scores based on the concatenated features and the user pre-classification results.

[0057] Specifically, the kinematic data matrix is ​​used as the joint motion trajectory feature and input into the Serial Attention Unit (SAU) of the joint collaborative attention model. The Serial Attention Unit can concatenate the joint motion trajectory features to generate concatenated features carrying attention weights.

[0058] The processing of the concatenated attention module can be represented by the following formula:

[0059] ;

[0060] ;

[0061] ;

[0062] in, For the first One joint marker point; For the first Joint markers; equation Representative will feature and characteristics Series; For the first Input features of each joint marker point; Transpose of representative features. For the first Transpose of the input features of each joint marker point; for Attention weights; for Attention weights.

[0063] The initial values ​​for the attention weights are random values ​​between (0, 1).

[0064] Furthermore, users are generally divided into two categories: patients and healthy individuals. The concatenated features carrying attention weights are input into the graph convolutional network pre-classification module. The graph convolutional network pre-classification module can pre-classify patients and healthy individuals based on the concatenated features and generate user pre-classification results.

[0065] It should be noted that, considering the image characteristics and temporal sequence of the video to be analyzed, the trajectory graph convolutional network algorithm was selected to construct the graph convolutional network pre-classification module, so as to retain sufficient temporal information in the length dimension and improve the accuracy of pre-classification.

[0066] Furthermore, the concatenated features and user pre-classification results are input into the cyclic masking module, which can generate joint covariance scores for different joint markers of the user based on the concatenated features and user pre-classification results.

[0067] In some embodiments, the joint markers include the left hip joint and the right hip joint; the balance assessment result is determined by the following steps: determining a first dynamic trajectory of the angle between the left hip joint and the horizontal axis based on the kinematic data matrix; determining a second dynamic trajectory of the angle between the right hip joint and the horizontal axis based on the kinematic data matrix; and determining the balance assessment result based on the first dynamic trajectory and the second dynamic trajectory.

[0068] Understandably, using only the joint coordination variability score as the result of joint coordination quantitative analysis has the problem of limited evaluation dimensions. Based on this, this embodiment proposes a multidimensional evaluation index system, which combines the joint coordination variability score and the multidimensional evaluation index system to realize the user's joint coordination quantitative analysis.

[0069] The multidimensional evaluation index system includes five dimensions of evaluation indicators: balance, stability, rhythmicity, reconfiguration, and maximum joint motion angle.

[0070] Balance is an assessment indicator that quantifies motor function by analyzing the symmetry of a user's movements on both sides. The intelligent joint coordination quantitative analysis system can set a baseline value of 0 for the balance assessment indicator, representing an idealized, unbiased state.

[0071] Specifically, after obtaining the kinematic data matrix, the intelligent joint coordination quantitative analysis system can calculate the spatial angle between the user's left hip joint and the unit vector (1, 0, 0) of the horizontal axis (i.e., the x-axis), and the spatial angle between the user's right hip joint and the unit vector (1, 0, 0) of the horizontal axis (i.e., the x-axis), and normalize the angles on both sides.

[0072] Furthermore, the intelligent joint coordination quantitative analysis system can analyze at least 100 consecutive video frames in the video to be analyzed during the user's walking process from far to near, based on the kinematic data matrix and the spatial angle between the left hip joint and the unit vector (1, 0, 0) of the horizontal axis (i.e., the x-axis), and extract the first dynamic trajectory of the angle between the user's left hip joint and the horizontal axis (i.e., the x-axis).

[0073] Similarly, the intelligent joint coordination quantitative analysis system can analyze at least 100 consecutive video frames in the video to be analyzed during the user's walking process from far to near, based on the kinematic data matrix and the spatial angle between the right hip joint and the unit vector (1, 0, 0) of the horizontal axis (i.e., the x-axis), and extract the second dynamic trajectory of the angle between the user's right hip joint and the horizontal axis (i.e., the x-axis).

[0074] Furthermore, based on the first dynamic trajectory and the second dynamic trajectory, the angular difference between the left and right hip joints is determined, and statistical analysis is performed on the angular difference between the left and right hip joints to achieve symmetry quantitative assessment and generate balance assessment results.

[0075] In some embodiments, the joint markers include the left hip joint and the right hip joint; the stability assessment result is determined by the following steps: determining the third dynamic trajectory of the user's center of gravity in the sagittal plane based on the kinematic data matrix; the user's center of gravity is the midpoint of the line connecting the left hip joint and the right hip joint; determining the standard deviation of the center of gravity trajectory deviation of the third dynamic trajectory; and determining the stability assessment result based on the standard deviation of the center of gravity trajectory deviation.

[0076] Stability is an indicator that assesses a user's motor control ability by quantifying the degree of center-of-gravity shift during movement. Its technical principle is based on the human center-of-gravity theory defined in human kinetics. In this theory, the midpoint of the line connecting the user's two hip joints is selected as the coordinate of the human body's center of gravity. By calculating the standard deviation of the trajectory of this center-of-gravity coordinate, a model of the body's center-of-gravity shift can be completed. The intelligent joint coordination quantitative analysis system allows setting a baseline value of 0 for the stability assessment indicator, representing an idealized, unshifted state.

[0077] Specifically, after obtaining the kinematic data matrix, the intelligent joint coordination quantitative analysis system can extract and analyze continuous video frame data of the user's walking process from far to near in the video to be analyzed, and extract the third dynamic trajectory of the user's center of gravity in the sagittal plane, with the user's center of gravity being the midpoint of the line connecting the left hip joint and the right hip joint.

[0078] Furthermore, the mean normalization method is used to eliminate differences in user body shape, and the standard deviation (SD) of the center of gravity trajectory deviation of the third dynamic trajectory is calculated. The stability of the user during movement is evaluated by the standard deviation of the center of gravity trajectory deviation, and the stability evaluation result is determined. The stability evaluation result can directly reflect the user's ability to maintain dynamic balance.

[0079] In some embodiments, the rhythmicity assessment result is determined by the following steps: determining the dynamic trajectory of the left foot and the dynamic trajectory of the right foot based on the kinematic data matrix; determining the user's gait cycle based on the dynamic trajectory of the left foot and the dynamic trajectory of the right foot; aligning the dynamic trajectory of the left foot and the dynamic trajectory of the right foot based on the dynamic time warping algorithm and determining the variance of the gait cycle; and determining the rhythmicity assessment result based on the variance.

[0080] Rhythmology is a core indicator that quantifies a user's motor coordination by analyzing the temporal uniformity of their gait cycles. The intelligent joint coordination quantitative analysis system allows setting a baseline value of 0 for the rhythmology assessment indicator, representing an idealized, perfectly uniform gait.

[0081] Specifically, after obtaining the kinematic data matrix, the intelligent joint coordination quantitative analysis system can determine the user's left foot dynamic trajectory and right foot dynamic trajectory based on the kinematic data matrix, and determine the user's gait cycle based on the left foot dynamic trajectory and right foot dynamic trajectory.

[0082] The specific methods for calculating gait period include the following two:

[0083] (1) The dynamic trajectory of the left foot can be the trajectory of the left heel in the sagittal plane (i.e., the y-axis), and the dynamic trajectory of the right foot can be the trajectory of the right heel in the sagittal plane (i.e., the y-axis). Based on the dynamic trajectories of the left and right feet, the time interval between the user's left and right feet reaching the maximum forward swing amplitude is detected and used as the user's gait cycle.

[0084] (2) The dynamic trajectory of the left foot can be the vertical axis (i.e., the z-axis) trajectory of the left toe, and the dynamic trajectory of the right foot can be the vertical axis (i.e., the z-axis) trajectory of the right toe. Based on the dynamic trajectory of the left foot and the dynamic trajectory of the right foot, the time difference between the two occurrences of the lowest point of the z-axis when the user's foot lands is detected and used as the user's gait cycle.

[0085] Furthermore, in the trajectory analysis stage, the intelligent joint coordination quantitative analysis system can capture and analyze continuous video frame data of the user walking from far to near in the video to be analyzed. Through the dynamic time warping algorithm, the dynamic trajectory of the left foot and the dynamic trajectory of the right foot are aligned, and the variance of the gait cycle is calculated.

[0086] Furthermore, the variance of the gait cycle is normalized to eliminate individual gait differences, and the rhythmic assessment results are generated based on the degree to which the variance after range normalization deviates from the baseline value of the rhythmic assessment index.

[0087] In some embodiments, the reconstructive assessment result is determined by the following steps: obtaining a spatiotemporal vector matrix through nonnegative matrix decomposition based on the kinematic data matrix; generating a reconstructive kinematic data matrix based on the spatiotemporal vector matrix; determining the user's reconstructive percentage based on the reconstructive kinematic data matrix and the kinematic data matrix; and comparing the reconstructive percentage with the reconstructive percentage of healthy individuals to determine the reconstructive assessment result.

[0088] Reconstructivity is an assessment indicator that quantifies the differences between patient gait patterns and those of healthy individuals using spatiotemporal decomposition technology. The intelligent joint-coordinated quantitative analysis system allows setting a baseline value of 0 for the reconstructivity assessment indicator, representing a complete and idealized reconstruction of characteristics.

[0089] Specifically, the intelligent joint coordination quantitative analysis system can analyze the kinematic data matrix using nonnegative matrix factorization (NMF) to generate a spatiotemporal vector matrix, and then reconstruct features based on the spatiotemporal vector matrix to generate a reconstructed kinematic data matrix. Furthermore, it calculates the similarity between the reconstructed kinematic data matrix and the kinematic data matrix, and determines the user's reconstruction percentage based on the cosine similarity, converting the reconstruction percentage into... After standardizing the indicators, a comparative analysis was conducted with the percentage of remodeling in the healthy population to determine the remodeling assessment results, in order to assess the differences between the patient's gait pattern and that of the healthy population.

[0090] Optionally, after obtaining the kinematic data matrix, the intelligent joint coordination quantitative analysis system can extract and analyze continuous video frame data of the user's walking process from far to near in the video to be analyzed, based on the kinematic data matrix, and extract the user's walking cycle data.

[0091] Furthermore, based on walking cycle data, the user's movement pattern characteristics are extracted through a sliding window. The movement pattern characteristics are compared and analyzed with a database of healthy people to determine the user's standardized reconstruction score. Based on the degree to which the standardized reconstruction score deviates from the baseline value of the reconstruction assessment index, the user's movement pattern abnormality is assessed, and a reconstruction assessment result is generated.

[0092] The smaller the deviation of the standardized reconstruction score from the baseline value of the reconstruction assessment index, the closer the user's gait characteristics are to the normal physiological pattern of healthy people.

[0093] In some embodiments, the joint markers include the left hip joint, right hip joint, left knee joint, and right knee joint; the maximum joint motion angle is determined by the following steps: based on the kinematic data matrix, a first spatial angle is determined between the first joint line and the vertical axis; the first joint line is the line connecting the left hip joint and the left knee joint; based on the kinematic data matrix, a second spatial angle is determined between the second joint line and the vertical axis; the second joint line is the line connecting the right hip joint and the right knee joint; the first spatial angle and the second spatial angle are filtered to determine the maximum joint motion angle.

[0094] The maximum joint motion angle is the maximum motion angle of the hip joint, which is a core indicator for quantifying the sagittal plane motion range of the hip joint when a user walks.

[0095] Specifically, after obtaining the kinematic data matrix, the intelligent joint coordination quantitative analysis system can calculate the first spatial angle between the first joint line and the unit vector (0, 0, -1) of the vertical axis (i.e., the z-axis), and the second spatial angle between the second joint line and the unit vector (0, 0, -1) of the vertical axis (i.e., the z-axis), based on the kinematic data matrix.

[0096] The first joint line is the line connecting the left hip joint and the left knee joint, and the second joint line is the line connecting the right hip joint and the right knee joint.

[0097] Furthermore, within the user's action cycle, the first spatial angle and the second spatial angle are filtered, and the maximum angle value is selected as the maximum joint movement angle.

[0098] Optionally, since the theoretical mean of the normalized maximum joint motion angle is 60°, the intelligent joint coordination quantitative analysis system can set the baseline value corresponding to the theoretical mean of the maximum joint motion angle to 0. After determining the user's maximum joint motion angle, the user's maximum joint motion angle can be mapped to the interval [-1, 1] by the range method, which is convenient for comparison with the baseline value. This value can directly reflect the functional compensation level of the user's two hip joints during the gait propulsion period.

[0099] Furthermore, after determining the user's joint coordination variability score, balance assessment results, stability assessment results, rhythmicity assessment results, reconfiguration assessment results, and maximum joint motion angle, the intelligent joint coordination quantitative analysis system can determine the user's joint coordination quantitative analysis results based on these parameters.

[0100] It should be noted that the video of the user performing the exercise assessment can be transmitted in real time via the Internet to the intelligent joint coordination quantitative analysis system. Rehabilitation physicians can use the cloud analysis platform to dynamically monitor the user's rehabilitation progress and provide personalized professional rehabilitation guidance based on the user's joint coordination quantitative analysis results.

[0101] Furthermore, the intelligent joint collaborative quantitative analysis system is suitable for dual-scenario applications: in the home setting, mobile devices can achieve real-time video capture and human posture tracking with low power consumption through lightweight monocular algorithms; in the medical setting, mobile devices can be high-precision binocular camera systems, which can use 3D reconstruction technology to capture the motion trajectory of human joints and introduce dynamic time warping (DTW) algorithms to achieve cross-modal data alignment. This dual-scenario application of the intelligent joint collaborative quantitative analysis system provides users with a closed-loop assessment covering the entire lifecycle from home screening to professional diagnosis.

[0102] The joint coordination quantitative analysis method and intelligent joint coordination quantitative analysis system provided in this embodiment can significantly reduce equipment complexity and usage threshold while maintaining clinical-grade accuracy, enabling high-quality motor function assessment to extend to community and home settings, and providing users with a brand-new technical solution for stroke rehabilitation.

[0103] This invention also provides a joint coordination quantitative analysis device. Please refer to [link / reference]. Figure 4 , Figure 4 This is a schematic diagram of the joint coordination quantitative analysis device provided by the present invention. In this embodiment, the joint coordination quantitative analysis device includes a coordinate data determination module 410, a kinematic data determination module 420, a coordination variability score determination module 430, and an index evaluation module 440.

[0104] The coordinate data determination module 410 is used to analyze the video to be analyzed based on the BlazePose algorithm to obtain the coordinate data of the joint marker points.

[0105] The videos to be analyzed are those of users performing motion assessment actions.

[0106] The kinematic data determination module 420 is used to determine the user's kinematic data matrix based on coordinate data.

[0107] The co-variance score determination module 430 is used to input the kinematic data matrix into the joint co-attention model and obtain the joint co-variance score output by the joint co-attention model.

[0108] The indicator evaluation module 440 is used to determine the balance evaluation results, stability evaluation results, rhythmic evaluation results, reconfiguration evaluation results, and maximum joint motion angle based on the kinematic data matrix.

[0109] The results of the user's joint coordination quantitative analysis include joint coordination variability score, balance assessment results, stability assessment results, rhythmic assessment results, remodeling assessment results, and maximum joint motion angle.

[0110] In some embodiments, the kinematic data determination module 420 is used to generate time-series trajectory data based on coordinate data by performing kinematic analysis algorithm, and to calculate time-series angle data of joint marker points; and to perform standardization processing on the time-series trajectory data and time-series angle data based on second-order B-spline interpolation algorithm to generate a kinematic data matrix.

[0111] In some embodiments, the joint coordination attention model includes a concatenated attention module, a graph convolutional network pre-classification module, and a cyclic masking module; the concatenated attention module is used to concatenate features of the kinematic data matrix to generate concatenated features carrying attention weights; the graph convolutional network pre-classification module is used to generate user pre-classification results based on the concatenated features; and the cyclic masking module is used to generate joint coordination variability scores based on the concatenated features and the user pre-classification results.

[0112] In some embodiments, the joint markers include the left hip joint and the right hip joint; the balance assessment result is determined by the following steps: determining a first dynamic trajectory of the angle between the left hip joint and the horizontal axis based on the kinematic data matrix; determining a second dynamic trajectory of the angle between the right hip joint and the horizontal axis based on the kinematic data matrix; and determining the balance assessment result based on the first dynamic trajectory and the second dynamic trajectory.

[0113] In some embodiments, the joint markers include the left hip joint and the right hip joint; the stability assessment result is determined by the following steps: determining the third dynamic trajectory of the user's center of gravity in the sagittal plane based on the kinematic data matrix; the user's center of gravity is the midpoint of the line connecting the left hip joint and the right hip joint; determining the standard deviation of the center of gravity trajectory deviation of the third dynamic trajectory; and determining the stability assessment result based on the standard deviation of the center of gravity trajectory deviation.

[0114] In some embodiments, the rhythmicity assessment result is determined by the following steps: determining the dynamic trajectory of the left foot and the dynamic trajectory of the right foot based on the kinematic data matrix; determining the user's gait cycle based on the dynamic trajectory of the left foot and the dynamic trajectory of the right foot; aligning the dynamic trajectory of the left foot and the dynamic trajectory of the right foot based on the dynamic time warping algorithm and determining the variance of the gait cycle; and determining the rhythmicity assessment result based on the variance.

[0115] In some embodiments, the reconstructive assessment result is determined by the following steps: obtaining a spatiotemporal vector matrix through nonnegative matrix decomposition based on the kinematic data matrix; generating a reconstructive kinematic data matrix based on the spatiotemporal vector matrix; determining the user's reconstructive percentage based on the reconstructive kinematic data matrix and the kinematic data matrix; and comparing the reconstructive percentage with the reconstructive percentage of healthy individuals to determine the reconstructive assessment result.

[0116] In some embodiments, the joint markers include the left hip joint, right hip joint, left knee joint, and right knee joint; the maximum joint motion angle is determined by the following steps: based on the kinematic data matrix, a first spatial angle is determined between the first joint line and the vertical axis; the first joint line is the line connecting the left hip joint and the left knee joint; based on the kinematic data matrix, a second spatial angle is determined between the second joint line and the vertical axis; the second joint line is the line connecting the right hip joint and the right knee joint; the first spatial angle and the second spatial angle are filtered to determine the maximum joint motion angle.

[0117] The present invention also provides an electronic device. Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 5 As shown, the electronic device may include a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute the joint co-quantization analysis method.

[0118] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0119] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0120] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for quantitative analysis of joint coordination, characterized in that, include: Based on the BlazePose algorithm, the video to be analyzed is analyzed to obtain the coordinate data of the joint marker points; The video to be analyzed is a video of the user performing motion assessment actions; Based on the coordinate data, determine the user's kinematic data matrix; The kinematic data matrix is ​​input into the joint coordination attention model to obtain the joint coordination variability score output by the joint coordination attention model; Based on the kinematic data matrix, the balance assessment results, stability assessment results, rhythmic assessment results, remodeling assessment results, and maximum joint motion angle are determined. The user's joint coordination quantitative analysis results include the joint coordination variability score, the balance assessment result, the stability assessment result, the rhythmicity assessment result, the reconfigurability assessment result, and the maximum joint motion angle; The joint collaborative attention model includes a concatenated attention module, a graph convolutional network pre-classification module, and a cyclic masking module. The concatenated attention module is used to concatenate features of the kinematic data matrix to generate concatenated features carrying attention weights. The graph convolutional network pre-classification module is used to generate user pre-classification results based on the concatenated features; The cyclic mask module is used to generate the joint covariance score based on the concatenated features and the user pre-classification result; The reconstructive evaluation results were determined through the following steps: Based on the kinematic data matrix, the spatiotemporal vector matrix is ​​obtained through nonnegative matrix decomposition. Based on the spatiotemporal vector matrix, a reconstructed kinematic data matrix is ​​generated; Based on the reconstructed kinematic data matrix and the kinematic data matrix, the reconstructed percentage of the user is determined; The reconstruction percentage is compared with the reconstruction percentage of the healthy population to determine the reconstructive assessment result.

2. The joint coordination quantitative analysis method according to claim 1, characterized in that, Determining the user's kinematic data matrix based on the coordinate data includes: Based on the coordinate data, kinematic analysis algorithm is used to analyze the data, generate temporal trajectory data, and calculate the temporal angle data of the joint marker points; Based on the second-order B-spline interpolation algorithm, the time-series trajectory data and the time-series angle data are standardized to generate the kinematic data matrix.

3. The joint coordination quantitative analysis method according to claim 1, characterized in that, The joint markers include the left hip joint and the right hip joint; The balance assessment result is determined through the following steps: Based on the kinematic data matrix, the first dynamic trajectory of the angle between the left hip joint and the horizontal axis is determined; Based on the kinematic data matrix, a second dynamic trajectory of the angle between the right hip joint and the horizontal axis is determined; The balance assessment result is determined based on the first dynamic trajectory and the second dynamic trajectory.

4. The joint coordination quantitative analysis method according to claim 1, characterized in that, The joint markers include the left hip joint and the right hip joint; The stability assessment results were determined through the following steps: Based on the kinematic data matrix, a third dynamic trajectory of the user's center of gravity in the sagittal plane is determined; the user's center of gravity is the midpoint of the line connecting the left hip joint and the right hip joint. Determine the standard deviation of the centroid trajectory offset of the third dynamic trajectory; The stability assessment result is determined based on the standard deviation of the center of gravity trajectory offset.

5. The joint coordination quantitative analysis method according to claim 1, characterized in that, The rhythmicity assessment results were determined through the following steps: Based on the kinematic data matrix, the dynamic trajectories of the left and right feet are determined. Based on the dynamic trajectory of the left foot and the dynamic trajectory of the right foot, the user's gait cycle is determined; Based on the dynamic time warping algorithm, the dynamic trajectory of the left foot and the dynamic trajectory of the right foot are aligned, and the variance of the gait cycle is determined. The rhythmicity assessment result is determined based on the variance value.

6. The joint coordination quantitative analysis method according to claim 1, characterized in that, The joint markers include the left hip joint, right hip joint, left knee joint, and right knee joint; The maximum joint motion angle is determined through the following steps: Based on the kinematic data matrix, a first spatial angle is determined between the first joint line and the vertical axis; the first joint line is the line connecting the left hip joint and the left knee joint. Based on the kinematic data matrix, a second spatial angle is determined between the second joint line and the vertical axis; the second joint line is the line connecting the right hip joint and the right knee joint. The first spatial angle and the second spatial angle are filtered to determine the maximum joint motion angle.

7. A joint coordination quantitative analysis device, characterized in that, include: The coordinate data determination module is used to analyze the video to be analyzed based on the BlazePose algorithm and obtain the coordinate data of the joint marker points. The video to be analyzed is a video of the user performing motion assessment actions; A kinematic data determination module is used to determine the user's kinematic data matrix based on the coordinate data; The co-variance score determination module is used to input the kinematic data matrix into the joint co-attention model and obtain the joint co-variance score output by the joint co-attention model. The index evaluation module is used to determine the balance evaluation result, stability evaluation result, rhythmic evaluation result, reconfiguration evaluation result, and maximum joint motion angle based on the kinematic data matrix. The user's joint coordination quantitative analysis results include the joint coordination variability score, the balance assessment result, the stability assessment result, the rhythmicity assessment result, the reconfigurability assessment result, and the maximum joint motion angle; The joint collaborative attention model includes a concatenated attention module, a graph convolutional network pre-classification module, and a cyclic masking module. The concatenated attention module is used to concatenate features of the kinematic data matrix to generate concatenated features carrying attention weights. The graph convolutional network pre-classification module is used to generate user pre-classification results based on the concatenated features; The cyclic mask module is used to generate the joint covariance score based on the concatenated features and the user pre-classification result; The reconstructive evaluation result is determined through the following steps: obtaining the spatiotemporal vector matrix through nonnegative matrix decomposition based on the kinematic data matrix; Based on the spatiotemporal vector matrix, a reconstructed kinematic data matrix is ​​generated; Based on the reconstructed kinematic data matrix and the kinematic data matrix, the reconstructed percentage of the user is determined; The reconstruction percentage is compared with the reconstruction percentage of the healthy population to determine the reconstructive assessment result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the joint coordination quantitative analysis method as described in any one of claims 1 to 6.

Images