Rehabilitation evaluation system and method based on data feedback

Abstract

The invention discloses a rehabilitation evaluation system and method based on data feedback, and the method comprises the steps: obtaining a continuous online rehabilitation video as rehabilitation training video frame data, and constructing a rehabilitation training motion data set; extracting joint angle features and skeleton point distance features from the action data set by adopting a label classification technology, and obtaining distance measurement of each human body joint skeleton point coordinate; matching the time sequence with the action features by adopting a dynamic adjustment algorithm for the action feature combination, and constructing a classification evaluation model of rehabilitation training actions; a data feedback mechanism is introduced into the classification evaluation model of the rehabilitation training actions, the relation between rehabilitation requirements and rehabilitation evaluation is mapped, the whole system achieves detailed analysis and dynamic adaptation of the rehabilitation training actions, accurate rehabilitation suggestions can be provided according to individual differences, the effectiveness of a treatment strategy is ensured through real-time monitoring and adjustment, and the rehabilitation training effect is improved. And the rehabilitation effect and efficiency are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of rehabilitation assessment technology, and specifically to a data feedback-based rehabilitation assessment system and method. Background Technology

[0002] As hospitals expand and the number of patients increases, the complexity of medical data management gradually grows. Traditional medical data often relies on manual recording, leading to incomplete or biased information and reduced work efficiency. To address this issue, rehabilitation assessment systems have been introduced into the healthcare industry, aiming to optimize medical services by accurately predicting patient preferences and behaviors. Although assessment systems have been widely used in various industries, they still have shortcomings in handling the sparsity and discriminability of medical data, especially in the field of rehabilitation medicine. Different patients often show only minor differences on the same evaluation indicators, which can represent significant medical distinctions. Currently, rehabilitation assessment systems and methods used in the field of rehabilitation medicine still have the following problems: (1) Current rehabilitation assessment systems often use a uniform standard to assess all patients, which makes it difficult to fully consider individual differences, such as age, gender, disease type and severity, resulting in rehabilitation plans that are not precise and effective and cannot meet the specific needs of different patients. (2) The data sources involved in the rehabilitation process are diverse, and effective rehabilitation training requires guidance from professional medical staff. Guiding people to complete rehabilitation training requires a large amount of medical resources, and the existing medical conditions cannot meet the ever-increasing demand for rehabilitation training. Summary of the Invention

[0003] The purpose of this invention is to provide a data-feedback-based rehabilitation assessment system and method to solve the technical problems in the prior art where rehabilitation programs are not precise and effective enough and cannot meet the specific needs of different patients.

[0004] To solve the above-mentioned technical problems, the present invention specifically provides the following technical solution: A first aspect of the present invention provides a data-driven rehabilitation assessment method, comprising the following steps: Continuous online rehabilitation videos are acquired as rehabilitation training video frame data. The video frame data is segmented into rehabilitation images. The rehabilitation images are processed using a rehabilitation posture estimation algorithm to obtain the coordinates of human joint bones. Based on the time series arrangement of the video frame data, a rehabilitation training action dataset is constructed. The action dataset is subjected to label classification technology to extract joint angle features and bone point distance features, and the distance metric of the coordinates of each human joint bone point is obtained. A classifier is trained separately for each distance metric to obtain the action feature combination of each bone point. A dynamic adjustment algorithm is used to match the time series with the action features to obtain an action feature matrix. The similarity of the action feature matrix is ​​calculated, and a classification and evaluation model for rehabilitation training actions is constructed to evaluate the rehabilitation training actions. The classification and evaluation model for the rehabilitation training movements introduces a data feedback mechanism. The distance metric is adjusted through a multidimensional feedback function, and a metric learning algorithm is used to convert the distance metric into a probability value. Based on the probability value, a rehabilitation recommendation decision is formed, mapping the relationship between rehabilitation needs and rehabilitation assessment.

[0005] In a preferred embodiment of the present invention, continuous online rehabilitation videos are acquired as rehabilitation training video frame data. The video frame data is then segmented into rehabilitation images. A rehabilitation posture estimation algorithm is used to process the rehabilitation images to obtain the coordinates of human joint and bone points. Based on the time-series arrangement of the video frame data, a rehabilitation training motion dataset is constructed, including: Image frames are extracted frame by frame from the online rehabilitation video, and each image frame is regarded as a rehabilitation picture. The BlaezPose algorithm for human pose estimation is used to obtain the skeletal point information of the rehabilitation patient, and the skeletal point coordinate information of the patient in the current state of each frame is obtained. , This represents the coordinates of the i-th skeletal point. and Let these represent the horizontal and vertical positions of the i-th bone point in the current image frame, respectively. Indicates the visibility of the i-th skeletal keypoint in the current image frame; The skeletal point coordinate information is used to predict and correct the trajectory based on the time series, and the coordinates of key points with timestamps are extracted; The coordinates of the key points are represented as a two-dimensional matrix. , The matrix representing the skeletal point information of the patient's pose in the j-th frame image. The information vector representing the first to nth skeletal points of the patient in the j-th frame image; The BlaezPose algorithm is used to process the two-dimensional matrix. Acquire the patient's movement data from the entire rehabilitation training video, and represent the movement data as a three-dimensional matrix. ; For the three-dimensional matrix Based on the key point coordinate data of the time series, action features are extracted to obtain an action dataset.

[0006] As a preferred embodiment of the present invention, the motion dataset is processed using label classification technology to extract joint angle features and bone point distance features, including: Extract the key point coordinate data from the motion dataset, calculate its bending angle to obtain joint angle data, and extract joint angle features based on the joint angle data. Calculate the Euclidean distance between any two key points from the key point coordinate data in the action dataset to obtain the bone point distance feature; Support Vector Machine (SVM) is used to classify and label the joint angle features and bone point distance features, and a clustering algorithm is used to obtain the label dataset of the action dataset.

[0007] As a preferred embodiment of the present invention, a distance metric is obtained for the coordinates of each human joint skeletal point through the labeled dataset, and a classifier is trained separately for each distance metric to obtain a combination of motion features for each skeletal point, including: The distance features of skeletal points and joint angle features within the labeled dataset are detected. The maximum cross-validation function is used to evaluate the matching degree between the keypoint coordinate data and the ground truth labeled keypoints. The expression is:

[0008] in, Indicates the first One key point, Indicates the estimation of key points and the real key points The Euclidean distance between them This represents the standard deviation of the distances between skeletal points in the labeled dataset. Indicates the true key points The scale factor, Indicates the true key points Visibility level Indicates an indicator function, when Returns 1 if the condition is met, otherwise returns 0. According to the maximum cross-evaluation function Filter the key point coordinate data to obtain the distance metric of the coordinates of each human joint bone point; The distance metric is trained separately using a logistic regression model, and the similarity metric is output from the logistic regression model as a feature to obtain the action feature combination of each skeletal point.

[0009] As a preferred embodiment of the present invention, a dynamic adjustment algorithm is used to match the time series with the action features to obtain an action feature matrix, including: Randomly select two time series and , in sequence and This represents the normalized distance feature of the time-stamped keypoint coordinates in the action feature combination on a two-dimensional plane, where n represents the sequence. Length, These represent the standardized distance values ​​of the 1st, 2nd, and nth joint angle features with timestamps in the motion feature combination, respectively. These represent the standardized distance values ​​of the 1st, 2nd, and nth skeletal points with timestamps in the action feature combination, respectively. in sequence and This represents the normalized distance feature of keypoint coordinate data in a standard template on a two-dimensional plane, where m represents the sequence. Length, Let represent the standardized distance values ​​of the 1st, 2nd, and mth joint angle features, respectively. These represent the standardized distance values ​​of the distance features of the 1st, 2nd, and mth skeletal points, respectively. Define a [amount] based on two action features within the same time series. A dynamic three-dimensional matrix D and a Given a two-dimensional matrix L, calculate the time series under the same time series. and The combined distance in the middle is expressed as:

[0010] in, This represents the relationship between the i-th element in sequence M and the sequence. The distance between the j-th elements in the middle. Represents the relationship between the i-th element in sequence N and the sequence. The distance between the j-th elements in the middle. Represents a sequence The i-th element and the sequence The combined distance between the j-th elements in the array. The weights represent the distance between the angular feature matches. The weights represent the standardized distance features of the skeleton points on the two-dimensional matrix L plane; The two-dimensional matrix is ​​calculated using a recursive relationship. For all elements in the matrix, obtain the optimal combination distance, and use the sequence corresponding to the optimal combination distance as the action feature matrix.

[0011] As a preferred embodiment of the present invention, the similarity of the action feature matrix is ​​calculated, a classification and evaluation model for rehabilitation training actions is constructed, and the rehabilitation training actions are evaluated, including: By calculating the time series and The Euclidean distance is used to compare the similarity distance d between the movement characteristics of rehabilitation training data and the movement characteristics of standard templates. The action similarity is calculated using a scoring function based on the similarity distance d. Its expression is:

[0012] in, This represents the exponential operation with the natural constant e as the base and the similarity distance d as the exponent; Based on the action similarity Extract the key point coordinate data of the rehabilitation training data, obtain the skeletal point coordinate information of each frame of rehabilitation training data, and extract the rehabilitation training action features based on the skeletal point coordinate information. By acquiring the range of motion and duration of motion of multiple training limbs through motion characteristics, the multi-limb training movements are classified and evaluated. The duration of the motion state during rehabilitation training is recorded, and a classification and evaluation model of rehabilitation training movements is constructed for multi-limb movements.

[0013] As a preferred embodiment of the present invention, the classification and evaluation model is trained on the motion feature data using an LSTM deep learning model to evaluate rehabilitation training movements, including: The time series of the action feature matrix is ​​divided into several time units t using an LSTM deep learning model. For any time unit t, the output of the LSTM deep learning model in the previous time unit t-1 is... As input, construct the unit state function Its expression is:

[0014] in, , , This represents the parameters of the LSTM deep learning model. This indicates the relationship between input and output. Indicates the correlation of historical data. Indicates the offset value. This represents the state value of the current candidate time unit. This represents the output of the LSTM deep learning model at the current time state; According to the unit state function Through the above Adjust the parameters of the LSTM network model, process the time series information and action features of continuous action features in rehabilitation training, and obtain the weights of different action features in the classification unit; Based on the weights of different action features in the classification unit, key nodes and other nodes are established in the action feature matrix. Using the LSTM deep learning model to train rehabilitation data, all key nodes and other nodes are strongly connected. The connection paths of the key nodes are evaluated to obtain evaluation values ​​for different types of rehabilitation training actions.

[0015] As a preferred embodiment of the present invention, a data feedback mechanism is introduced into the classification and evaluation model of the rehabilitation training movements, and the distance metric is adjusted through a multidimensional feedback function, including: The classification and assessment model is used to construct a binary interaction matrix between patients and rehabilitation programs. For the binary interaction matrix Construct the distance matrix Its expression is:

[0016] in, Representing a binary interaction matrix The elements in Indicates the patient, Indicates rehabilitation program, Indicates the patient With rehabilitation projects The distance between them; Grey relational analysis was used to analyze the distance matrix. The analysis was conducted to obtain similarities between patient characteristics and rehabilitation programs; For the distance matrix A feedback adjustment function is introduced to feed patient training data back to the classification and evaluation model in a timely manner, adjusting the distance metric in real time. The expression for the feedback adjustment function is:

[0017] in, , These represent the locations of the recovering patient and the recovery program, respectively, in terms of distance measurement. This indicates the distance between the recovering patient and the recovery program. This indicates the degree of similarity between patient characteristics and rehabilitation programs. This indicates the distance between the recovering patient and the recovery program after the feedback adjustment.

[0018] As a preferred embodiment of the present invention, a metric learning algorithm is used to convert the distance metric into a probability value, and a rehabilitation recommendation decision is formed based on the probability value, mapping the relationship between rehabilitation needs and rehabilitation assessment, including: The distance metric is optimized using a loss function to obtain the difference between the rehabilitation training data and the standard sample data. The expression for the loss function is:

[0019] in, This represents a feature mapping function that maps the raw data from rehabilitation training to a feature space. , and Let represent the anchor point, positive sample, and negative sample of the i-th rehabilitation training sample in the distance metric, respectively. This means mapping the anchor point data of the i-th rehabilitation training sample in the distance metric to the feature space. This represents mapping the positive sample of the i-th rehabilitation training sample in the distance metric to the feature space. This means mapping the negative samples of the i-th rehabilitation training sample in the distance metric to the feature space. Indicates the preset level value; The loss function is used to remove samples in the distance metric where the rehabilitation training data and the standard sample data have large differences. The distance metric between the anchor point and the positive and negative samples is compared to drive the classification and evaluation model to perform iterative training. The probability of generating data with a small difference between the rehabilitation training data and the standard sample data is calculated, and a rehabilitation recommendation decision is formed based on the probability, mapping the relationship between rehabilitation needs and rehabilitation assessment.

[0020] A second aspect of the present invention provides a data-feedback-based rehabilitation assessment system for implementing a data-feedback-based rehabilitation assessment method. The system includes: The data acquisition module acquires real-time rehabilitation training video data and collects the physiological and motor parameters of rehabilitation patients in real time. The data processing module processes the physiological and motor parameters of the rehabilitation patients, extracts the rehabilitation patients' training data, and generates a distance matrix between the rehabilitation patients' training data and standard data based on the time series. The feedback adjustment module establishes a rehabilitation assessment model, monitors the training data of rehabilitation patients in real time, and adjusts the training according to the distance matrix through a feedback adjustment function. The user interface displays rehabilitation status assessment reports and provides personalized rehabilitation suggestions to patients.

[0021] Compared with the prior art, the present invention has the following advantages: This invention employs the BlaezPose algorithm for human posture estimation, from acquiring continuous online rehabilitation videos to extracting key point coordinate information and calculating joint angle features and bone point distance features. This ensures comprehensive capture and detailed analysis of rehabilitation training movements. By introducing time series data to dynamically adapt to changes in the patient's rehabilitation progress, it provides more accurate movement features. Furthermore, by utilizing label classification technology and metric learning algorithms to fully consider the individual differences of each patient, it formulates the most suitable rehabilitation plan for each individual's situation. Through a real-time feedback mechanism, it continuously adjusts distance metrics and rehabilitation suggestions, ensuring that the rehabilitation plan always aligns with the patient's latest needs and development trends. Combining binary interaction matrices and distance matrices, and applying feedback adjustment functions, it achieves intelligent monitoring and real-time feedback on the rehabilitation training effect, facilitating timely adjustments to treatment strategies and greatly improving rehabilitation effectiveness and efficiency. Moreover, the entire system, from data acquisition to final rehabilitation recommendation decisions, is fully automated, reducing the need for manual intervention and improving work efficiency and service quality. Attached Figure Description

[0022] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0023] Figure 1 A flowchart of a data feedback-based rehabilitation assessment method provided in an embodiment of the present invention; Figure 2 A block diagram of a data feedback-based rehabilitation assessment system provided in an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] like Figure 1 and Figure 2 As shown, this invention provides a data-driven rehabilitation assessment method, comprising the following steps: Continuous online rehabilitation videos are acquired as rehabilitation training video frame data. The video frame data is segmented into rehabilitation images. The rehabilitation images are processed using a rehabilitation posture estimation algorithm to obtain the coordinates of human joint bones. Based on the time series arrangement of the video frame data, a rehabilitation training action dataset is constructed. In this embodiment, by extracting keyframes from continuous rehabilitation videos and using a rehabilitation posture estimation algorithm to determine the positions of human joints and skeletal points, detailed and accurate analysis of human movements can be achieved. This helps to identify the specific movement patterns of patients during rehabilitation training and can also monitor subtle changes in movement.

[0026] In this embodiment, the rehabilitation training action dataset constructed based on time series arrangement can capture the changes and development trends of actions during the rehabilitation process. This dynamic data collection method allows for comprehensive tracking of rehabilitation progress, thereby better understanding the patient's rehabilitation process and adjusting the rehabilitation plan in a timely manner.

[0027] The action dataset is subjected to label classification technology to extract joint angle features and bone point distance features, and the distance metric of the coordinates of each human joint bone point is obtained. A classifier is trained separately for each distance metric to obtain the action feature combination of each bone point. In this embodiment, by extracting joint angle features and bone point distance features, the specific details of human movement during rehabilitation training can be captured more accurately. This not only identifies the key features of different rehabilitation stages, but also effectively distinguishes between normal and abnormal rehabilitation movement patterns.

[0028] In this embodiment, a classifier is trained separately for each distance metric, enabling the system to process the data of each skeletal point in a customized manner according to individual differences. This makes the final combination of motion features more closely match the specific situation of each patient, which helps to develop a more personalized rehabilitation plan. The rehabilitation strategy can be adjusted according to the patient's special needs, thereby improving the rehabilitation effect.

[0029] A dynamic adjustment algorithm is used to match the time series with the action features to obtain an action feature matrix. The similarity of the action feature matrix is ​​calculated, and a classification and evaluation model for rehabilitation training actions is constructed to evaluate the rehabilitation training actions. In this embodiment, by matching time series data with motion features through a dynamic adjustment algorithm, the specific patterns of each motion change over time during rehabilitation training can be captured. This allows for more accurate identification of the characteristics and evolution trends of the motions performed by patients at different rehabilitation stages, thereby providing a scientific basis for formulating or adjusting rehabilitation plans.

[0030] In this embodiment, calculating the similarity of the action feature matrix helps to quantify the similarity between different rehabilitation actions. The classification and evaluation model constructed based on the similarity information can more accurately evaluate the effect of rehabilitation training and ensure the objectivity and reliability of the evaluation results.

[0031] In this embodiment, by utilizing the results of motion feature matrix and similarity analysis, the system can dynamically adjust the rehabilitation plan according to the rehabilitation progress of each patient. It can not only formulate a preliminary rehabilitation plan based on the initial assessment results, but also continuously optimize the treatment measures as the rehabilitation process progresses, so as to better meet individual needs and improve rehabilitation effects.

[0032] The classification and evaluation model for the rehabilitation training movements introduces a data feedback mechanism. The distance metric is adjusted through a multidimensional feedback function, and a metric learning algorithm is used to convert the distance metric into a probability value. Based on the probability value, a rehabilitation recommendation decision is formed, mapping the relationship between rehabilitation needs and rehabilitation assessment.

[0033] In this embodiment, a multidimensional feedback function is used to adjust the distance metric, enabling the system to comprehensively consider multiple factors. This helps to develop a more comprehensive and detailed rehabilitation plan. At the same time, the entire rehabilitation process is adjusted based on real-time feedback, so it can quickly respond to the patient's rehabilitation progress or any changes, ensuring that the rehabilitation plan always meets the patient's actual needs.

[0034] In this embodiment, by converting distance metrics into probability values ​​to form rehabilitation recommendation decisions, the system can not only determine whether rehabilitation goals have been achieved, but also predict possible future trends, providing a basis for formulating the next stage of rehabilitation plans. This helps to identify potential problems in advance, adjust treatment strategies in a timely manner, and thus continuously optimize rehabilitation outcomes.

[0035] Continuous online rehabilitation videos are acquired as rehabilitation training video frame data. The video frame data is then segmented into rehabilitation images. A rehabilitation posture estimation algorithm is used to process the rehabilitation images to obtain the coordinates of human joint bones. Based on the time-series arrangement of the video frame data, a rehabilitation training motion dataset is constructed, including: Image frames are extracted frame by frame from the online rehabilitation video, and each image frame is regarded as a rehabilitation picture. The BlaezPose algorithm for human pose estimation is used to obtain the skeletal point information of the rehabilitation patient, and the skeletal point coordinate information of the patient in the current state of each frame is obtained. , This represents the coordinates of the i-th skeletal point. and Let these represent the horizontal and vertical positions of the i-th bone point in the current image frame, respectively. Indicates the visibility of the i-th skeletal keypoint in the current image frame; In this embodiment, by extracting images frame by frame and treating each image frame as a rehabilitation image, extremely detailed capture of human posture can be achieved. The BlaezPose human posture estimation algorithm can accurately identify the skeletal information of rehabilitation patients, including the position and visibility of each key point, ensuring the capture of even the most subtle changes in movement.

[0036] The skeletal point coordinate information is used to predict and correct the trajectory based on the time series, and the coordinates of key points with timestamps are extracted; In this embodiment, trajectory prediction and correction of skeletal point coordinate information based on time series not only provides a smoother and more coherent motion trajectory, but also improves the accuracy of motion feature extraction. The time-based data processing method enables the system to better understand the development trend and dynamic characteristics of the motion, which helps to discover potential problems.

[0037] The coordinates of the key points are represented as a two-dimensional matrix. , The matrix representing the skeletal point information of the patient's pose in the j-th frame image. The information vector representing the first to nth skeletal points of the patient in the j-th frame image; The BlaezPose algorithm is used to process the two-dimensional matrix. Acquire the patient's movement data from the entire rehabilitation training video, and represent the movement data as a three-dimensional matrix. ; For the three-dimensional matrix Based on the key point coordinate data of the time series, action features are extracted to obtain an action dataset.

[0038] In this embodiment, by comprehensively collecting and analyzing the motion data throughout the entire rehabilitation training process, a personalized rehabilitation plan can be tailored to each patient's specific situation. This not only improves the relevance and effectiveness of the rehabilitation plan but also enhances the patient's sense of participation and treatment compliance.

[0039] The motion dataset is subjected to label classification technology to extract joint angle features and bone point distance features, including: Extract the key point coordinate data from the motion dataset, calculate its bending angle to obtain joint angle data, and extract joint angle features based on the joint angle data. In this embodiment, joint angle data is obtained by calculating the bending angle between key point coordinate data, which can accurately capture the specific movement of each joint in the human body during rehabilitation training.

[0040] Calculate the Euclidean distance between any two key points from the key point coordinate data in the action dataset to obtain the bone point distance feature; In this embodiment, the Euclidean distance between any two key points is calculated to obtain the skeletal point distance features, which helps to obtain the relative positional relationships and changes between different body parts.

[0041] Support Vector Machine (SVM) is used to classify and label the joint angle features and bone point distance features, and a clustering algorithm is used to obtain the label dataset of the action dataset.

[0042] In this embodiment, using Support Vector Machine (SVM) for classification and labeling can effectively divide complex action datasets into different categories or states. Further processing of these feature data through clustering algorithms can help discover patterns and structures in unlabeled data, automatically forming labeled datasets. This not only improves classification accuracy but also allows the system to learn from a large amount of unlabeled data, increasing the model's generalization ability.

[0043] The distance metric for the coordinates of each human joint skeletal point is obtained from the labeled dataset. A classifier is trained separately for each distance metric to obtain a combination of motion features for each skeletal point, including: The distance features of skeletal points and joint angle features within the labeled dataset are detected. The maximum cross-validation function is used to evaluate the matching degree between the keypoint coordinate data and the ground truth labeled keypoints. The expression is:

[0044] in, Indicates the first One key point, Indicates the estimation of key points and the real key points The Euclidean distance between them This represents the standard deviation of the distances between skeletal points in the labeled dataset. Indicates the true key points The scale factor, Indicates the true key points Visibility level Indicates an indicator function, when Returns 1 if the condition is met, otherwise returns 0. In this embodiment, the maximum cross-validation function is used. Assessing the degree of matching between keypoint coordinate data and actual labeled keypoints can more accurately measure the difference between estimated keypoints and the actual situation. It not only considers Euclidean distance, but also incorporates multi-dimensional information such as standard deviation, scale factor and visibility, thereby improving the accuracy of matching.

[0045] According to the maximum cross-evaluation function Filter the key point coordinate data to obtain the distance metric of the coordinates of each human joint bone point; The distance metric is trained separately using a logistic regression model, and the similarity metric is output from the logistic regression model as a feature to obtain the action feature combination of each skeletal point.

[0046] In this embodiment, by training a classifier separately for each distance metric, the system can generate a model specifically tailored to the characteristics of each skeletal point. This personalized approach enhances the model's adaptability to different data characteristics and maintains high accuracy even when faced with noise or outliers in the data, thereby increasing the robustness of the entire system.

[0047] In this embodiment, the similarity metric output by the logistic regression model is used as a feature, so that the combination of motion features of each skeletal point is more in line with the actual situation. These features can not only reflect the specific motion state of each joint in the individual's rehabilitation training, but also capture subtle changing trends, which helps to understand the rehabilitation effect and progress more deeply.

[0048] A dynamic adjustment algorithm is used to match the time series with the action features to obtain an action feature matrix, including: Randomly select two time series and , in sequence and This represents the normalized distance feature of the time-stamped keypoint coordinates in the action feature combination on a two-dimensional plane, where n represents the sequence. Length, These represent the standardized distance values ​​of the 1st, 2nd, and nth joint angle features with timestamps in the motion feature combination, respectively. These represent the standardized distance values ​​of the 1st, 2nd, and nth skeletal points with timestamps in the action feature combination, respectively. in sequence and This represents the normalized distance feature of keypoint coordinate data in a standard template on a two-dimensional plane, where m represents the sequence. Length, Let represent the standardized distance values ​​of the 1st, 2nd, and mth joint angle features, respectively. These represent the standardized distance values ​​of the distance features of the 1st, 2nd, and mth skeletal points, respectively. In this embodiment, different time series can be randomly selected for comparison. This flexibility allows the system to adapt to various rehabilitation training scenarios and individual differences.

[0049] Define a [amount] based on two action features within the same time series. A dynamic three-dimensional matrix D and a Given a two-dimensional matrix L, calculate the time series under the same time series. and The combined distance in the middle is expressed as:

[0050] in, This represents the relationship between the i-th element in sequence M and the sequence. The distance between the j-th elements in the middle. Represents the relationship between the i-th element in sequence N and the sequence. The distance between the j-th elements in the middle. Represents a sequence The i-th element and the sequence The combined distance between the j-th elements in the array. The weights represent the distance between the angular feature matches. The weights represent the standardized distance features of the skeleton points on the two-dimensional matrix L plane; In this embodiment, the combined distance between two action features in the same time series is calculated using a three-dimensional matrix D and a two-dimensional matrix L. This method can capture the temporal changes and spatial relationships between different action features in detail. It not only considers the standardized distance feature but also combines the angular feature matching distance, thereby more comprehensively and accurately evaluating the similarity between actions.

[0051] The two-dimensional matrix is ​​calculated using a recursive relationship. For all elements in the matrix, obtain the optimal combination distance, and use the sequence corresponding to the optimal combination distance as the action feature matrix.

[0052] In this embodiment, by calculating all elements in the two-dimensional matrix through recursion, the optimal combination distance can be found. This provides strong support for processing complex and ever-changing motion data and enhances the system's adaptability and flexibility.

[0053] Calculate the similarity of the movement feature matrix, construct a classification and evaluation model for rehabilitation training movements, and evaluate the rehabilitation training movements, including: By calculating the time series and The Euclidean distance is used to compare the similarity distance d between the movement characteristics of rehabilitation training data and the movement characteristics of standard templates. In this embodiment, the similarity between rehabilitation training data and standard template movement features is compared by calculating the Euclidean distance between time series, thereby providing a quantitative indicator, similarity distance d, to accurately measure the gap between the patient's performed movement and the ideal pattern, which helps to accurately identify the specific aspects that need improvement.

[0054] The action similarity is calculated using a scoring function based on the similarity distance d. Its expression is:

[0055] in, This represents the exponential operation with the natural constant e as the base and the similarity distance d as the exponent; Based on the action similarity Extract the key point coordinate data of the rehabilitation training data, obtain the skeletal point coordinate information of each frame of rehabilitation training data, and extract the rehabilitation training action features based on the skeletal point coordinate information. In this embodiment, action similarity is used. By conducting detailed analysis of movement characteristics and classifying and evaluating multi-limb training movements, the system can provide personalized rehabilitation suggestions based on each patient's actual situation. For example, it can adjust the training intensity, frequency, or focus on exercising certain specific movements for different parts or types of rehabilitation needs, ensuring that each patient can obtain the most suitable treatment plan for themselves.

[0056] By acquiring the range of motion and duration of motion of multiple training limbs through motion characteristics, the multi-limb training movements are classified and evaluated. The duration of the motion state during rehabilitation training is recorded, and a classification and evaluation model of rehabilitation training movements is constructed for multi-limb movements.

[0057] In this embodiment, the characteristics of rehabilitation training movements are extracted and the range and duration of movement of multiple limbs are analyzed. This not only focuses on the completion of the movements, but also considers the speed and fluency of the movements, providing a basis for a comprehensive evaluation of the rehabilitation effect.

[0058] The classification and evaluation model uses an LSTM deep learning model to train on motion feature data and evaluates rehabilitation training movements, including: The time series of the action feature matrix is ​​divided into several time units t using an LSTM deep learning model. For any time unit t, the output of the LSTM deep learning model in the previous time unit t-1 is... As input, construct the unit state function Its expression is:

[0059] in, , , This represents the parameters of the LSTM deep learning model. This indicates the relationship between input and output. Indicates the correlation of historical data. Indicates the offset value. This represents the state value of the current candidate time unit. This represents the output of the LSTM deep learning model at the current time state; In this embodiment, by dividing the time series of the action feature matrix into several time units t, and using the output of the previous time unit t-1 as the input of the current time unit t, the dynamic characteristics of rehabilitation training actions changing over time can be effectively captured, and the development trend of the actions and their complex time dependencies can be better understood.

[0060] In this embodiment, the LSTM deep learning model, through its unique unit state function, including input gate, forget gate and output gate mechanism, can selectively remember or forget historical information, thereby maintaining high accuracy and stability when processing long-term sequences. This enables LSTM to take into account movement patterns over a long time span when analyzing rehabilitation training movements, improving the understanding and classification ability of complex movement sequences.

[0061] According to the unit state function Through the above Adjust the parameters of the LSTM network model, process the time series information and action features of continuous action features in rehabilitation training, and obtain the weights of different action features in the classification unit; In this embodiment, by adjusting the parameters of the LSTM network model to process the time-series information of continuous movement features in rehabilitation training, the weights of different movement features in the classification unit can be obtained. These weights reflect the importance of each movement feature to the rehabilitation effect, which helps to formulate personalized rehabilitation plans and provide targeted guidance to patients.

[0062] Based on the weights of different action features in the classification unit, key nodes and other nodes are established in the action feature matrix. Using the LSTM deep learning model to train rehabilitation data, all key nodes and other nodes are strongly connected. The connection paths of the key nodes are evaluated to obtain evaluation values ​​for different types of rehabilitation training actions.

[0063] In this embodiment, key nodes and other nodes are established in the action feature matrix, and rehabilitation data is continuously trained through an LSTM model. This can identify the key nodes and their connection paths that have the greatest impact on rehabilitation effects. This method not only helps to deeply understand the important links in the rehabilitation training process, but also provides more accurate action evaluation values ​​by evaluating the connection paths of these key nodes, thereby further optimizing the rehabilitation plan.

[0064] In this embodiment, as more rehabilitation training data accumulates, the LSTM model can gradually improve its performance and adaptability through continuous self-training and parameter adjustment, enabling the system to become increasingly intelligent with time and experience, and better meet the rehabilitation needs of different patients.

[0065] The classification and assessment model for the rehabilitation training movements incorporates a data feedback mechanism, adjusting the distance metric through a multidimensional feedback function, including: The classification and assessment model is used to construct a binary interaction matrix between patients and rehabilitation programs. For the binary interaction matrix Construct the distance matrix Its expression is:

[0066] in, Representing a binary interaction matrix The elements in Indicates the patient, Indicates rehabilitation program, Indicates the patient With rehabilitation projects The distance between them; Grey relational analysis was used to analyze the distance matrix. The analysis was conducted to obtain similarities between patient characteristics and rehabilitation programs; In this embodiment, grey relational analysis is an effective tool for analyzing the correlation of data under conditions of small sample size and incomplete information. Through this method, the similarity between patient characteristics and rehabilitation programs can be measured more accurately, thereby providing a scientific basis for the formulation of rehabilitation plans.

[0067] In this embodiment, by constructing a binary interaction matrix between patients and rehabilitation programs and using grey relational analysis to analyze the distance matrix, the similarity between patient characteristics and rehabilitation programs can be captured in real time. The distance metric can be dynamically adjusted according to the patient's rehabilitation progress and actual performance, ensuring that the classification and assessment model always fits the patient's actual situation.

[0068] For the distance matrix A feedback adjustment function is introduced to feed patient training data back to the classification and evaluation model in a timely manner, adjusting the distance metric in real time. The expression for the feedback adjustment function is:

[0069] in, , These represent the locations of the recovering patient and the recovery program, respectively, in terms of distance measurement. This indicates the distance between the recovering patient and the recovery program. This indicates the degree of similarity between patient characteristics and rehabilitation programs. This indicates the distance between the recovering patient and the recovery program after the feedback adjustment.

[0070] In this embodiment, by introducing a feedback adjustment function, the system can promptly feed back patient training data to the classification and assessment model and update the distance metric in real time. This dynamic adjustment mechanism makes the model more adaptable and flexible, and can cope with various changes that may occur during the rehabilitation process.

[0071] In this embodiment, a binary interaction matrix between the patient and the rehabilitation program is constructed and combined with a feedback adjustment function. The rehabilitation program can be dynamically adjusted according to the specific situation of each patient. For example, the system can recommend the most suitable rehabilitation program based on the patient's physical condition, rehabilitation needs and progress speed, and provide personalized rehabilitation guidance.

[0072] A metric learning algorithm is used to convert the distance metric into probability values. Based on these probability values, rehabilitation recommendation decisions are formed, mapping the relationship between rehabilitation needs and rehabilitation assessments, including: The distance metric is optimized using a loss function to obtain the difference between the rehabilitation training data and the standard sample data. The expression for the loss function is:

[0073] in, This represents a feature mapping function that maps the raw data from rehabilitation training to a feature space. , and Let represent the anchor point, positive sample, and negative sample of the i-th rehabilitation training sample in the distance metric, respectively. This means mapping the anchor point data of the i-th rehabilitation training sample in the distance metric to the feature space. This represents mapping the positive sample of the i-th rehabilitation training sample in the distance metric to the feature space. This means mapping the negative samples of the i-th rehabilitation training sample in the distance metric to the feature space. Indicates the preset level value; In this embodiment, the triplet loss function is used to optimize the distance metric, which can effectively reduce the gap between rehabilitation training data and standard sample data. This ensures that the model can correctly close the distance between similar samples in the feature space, while pushing away dissimilar samples, which helps to improve the accuracy of the model in classifying different rehabilitation movements.

[0074] The loss function is used to remove samples in the distance metric where the rehabilitation training data and the standard sample data have large differences. The distance metric between the anchor point and the positive and negative samples is compared to drive the classification and evaluation model to perform iterative training. In this embodiment, based on the results of the loss function, the system can identify and remove samples that differ significantly from the standard sample data, thereby reducing the impact of outliers on model performance, improving the quality of the dataset, and making the final classification results more reliable.

[0075] In this embodiment, the classification and evaluation model is iteratively trained by comparing the distance metric between the anchor point and the positive and negative samples. This allows the model to gradually adapt to more rehabilitation training scenarios, which not only enhances the robustness of the model but also improves its generalization ability when faced with new data.

[0076] The probability of generating data with a small difference between the rehabilitation training data and the standard sample data is calculated, and a rehabilitation recommendation decision is formed based on the probability, mapping the relationship between rehabilitation needs and rehabilitation assessment.

[0077] In this embodiment, for data that differs little from standard sample data, calculating its generation probability can help determine which rehabilitation training actions are closer to the ideal state. Based on these probability values, the system can better understand the patient's rehabilitation needs, provide personalized rehabilitation suggestions for each patient, and match them with specific rehabilitation assessment results.

[0078] In the second embodiment, a data-feedback-based rehabilitation assessment system is provided to implement a data-feedback-based rehabilitation assessment method. The system includes: The data acquisition module acquires real-time rehabilitation training video data and collects the physiological and motor parameters of rehabilitation patients in real time. The data processing module processes the physiological and motor parameters of the rehabilitation patients, extracts the rehabilitation patients' training data, and generates a distance matrix between the rehabilitation patients' training data and standard data based on the time series. The feedback adjustment module establishes a rehabilitation assessment model, monitors the training data of rehabilitation patients in real time, and adjusts the training according to the distance matrix through a feedback adjustment function. The user interface displays rehabilitation status assessment reports and provides personalized rehabilitation suggestions to patients.

[0079] This invention employs the BlaezPose algorithm for human posture estimation, from acquiring continuous online rehabilitation videos to extracting key point coordinate information and calculating joint angle features and bone point distance features. This ensures comprehensive capture and detailed analysis of rehabilitation training movements. By introducing time series data to dynamically adapt to changes in the patient's rehabilitation progress, it provides more accurate movement features. Furthermore, by utilizing label classification technology and metric learning algorithms to fully consider the individual differences of each patient, it formulates the most suitable rehabilitation plan for each individual's situation. Through a real-time feedback mechanism, it continuously adjusts distance metrics and rehabilitation suggestions, ensuring that the rehabilitation plan always aligns with the patient's latest needs and development trends. Combining binary interaction matrices and distance matrices, and applying feedback adjustment functions, it achieves intelligent monitoring and real-time feedback on the rehabilitation training effect, facilitating timely adjustments to treatment strategies and greatly improving rehabilitation effectiveness and efficiency. Moreover, the entire system, from data acquisition to final rehabilitation recommendation decisions, is fully automated, reducing the need for manual intervention and improving work efficiency and service quality.

[0080] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. A data-driven rehabilitation assessment method, characterized in that, Includes the following steps: Continuous online rehabilitation videos are acquired as rehabilitation training video frame data. The video frame data is segmented into rehabilitation images. The rehabilitation images are processed using a rehabilitation posture estimation algorithm to obtain the coordinates of human joint bones. Based on the time series arrangement of the video frame data, a rehabilitation training action dataset is constructed. The action dataset is subjected to label classification technology to extract joint angle features and bone point distance features, and the distance metric of the coordinates of each human joint bone point is obtained. A classifier is trained separately for each distance metric to obtain the action feature combination of each bone point. A dynamic adjustment algorithm is used to match the time series with the action features to obtain an action feature matrix. The similarity of the action feature matrix is ​​calculated, and a classification and evaluation model for rehabilitation training actions is constructed to evaluate the rehabilitation training actions. The classification and evaluation model for the rehabilitation training movements introduces a data feedback mechanism. The distance metric is adjusted through a multidimensional feedback function, and a metric learning algorithm is used to convert the distance metric into a probability value. Based on the probability value, a rehabilitation recommendation decision is formed, mapping the relationship between rehabilitation needs and rehabilitation assessment.

2. The rehabilitation assessment method based on data feedback according to claim 1, characterized in that, Continuous online rehabilitation videos are acquired as rehabilitation training video frame data. The video frame data is then segmented into rehabilitation images. A rehabilitation posture estimation algorithm is used to process the rehabilitation images to obtain the coordinates of human joint bones. Based on the time-series arrangement of the video frame data, a rehabilitation training motion dataset is constructed, including: Image frames are extracted frame by frame from the online rehabilitation video, and each image frame is regarded as a rehabilitation picture. The BlaezPose algorithm for human pose estimation is used to obtain the skeletal point information of the rehabilitation patient, and the skeletal point coordinate information of the patient in the current state of each frame is obtained. , This represents the coordinates of the i-th skeletal point. and Let these represent the horizontal and vertical positions of the i-th bone point in the current image frame, respectively. Indicates the visibility of the i-th skeletal keypoint in the current image frame; The skeletal point coordinate information is used to predict and correct the trajectory based on the time series, and the coordinates of key points with timestamps are extracted; The coordinates of the key points are represented as a two-dimensional matrix. , The matrix representing the skeletal point information of the patient's pose in the j-th frame image. The information vector representing the first to nth skeletal points of the patient in the j-th frame image; The BlaezPose algorithm is used to process the two-dimensional matrix. Acquire the patient's movement data from the entire rehabilitation training video, and represent the movement data as a three-dimensional matrix. ; For the three-dimensional matrix Based on the key point coordinate data of the time series, action features are extracted to obtain an action dataset.

3. The rehabilitation assessment method based on data feedback according to claim 2, characterized in that, The motion dataset is subjected to label classification technology to extract joint angle features and bone point distance features, including: Extract the key point coordinate data from the motion dataset, calculate its bending angle to obtain joint angle data, and extract joint angle features based on the joint angle data. Calculate the Euclidean distance between any two key points from the key point coordinate data in the action dataset to obtain the bone point distance feature; Support Vector Machine (SVM) is used to classify and label the joint angle features and bone point distance features, and a clustering algorithm is used to obtain the label dataset of the action dataset.

4. The rehabilitation assessment method based on data feedback according to claim 3, characterized in that, The distance metric for the coordinates of each human joint skeletal point is obtained from the labeled dataset. A classifier is trained separately for each distance metric to obtain a combination of motion features for each skeletal point, including: The distance features of skeletal points and joint angle features within the labeled dataset are detected. The maximum cross-validation function is used to evaluate the matching degree between the keypoint coordinate data and the ground truth labeled keypoints. The expression is: in, Indicates the first One key point, Indicates the estimation of key points and the real key points The Euclidean distance between them This represents the standard deviation of the distances between skeletal points in the labeled dataset. Indicates the true key points The scale factor, Indicates the true key points Visibility level Indicates an indicator function, when Returns 1 if the condition is met, otherwise returns 0. According to the maximum cross-evaluation function Filter the key point coordinate data to obtain the distance metric of the coordinates of each human joint bone point; The distance metric is trained separately using a logistic regression model, and the similarity metric is output from the logistic regression model as a feature to obtain the action feature combination of each skeletal point.

5. The rehabilitation assessment method based on data feedback according to claim 3, characterized in that, A dynamic adjustment algorithm is used to match the time series with the action features to obtain an action feature matrix, including: Randomly select two time series and , in sequence and This represents the normalized distance feature of the time-stamped keypoint coordinates in the action feature combination on a two-dimensional plane, where n represents the sequence. Length, These represent the standardized distance values ​​of the 1st, 2nd, and nth joint angle features with timestamps in the motion feature combination, respectively. These represent the standardized distance values ​​of the 1st, 2nd, and nth skeletal points with timestamps in the action feature combination, respectively. in sequence and This represents the normalized distance feature of keypoint coordinate data in a standard template on a two-dimensional plane, where m represents the sequence. Length, These represent the standardized distance values ​​of the 1st, 2nd, and mth joint angle features in the standard template, respectively. These represent the standardized distance values ​​of the distance features of the 1st, 2nd, and mth skeletal points in the standard template, respectively. Define a [amount] based on two action features within the same time series. A dynamic three-dimensional matrix D and a Given a two-dimensional matrix L, calculate the time series under the same time series. and The combined distance in the middle is expressed as: in, This represents the relationship between the i-th element in sequence M and the sequence. The distance between the j-th elements in the middle. Represents the relationship between the i-th element in sequence N and the sequence. The distance between the j-th elements in the middle. Represents a sequence The i-th element and the sequence The combined distance between the j-th elements in the array. The weights represent the distance between the angular feature matches. The weights represent the standardized distance features of the skeleton points on the two-dimensional matrix L plane; The two-dimensional matrix is ​​calculated using a recursive relationship. For all elements in the matrix, obtain the optimal combination distance, and use the sequence corresponding to the optimal combination distance as the action feature matrix.

6. The rehabilitation assessment method based on data feedback according to claim 5, characterized in that, Calculate the similarity of the movement feature matrix, construct a classification and evaluation model for rehabilitation training movements, and evaluate the rehabilitation training movements, including: By calculating the time series and The Euclidean distance is used to compare the similarity distance d between the movement characteristics of rehabilitation training data and the movement characteristics of standard templates. The action similarity is calculated using a scoring function based on the similarity distance d. Its expression is: in, This represents the exponential operation with the natural constant e as the base and the similarity distance d as the exponent; Based on the action similarity Extract the key point coordinate data of the rehabilitation training data, obtain the skeletal point coordinate information of each frame of rehabilitation training data, and extract the rehabilitation training action features based on the skeletal point coordinate information. By acquiring the range of motion and duration of motion of multiple training limbs through motion characteristics, the multi-limb training movements are classified and evaluated. The duration of the motion state during rehabilitation training is recorded, and a classification and evaluation model of rehabilitation training movements is constructed for multi-limb movements.

7. A rehabilitation assessment method based on data feedback according to claim 6, characterized in that, The classification and evaluation model uses an LSTM deep learning model to train on motion feature data and evaluates rehabilitation training movements, including: The time series of the action feature matrix is ​​divided into several time units t using an LSTM deep learning model. For any time unit t, the output of the LSTM deep learning model in the previous time unit t-1 is... As input, construct the unit state function Its expression is: in, , , This represents the parameters of the LSTM deep learning model. This indicates the relationship between input and output. Indicates the correlation of historical data. Indicates the offset value. This represents the state value of the current candidate time unit. This represents the output of the LSTM deep learning model at the current time state; According to the unit state function Through the above Adjust the parameters of the LSTM network model, process the time series information and action features of continuous action features in rehabilitation training, and obtain the weights of different action features in the classification unit; Based on the weights of different action features in the classification unit, key nodes and other nodes are established in the action feature matrix. Using the LSTM deep learning model to train rehabilitation data, all key nodes and other nodes are strongly connected. The connection paths of the key nodes are evaluated to obtain evaluation values ​​for different types of rehabilitation training actions.

8. A data-feedback-based rehabilitation assessment method according to claim 7, characterized in that, The classification and assessment model for the rehabilitation training movements incorporates a data feedback mechanism, adjusting the distance metric through a multidimensional feedback function, including: The classification and assessment model is used to construct a binary interaction matrix between patients and rehabilitation programs. For the binary interaction matrix Construct the distance matrix Its expression is: in, Representing a binary interaction matrix The elements in Indicates the patient, Indicates rehabilitation program, Indicates the patient With rehabilitation projects The distance between them; Grey relational analysis was used to analyze the distance matrix. The analysis was conducted to obtain similarities between patient characteristics and rehabilitation programs; For the distance matrix A feedback adjustment function is introduced to feed patient training data back to the classification and evaluation model in a timely manner, adjusting the distance metric in real time. The expression for the feedback adjustment function is: in, , These represent the locations of the recovering patient and the recovery program, respectively, in terms of distance measurement. This indicates the distance between the recovering patient and the recovery program. This indicates the degree of similarity between patient characteristics and rehabilitation programs. This indicates the distance between the recovering patient and the recovery program after the feedback adjustment.

9. A data-feedback-based rehabilitation assessment method according to claim 8, characterized in that, A metric learning algorithm is used to convert the distance metric into probability values. Based on these probability values, rehabilitation recommendation decisions are formed, mapping the relationship between rehabilitation needs and rehabilitation assessments, including: The distance metric is optimized using a loss function to obtain the difference between the rehabilitation training data and the standard sample data. The expression for the loss function is: in, This represents a feature mapping function that maps the raw data from rehabilitation training to a feature space. , and Let represent the anchor point, positive sample, and negative sample of the i-th rehabilitation training sample in the distance metric, respectively. This means mapping the anchor point data of the i-th rehabilitation training sample in the distance metric to the feature space. This represents mapping the positive sample of the i-th rehabilitation training sample in the distance metric to the feature space. This means mapping the negative samples of the i-th rehabilitation training sample in the distance metric to the feature space. Indicates the preset level value; The loss function is used to remove samples in the distance metric where the rehabilitation training data and the standard sample data have large differences. The distance metric between the anchor point and the positive and negative samples is compared to drive the classification and evaluation model to perform iterative training. The probability of generating data with a small difference between the rehabilitation training data and the standard sample data is calculated, and a rehabilitation recommendation decision is formed based on the probability, mapping the relationship between rehabilitation needs and rehabilitation assessment.

10. A data-feedback-based rehabilitation assessment system, characterized in that, The system for implementing the data feedback-based rehabilitation assessment method according to any one of claims 1-9 includes: The data acquisition module acquires real-time rehabilitation training video data and collects the physiological and motor parameters of rehabilitation patients in real time. The data processing module processes the physiological and motor parameters of the rehabilitation patients, extracts the rehabilitation patients' training data, and generates a distance matrix between the rehabilitation patients' training data and standard data based on the time series. The feedback adjustment module establishes a rehabilitation assessment model, monitors the training data of rehabilitation patients in real time, and adjusts the training according to the distance matrix through a feedback adjustment function. The user interface displays rehabilitation status assessment reports and provides personalized rehabilitation suggestions to patients.

Images