Knee osteoarthritis rehabilitation training evaluation method and system based on pose sensor and virtual interaction platform

Abstract

The invention discloses a knee osteoarthritis rehabilitation training evaluation method and system based on a pose sensor and a virtual interaction platform, and the method skillfully captures the motion postures of a user during training through a pose sensor module, and corrects the collected posture data through introducing a two-stage extended Kalman filtering algorithm. According to the scheme, a staged advanced training mode (a cognition stage, a contact stage and an automation stage) is further introduced, each stage focuses on different learning requirements, the training difficulty is gradually improved, it is ensured that the patient gradually masters actions according to the rehabilitation progress, and it is avoided that the patient enters the advanced training stage too early or too late. The mode effectively solves the problem of memory forgetting possibly encountered by a patient with knee osteoarthritis in Tai Chi rehabilitation, helps the patient to consolidate each action step by step, and dynamically adjusts the training content through a personalized feedback mechanism to improve the rehabilitation effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical rehabilitation training assistance and virtual reality interaction, and in particular relates to a knee osteoarthritis rehabilitation training evaluation method and system based on a pose sensor and a virtual interaction platform. BACKGROUND

[0002] Knee osteoarthritis is a common degenerative joint disease. The dysfunction of patients with knee osteoarthritis is mainly caused by joint structure degeneration and secondary muscle atrophy, especially the weakness of the quadriceps femoris, the hamstrings and the hip girdle muscles. The decline in the function of these muscles will lead to a decrease in joint stability and an increase in abnormal load, further aggravating pain and dysfunction. Therefore, the core of rehabilitation for patients with knee osteoarthritis lies in the recovery and strengthening of muscle function. Through systematic strength training, coordination exercises and functional movements, the stability of the knee joint can be effectively improved, pain can be reduced, and the daily activity ability of patients can be gradually restored.

[0003] With the development of technology, lower limb rehabilitation training has gradually changed from the traditional manual guidance mode to intelligent and automatic direction. Although some hospitals still use the conventional training mode of one-on-one supervision by rehabilitation physicians, large city hospitals mainly use lower limb rehabilitation robot equipment for rehabilitation training. Although the lower limb rehabilitation robots on the market have different focuses in terms of function, they generally have key defects such as high price, weak evaluation function and complex operation. Therefore, some research attempts to introduce virtual reality technology, such as using optical motion capture devices such as Kinect to build interactive scenes. Although this type of solution can improve the interest of training, it still faces significant challenges in medical-level applications. In complex motion scenarios, there are problems such as three-dimensional data missing, insufficient accuracy, motion with overlapping skeletal points, and susceptibility to light and occlusion interference. These technical limitations not only affect the accuracy of training, but also may endanger the safety of training. SUMMARY

[0004] In view of the above, the purpose of the present application is to provide a knee osteoarthritis rehabilitation training evaluation method and system based on a pose sensor and a virtual interaction platform, which is reliable to implement, flexible to apply, convenient and intuitive to interact, and has good data acquisition accuracy.

[0005] In order to achieve the above technical purpose, the technical solution adopted by the present application is:

[0006] A knee osteoarthritis rehabilitation training evaluation method based on a pose sensor and a virtual interaction platform, comprising:

[0007] S1, wearing the pose sensor module on the user's limbs according to the preset requirements, and collecting the three-dimensional space motion data of the limbs of the user during motion through the pose sensor module;

[0008] S2, applying a double-stage extended Kalman filter algorithm to the data collected by the pose sensor module to compensate for errors and obtain preprocessed attitude data;

[0009] S3, correcting the attitude data according to a preset condition in combination with individual characteristics of the user;

[0010] S4, collecting attitude data generated by the user wearing the pose sensor module during the rehabilitation training process, then comparing and analyzing the attitude data with the preset reference data, and simultaneously transmitting the attitude data to a virtual interaction platform for display, and the virtual interaction platform guides the user according to the attitude data.

[0011] As a possible implementation, further, in the scheme S01, the limbs include limbs, waist, head, the pose sensor module is a plurality of, which are respectively worn on the limbs, waist and head of the user; the three-dimensional space motion data of the limbs includes the joint angle, angular velocity and / or acceleration of the corresponding wearing area of the pose sensor module.

[0012] As a possible implementation, further, in the scheme S02, the double-stage extended Kalman filter algorithm includes a first-stage filter processing and a second-stage filter processing, in the first-stage filter processing, the observation noise matrix is adaptively adjusted according to the acceleration observation error covariance to reduce the short-term linear acceleration interference, so as to improve the estimation accuracy of the pitch angle and roll angle of the pose sensor module; in the second-stage filter processing, the integral drift of the yaw angle over time is compensated by fusing the magnetometer data to output stable high-precision attitude data.

[0013] As a possible implementation, further, the scheme S3 includes: correcting the corresponding attitude data of the patient according to the individual characteristics of the patient, which includes:

[0014] When wearing the pose sensor module for the first time, the static standard posture of the user in T-shaped standing, sitting posture (knee joint flexion 90°) and affected side single leg support is collected, and the Kabsch algorithm is used to minimize the deviation between the gravity direction measured by the pose sensor module and the ideal gravity direction in the static posture to calculate the rotation matrix for coordinate alignment, and the standard proportion coefficient of the limb length is calculated based on the height and gender of the patient, and the attitude data is linearly scaled and corrected according to the proportion coefficient.

[0015] As a preferred selection implementation, preferably, the scheme S3 includes:

[0016] S31, when wearing for the first time, the user completes the static standard posture in T-shaped standing, sitting posture (knee joint flexion 90°) and / or affected side single leg support, and obtains the initial attitude data through the pose sensor module Meanwhile, user basic information and pathological feature information are acquired, wherein the basic information includes height h, weight w and / or gender s of the user, and the pathological feature information includes limb affected side information, subjective pain score VAS and / or maximum knee flexion angle θ max ; the initial posture data includes a limb segment position vector P raw corresponding to the user's body shape and a gravity direction vector

[0017] S32, a homogeneous transformation matrix from the actual coordinate system of the user to the reference coordinate system is constructed, which is defined as follows:

[0018]

[0019] wherein R is a rotation matrix for posture direction correction, and t is a translation vector for limb segment length proportion compensation;

[0020] The rotation matrix R minimizes the gravity direction error measured in the static posture by using the Kabsch algorithm to solve the optimal rotation matrix, which is defined as follows:

[0021]

[0022] is an ideal gravity vector; is a gravity direction vector measured by the posture sensor module;

[0023] The translation vector t is determined based on the height h and the gender s, which is defined as follows:

[0024] t=α(s)·L std (h)

[0025] wherein α(s) is a gender difference coefficient, which is a preset value; L std (h) is a standard bone segment length vector corresponding to the height h, which is obtained by a preset query table method;

[0026] The formula for the preliminary correction of the posture data is defined as follows:

[0027] P corrected =R·P raw +t

[0028] wherein P corrected is the corrected posture data, P raw is the initial limb segment position vector, R is the rotation matrix, and t is the translation vector.

[0029] As a preferred selection implementation, preferably, the scheme S3 further includes:

[0030] S33, pathological correction is introduced to the action of the affected knee joint, the compensation time of the knee joint action start delay is calculated according to the subjective pain score VAS of the patient, and the compensation index is calculated according to the ratio of the maximum flexion angle of the hip joint to the knee joint, when the compensation index exceeds the preset threshold, the system triggers voice or visual prompt to correct the abnormal action;

[0031] The calculation of the compensation time Δt is defined as follows:

[0032] Δt=k·VAS

[0033] Wherein, k is the adjustment coefficient, its value range is 0.2~0.4; VAS is the subjective pain score, its score range is 0 to 10, indicating the degree of no pain to severe pain;

[0034] The calculation of the compensation index C is defined as follows:

[0035]

[0036] Wherein, θ hip , θ knee are the maximum flexion angles of the hip joint and the knee joint in the action cycle of the user during the action; when C>1.5, the set threshold is exceeded, the voice and visual prompt are triggered, and the user is guided to correct the action to ensure the safety of training.

[0037] As a preferred selection implementation, preferably, in the present scheme S3, the corrected posture data is also normalized to the interval [0, 1].

[0038] In addition to the above, the present scheme S3 further comprises: updating the alignment transformation matrix T according to the action data in the posture data and the standard action data in the corresponding preset reference data, which is defined as follows:

[0039]

[0040] Wherein, η t is the time-dependent learning rate, is the observed action data at time t in the posture data, is the standard action data at time t in the preset reference data, T t is the homogeneous transformation matrix at time t, T t+1 is the homogeneous transformation matrix at time t+1 after updating.

[0041] As a preferred selection embodiment, preferably, in the scheme S4, the posture data generated by the rehabilitation training process is sent to a three-dimensional virtual interaction platform for display after preprocessing and correction, and the virtual interaction platform extracts different training schemes according to the preset cognitive stage, contact stage and automation stage to provide standard taijiquan rehabilitation action demonstration, voice prompt and visual guidance, and generates a rehabilitation evaluation report containing action similarity, motion stability and / or VAS score information after each training cycle.

[0042] As a preferred selection embodiment, preferably, in the scheme S4, the posture data generated by the rehabilitation training process is sent to a three-dimensional virtual interaction platform for display after preprocessing and correction, and the virtual interaction platform extracts different training schemes according to the preset cognitive stage, contact stage and automation stage to provide standard taijiquan rehabilitation action demonstration, voice prompt and visual guidance, and generates a rehabilitation evaluation report containing action similarity, motion stability and / or VAS score information after each training cycle.

[0043] The scheme is aimed at the problems existing in the current knee osteoarthritis rehabilitation training, such as inaccurate action evaluation, lack of quantitative standard for training scheme, and low patient compliance. The knee osteoarthritis rehabilitation training evaluation method based on pose sensor and virtual interaction platform is innovatively proposed. The action data of the patient is captured in real time by the high-precision pose sensor, combined with the taijiquan action library of different difficulty and the intelligent evaluation algorithm, the action deviation detection is realized and the report is formed, and the training difficulty is adjusted according to the training performance of the patient, so as to solve one or more problems in the prior art, such as expensive equipment, weak evaluation function, complex operation, lack of three-dimensional data, insufficient precision, overlapping of skeletal points in motion, and interference of light and shielding.

[0044] Based on the above, the scheme also proposes a knee osteoarthritis rehabilitation training evaluation system based on pose sensor and virtual interaction platform, which comprises:

[0045] The pose sensor module is used to collect three-dimensional motion data of the user's limbs, which includes one or more of joint angle, angular velocity and acceleration.

[0046] The Zigbee communication module is respectively in communication connection with the plurality of pose sensor modules, and is used to upload the pose data collected by the pose sensor module.

[0047] The data processing module is used to receive the pose data, and apply a double-stage extended Kalman filtering algorithm to the data collected by the pose sensor module for error compensation to obtain preprocessed posture data; it is also used to correct the posture data according to the preset condition, and pathological action compensation and data normalization processing are combined with the individual characteristics of the user.

[0048] The virtual interactive training module is used to load the virtual interactive platform, provide standardized rehabilitation movement demonstrations, voice and visual prompts, and compare and guide the patient's movements in real time according to the training plan of the cognitive stage, the association stage and the automation stage.

[0049] The data storage module is used to collect posture data generated by the user's wearable posture sensor module during rehabilitation training.

[0050] The data evaluation module is used to evaluate the posture data generated by the user wearing the posture sensor module during rehabilitation training. After each training cycle, it generates a rehabilitation evaluation report containing information on movement similarity, movement stability and / or VAS score.

[0051] A medical assistance module is used to connect with the data evaluation module, transmit rehabilitation evaluation reports to the remote doctor's terminal, and adjust training parameters.

[0052] Compared with existing technologies, the present invention, employing the above technical solution, has the following beneficial effects: This solution ingeniously captures the user's posture during training using a pose sensor module, enabling the collection of three-dimensional spatial motion data of the user's limbs. Furthermore, it introduces a two-stage extended Kalman filter algorithm to correct the collected posture data, resulting in higher accuracy and ensuring data precision throughout the training process, thereby improving the reliability of the final training evaluation. In addition, this solution introduces a phased progressive training mode (cognitive stage, connection stage, and automation stage) based on the three-stage theory of motor learning. Each stage focuses on different learning needs, gradually increasing the training difficulty to ensure that patients master the movements according to their rehabilitation progress, avoiding premature or delayed entry into advanced training stages. This mode effectively solves the memory forgetting problem that knee osteoarthritis patients may encounter during Tai Chi rehabilitation, helping patients gradually consolidate each movement and dynamically adjusting training content through a personalized feedback mechanism to improve rehabilitation effects. Simultaneously, the phased training design increases the fun of training and patient participation, making the rehabilitation process more scientific, effective, and motivating. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a brief outline of the implementation process of this solution.

[0055] Figure 2 is a brief schematic diagram of the pose sensor module of the present scheme worn on the user's limbs, wherein the user's face is subjected to desensitization shielding treatment;

[0056] Figure 3 is one of the brief working logic flowcharts of the method of the present scheme when loaded on a computer system, wherein nodes A, B, C, D, E, and F are process nodes;

[0057] Figure 4 is another brief working logic flowchart of the method of the present scheme when loaded on a computer system, wherein nodes A, B, C, D, E, and F are process nodes corresponding to the nodes shown in the figure; Figure 3

[0058] Figure 5 is one of the brief schematic diagrams of the display training posture in the virtual interaction platform in the method of the present scheme;

[0059] Figure 6 is an interface set diagram of the method of the present scheme when loaded on a computer system;

[0060] Figure 7 is a diagram of each system layer when the method of the present scheme is integrated into a training system;

[0061] Figure 8 is a brief connection diagram of the unit modules of the system of the present scheme. DETAILED DESCRIPTION

[0062] The present application will be further described below in conjunction with the accompanying drawings and embodiments. It is particularly pointed out that the following embodiments are only used to illustrate the present application, but do not limit the scope of the present application. Similarly, the following embodiments are only part of the embodiments of the present application, not all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of the present application.

[0063] In conjunction with Figure 1 the present embodiment scheme is a knee osteoarthritis rehabilitation training evaluation method based on a pose sensor and a virtual interaction platform, which comprises:

[0064] S1, wearing a pose sensor module on the user's limbs according to the preset requirements, and collecting three-dimensional space motion data of the user's limbs when moving through the pose sensor module;

[0065] S2, applying a double-stage extended Kalman filtering algorithm to the data collected by the pose sensor module to compensate for errors and obtain preprocessed posture data;

[0066] S3, correcting the posture data according to the preset conditions in combination with the individual characteristics of the user;

[0067] ​S4, the posture data generated by the user wearing the posture sensor module during the rehabilitation training process is collected, and then compared and analyzed with the preset reference data, and transmitted to the virtual interaction platform for display, and the virtual interaction platform also guides the user according to the posture data.

[0068] Reference Figure 2 As shown in the possible embodiment of wearing the posture sensor module on the user's limbs, in the scheme S01, the limbs include limbs, waist, head, the posture sensor module is multiple, which is respectively worn on the user's limbs, waist, head; the three-dimensional space motion data of the limbs includes the joint angle, angular velocity and / or acceleration of the corresponding wearing area of the posture sensor module.

[0069] Compared with the traditional optical motion capture device such as Kinect, which is easily disturbed by environmental factors such as light and shielding, the high-precision inertial posture sensor is used as the component of the posture sensor assembly, which can significantly improve the stability and accuracy of motion capture in complex scenes, avoid the problems of overlapping of skeletal points and lack of three-dimensional data, and ensure the safety and scientific nature of rehabilitation training.

[0070] In the scheme, the shell of the posture sensor module can be made by 3D printing, and then combined with the binding belt for auxiliary wearing. In this way, the wearing reliability of the posture sensor module can be effectively improved, and the abnormal problem of large displacement of the user during wearing motion can be avoided as much as possible. The posture sensor used in the scheme can be purchased in the market, and its basic hardware functions (gyroscope, accelerometer) are basically the same as existing products, so details will not be described here.

[0071] Due to the cumulative error of the gyroscope of the posture sensor module and the non-linear distortion of the accelerometer, further, in the scheme S02, before processing the data collected by the posture sensor module, a double-stage extended Kalman filter architecture is introduced, and an intelligent filtering algorithm is used for error compensation. The double-stage extended Kalman filter algorithm includes primary filtering processing and secondary filtering processing. In the primary filtering processing, the observation noise matrix is adjusted adaptively according to the acceleration observation error covariance, the short-term linear acceleration interference is reduced, and the pitch angle and roll angle estimation accuracy of the posture sensor module is improved. In the secondary filtering processing, the integral drift of the yaw angle with time is effectively compensated by fusing the magnetometer data, so as to output stable and high-precision posture data. By loading the filtering module of the double-stage extended Kalman filter architecture, low-delay and high-accuracy posture basic data support is provided for the whole training system.

[0072] Due to the body shape difference of different users, in order to solve the posture data distortion problem caused by the body shape difference, wearing error and pathological action characteristics of different users (patients), the scheme introduces a "dynamic calibration correction mechanism based on individual characteristics" in step S3 to perform posture alignment, bone segment proportion adjustment and pathological characteristic compensation on the original sensor data. Further, the scheme S3 includes: correcting the posture data of the patient according to the individual characteristics of the patient, which includes:

[0073] When wearing the pose sensor module for the first time, the static standard posture of the user in the T-shaped standing, sitting posture (knee joint flexion 90°) and affected side single leg support is collected, and the Kabsch algorithm is used to minimize the deviation between the gravity direction measured by the pose sensor module in the static posture and the ideal gravity direction to calculate the rotation matrix for coordinate alignment, and the limb length standard proportion coefficient is calculated based on the height and gender of the patient, and the posture data is linearly scaled and corrected according to the proportion coefficient.

[0074] Specifically, the mechanism includes the following core processing steps:

[0075] S31, when wearing for the first time, the user completes the static standard posture in the T-shaped standing, sitting posture (knee joint flexion 90°) and / or affected side single leg support, and the initial posture data is obtained by the pose sensor module At the same time, the basic information and pathological characteristic information of the user are also obtained.

[0076] Among them, the basic information includes the height h, weight w and / or gender s corresponding to the user;

[0077] The pathological characteristic information includes the affected side information of the limb, the subjective pain score VAS and / or the maximum knee joint flexion angle θ max ; wherein the pathological characteristic information can be obtained in advance by doctor evaluation or through autonomous evaluation, for example, the maximum knee joint flexion angle θ max Obtained by active motion test;

[0078] In the present scheme, the initial posture data includes the limb segment position vector P raw and the gravity direction vector and other posture data corresponding to the body shape of the user;

[0079] The above information is used as the input basis for individualized posture correction and subsequent training grading evaluation;

[0080] S32, since the pose sensor module will have installation deviation when worn, therefore, unifying the coordinate reference is helpful to improve the reliability of subsequent data judgment, this step constructs the homogeneous transformation matrix from the actual coordinate system of the user to the reference coordinate system, which is defined as follows:

[0081]

[0082] wherein, E is a rotation matrix, which is used for pose direction correction; t is a translation vector, which is used for limb length proportion compensation;

[0083] In this step, the rotation matrix R is calculated by minimizing the error of the measured gravity direction in the static posture by Kabsch algorithm, and the optimal rotation matrix is defined as follows:

[0084]

[0085] which is the ideal gravity vector; which is the gravity direction vector measured by the pose sensor module;

[0086] The translation vector t is determined based on the height h and the gender s, and is defined as follows:

[0087] t = a(s)·L std (h)

[0088] wherein, a(s) is the gender difference coefficient, which is a preset value; L std (h) is the standard bone length vector corresponding to the height h, which is obtained by a preset query table method;

[0089] Finally, the formula for preliminary correction of the posture data is defined as follows:

[0090] P corrected = R·P raw +t

[0091] wherein, P corrected is the corrected posture data, P raw is the initial limb position vector, R is the rotation matrix, and t is the translation vector.

[0092] As a preferred implementation, the present scheme S3 further comprises:

[0093] S33, in order to adapt to the action delay, compensatory behavior, etc. commonly seen in patients with knee osteoarthritis, the present scheme introduces a pathological correction term, which includes: introducing a pathological correction for the action of the affected knee joint, calculating the compensation time of the action start delay of the knee joint according to the subjective pain score VAS of the patient, and calculating the compensation index according to the ratio of the maximum flexion angle of the hip joint to the knee joint, when the compensation index exceeds the preset threshold, the system triggers a voice or visual prompt to correct the abnormal action;

[0094] The calculation of the compensation time Δt (or the action start delay time, unit: seconds) is defined as follows:

[0095] Δt = k·VAS

[0096] wherein k is an adjustment coefficient, and its value range is 0.2-0.4; VAS is a subjective pain score (Visual Analog Scale), and its score range is 0-10, representing the degree of no pain to severe pain;

[0097] The calculation of the compensation index C is defined as follows:

[0098]

[0099] wherein θ hip and θ knee are the maximum hip joint and knee joint flexion angles of the user in the action period when the user is in action; when C>1.5, the set threshold is exceeded, triggering voice and visual prompts to guide the user to correct the action to ensure training safety.

[0100] In order to improve the consistency of data processing, as a relatively optimal selection implementation, preferably, in the present scheme S3, the corrected posture data is also normalized to the interval [0, 1], and after data normalization, the algorithm stability and cross-population comparability can be improved. The core data fields include angle, angular velocity, joint center position, etc.

[0101] In addition to the above, in order to improve the flexibility of the correction matrix, and provide a more optimal correction for different situations, to improve the posture evaluation robustness and individual adaptation ability in long-term training. The present scheme S3 also includes: updating the homogeneous transformation matrix T according to the action data in the posture data and the standard action data in the corresponding preset reference data, which is defined as follows:

[0102]

[0103] wherein η t is a time-dependent learning rate, is the observed action data at time t in the posture data, is the standard action data at time t in the preset reference data, T t is the homogeneous transformation matrix at time t, T t+1 is the homogeneous transformation matrix at time t+1 after updating.

[0104] The traditional scheme can usually only provide a fixed correction scheme when facing posture data distortion caused by body shape differences, wearing errors and other factors of different patients. The present scheme adopts a dynamic calibration correction mechanism based on individual characteristics, and performs personalized posture correction according to the height, weight, gender, pathological characteristics and other information of the patient, thereby effectively reducing the data distortion caused by wearing errors and individual differences, and ensuring that the action evaluation in the training process is more accurate and reliable.

[0105] In the present scheme S4, the posture data generated during the rehabilitation training process is pre-processed and corrected, and then sent to a three-dimensional virtual interaction platform for display. The virtual interaction platform extracts different training programs according to the preset cognitive stage, contact stage and automation stage to provide standard Tai Chi rehabilitation action demonstration, voice prompt and visual guidance, and generates a rehabilitation evaluation report containing action similarity, motion stability and / or VAS score information after each training cycle.

[0106] Figure 3 、 Figure 4 is the brief working logic flow when the present scheme method is loaded on the computer system. It shows the dynamic evaluation and decision mechanism of the system during the patient training process after the present scheme method is loaded on the training system. The system can automatically match the training difficulty according to the patient's historical training data or individual situation, and collect three-dimensional motion data through the pose sensor, compare the motion template in the standard Tai Chi database in real time, and judge the action accuracy and training safety. In the case of action deviation, training stagnation or evaluation index not meeting the standard, the system will correct through voice guidance, or reduce the difficulty, pause the training in time, and trigger an abnormal report after multiple failures to prompt the doctor to intervene. After the training is completed, the system determines whether to enter the next difficulty stage according to the action score and pain improvement, realizing the intelligent, phased and personalized regulation of rehabilitation training.

[0107] As an example of step S4, during the rehabilitation training process, the real-time motion data recorded by the pose sensor module can be collected through the Zigbee serial transmission module, and the action accuracy is compared in cycles after the motion is completed, and the data is transmitted to the virtual interaction platform Unity 3D at the same time.

[0108] At the first training, the present scheme can comprehensively evaluate the patient's basic information and recommend the appropriate initial training difficulty, which is started after the doctor's evaluation of the patient's rehabilitation training; if it is the second and subsequent training, the training level of the patient is adjusted according to the latest evaluation report and the doctor's evaluation to realize the individualized rehabilitation path. Figure 5 As shown in the figure, the present scheme platform can build a virtual rehabilitation character model based on the Unity 3D engine, provide standardized Tai Chi rehabilitation action demonstration, and guide the patient to complete the target action with voice prompt and screen text. The system has a dynamic connection mechanism that allows the patient to gradually upgrade the training difficulty level or automatically return to the previous stage when necessary, realizing the dynamic adaptation of the individualized rehabilitation path.

[0109] In addition, as a preferred selection embodiment, the virtual interaction platform generates different types of scenes and outputs audio information according to preset conditions to assist in forming the scene atmosphere during the display.

[0110] Figure 6 is an interface collection diagram of the method of the present scheme when loaded on a computer system, which shows part of the function display of the virtual interaction platform in the method of the present scheme. Figure 7 is a system layer diagram of the method of the present scheme integrated into a training system.

[0111] In the selection of the training stage, the present scheme divides the rehabilitation training process into three modules, namely, the cognitive stage, the connection stage and the automation stage, according to the three-stage theory of motor learning proposed by psychologists Paul Fitts and Michael Posner in their classic work Human Performance in 1967, which is suitable for the training goals and user ability levels in the early, middle and late stages of rehabilitation. Each stage calls different training database templates and sets specific action comparison mechanisms, feedback modes and virtual guidance strategies.

[0112] As an example, the training stage of the present scheme specifically includes the following:

[0113] (1) Cognitive stage training module

[0114] The cognitive stage training module is designed for patients in the early stage of rehabilitation and elderly users with weak action imitation ability. The system uses a basic single action decomposition template and provides three demonstration modes of 0.5 times speed, 0.75 times speed and standard speed through a virtual guidance model. Patients can choose the appropriate training rhythm according to their own rehabilitation conditions. In the training process, the virtual interaction platform demonstrates each decomposed action step by step, automatically pauses after the demonstration is completed, and continues to the next step only after the patient completes the imitation action, thereby ensuring that the patient can learn at a controllable pace.

[0115] The training adopts a periodic evaluation mechanism, i.e., "triggering comparison once for every 3 action cycles", instead of the traditional real-time comparison mode. The training process is organized in a hierarchical structure of "3 complete actions as a training cycle, 3 training cycles as a large cycle, and 8 large cycles as a complete training". After each cycle, the background system of the virtual interaction platform analyzes the patient's action performance, avoiding the system load caused by real-time calculation. The comparison parameters cover joint angles, motion trajectories and action rhythm, etc. to ensure the comprehensiveness and accuracy of the evaluation.

[0116] To ensure the safety and effectiveness of training, the program can also provide early warnings through a hierarchical error intervention mechanism. When critical errors such as knee joint activity exceeding the limit are detected, the system immediately interrupts the training through voice prompts and freezes the virtual demonstration model through the virtual interaction platform, while triggering a doctor alert to inform medical personnel to intervene and assess whether the training program needs to be adjusted. If there are moderate errors such as significant deviation in movement rhythm, unstable body center of gravity, etc., the system will temporarily pause the current movement demonstration and guide the patient to correct through voice or visual prompts, and then continue training after correction, thus avoiding overall interruption. For non-critical errors such as insufficient amplitude and slight errors in joint angle, the system provides asynchronous feedback through side bar text prompts or icon markers, allowing patients to adjust the movement without interrupting the training.

[0117] At the end of each training cycle, the virtual interaction platform automatically generates an intelligent evaluation report, including the average similarity of the movement, the stability analysis of the movement, the training score, and the trend comparison with the historical data, helping patients and doctors to fully understand the rehabilitation progress. When the patient completes 3 sets of training, if the 3 movements all meet the set standard and there is no significant deviation, the system will generate a movement completion report and mark that the movement training has been completed. The patient needs to complete 8 large cycles in turn to complete the overall training task of the movement.

[0118] (2) Contact phase training module

[0119] In the contact phase training module, the system provides short routine combination training containing 5 standardized movements for mid-term rehabilitation patients who have completed basic movement learning. When the training starts, the virtual coach will demonstrate the target movement frame by frame at 0.75 times speed, and automatically pause after each movement demonstration. After the patient completes the corresponding imitation training, the system will continue the subsequent content. The patient needs to complete the training task of 2 different short routines, and each short routine combination needs to be repeated 2 times. The system will complete 2 sets of short routine training tasks for 2 times as a large training cycle.

[0120] This phase adopts a hierarchical evaluation mechanism, with the initial action similarity determination threshold set to 60%-70%. In the first 3 training cycles, the system uses a fixed threshold for standardized evaluation; starting from the 4th training cycle, the evaluation threshold is dynamically adjusted based on the patient's historical training data, with an adjustment range of ±2%. After each training cycle, the system will automatically generate a heat map report containing three-dimensional motion analysis, highlighting the 5 key movement frames with the largest error amplitude, and achieving visual error feedback through comparison with the standard movement template.

[0121] To ensure the safety and effectiveness of the training process, the same as the cognitive training part, the error intervention mechanism is implemented. When detecting key errors such as knee joint activity exceeding the limit, the system will immediately interrupt the training through voice prompts, freeze the virtual demonstration model, and trigger a doctor alert to inform medical personnel to intervene and assess whether the training program needs to be adjusted. If there are moderate errors such as significant deviation in movement rhythm, instability of body center of gravity, etc., the system will temporarily pause the current movement demonstration and guide the patient to correct through voice or visual prompts, and then continue training to avoid overall interruption. For non-critical errors such as insufficient amplitude and slight joint angle errors, the system provides feedback through side bar text prompts, and the patient can adjust the movement without interrupting the training.

[0122] The patient needs to complete a complete training unit of 12 large cycles (i.e. 24 short routine training) and the average movement similarity of the last 3 large cycles needs to reach more than 65%, while the pain VAS score shows a stable downward trend (decrease ≥5%), and after the doctor's evaluation, the next rehabilitation stage can be entered.

[0123] (3) Automatic stage module

[0124] When the automatic training stage starts, the system first loads the complete Tai Chi routine template as a standard reference, while performing two key settings: completely turning off the visual feedback system and starting the panoramic environment simulation function. The patient then starts to complete the entire Tai Chi movement independently, during which the system continuously monitors two key nodes: when the patient forgets the movement, the temporary guide can be retrieved by clicking "view Tai Chi movement content"; at the same time, the safety monitoring module judges in real time whether there is a risk of movement, and if a dangerous posture is detected, the emergency termination program is immediately executed and the doctor alert is started simultaneously.

[0125] After the patient completes the entire movement, the system enters the evaluation stage. At this time, two levels of automatic judgment are performed: first, the patient's movement data is compared with the template as a whole, and then it is checked whether the similarity of this training reaches the preset standard of 70%-80%. If it meets the standard, an intelligent evaluation report containing three-dimensional motion analysis is generated; if it does not meet the standard, the error correction mechanism is triggered - the system will automatically play back the patient's non-standard movement segment and compare it with the standard demonstration movement, while starting the targeted local reinforcement training through voice prompts "text prompts".

[0126] This stage sets strict cycle control logic, and must complete 3 times of standard training (each similarity ≥ threshold) to end this training. During repeated training, the system will dynamically adjust the prompt intensity based on historical data, and will strengthen the reminder for action nodes that repeatedly appear errors, such as reminding the patient to bend the knee angle before the knee bending action, to ensure the efficiency of error correction.

[0127] Through the step-by-step guidance of the above three-stage modules, the system can realize cognitive learning from low intensity and high guidance, connection training with medium complexity and feedback visualization, and finally transition to high difficulty, low interference and approximate reality environment automation operation, effectively covering the diversified training needs of users at different rehabilitation stages and ensuring the safety, individualization and scientificity of training.

[0128] The scheme introduces a phased progression training mode based on the three-stage theory of motor learning (cognitive stage, connection stage, automation stage), each stage focusing on different learning needs, gradually increasing the training difficulty, ensuring that patients gradually master the movements according to the rehabilitation progress, and avoiding premature or late entry into the advanced training stage. This mode effectively solves the memory forgetting problem that knee osteoarthritis patients may encounter in Tai Chi rehabilitation, helping patients gradually consolidate each movement, and dynamically adjusting the training content through personalized feedback mechanism to improve rehabilitation effect. At the same time, the phased training design improves the interestingness of training and the participation of patients, making the rehabilitation process more scientific, effective and motivating.

[0129] In addition, the prior art often relies on real-time action comparison, which is easily affected by system load and real-time calculation limitations, resulting in reduced evaluation accuracy. The scheme uses a periodic evaluation mechanism to set different cycle triggers for comparison at different stages, reducing the system burden, and automatically adjusting the evaluation standard according to the difficulty of different movement templates, improving the evaluation accuracy and calculation efficiency in the training process.

[0130] As an example, the scheme step S4 generates a rehabilitation evaluation report containing movement similarity, movement stability and / or VAS score information at the end of each training cycle. Based on this, after completing the set large cycle of each movement (cognitive stage 8 large cycles; connection stage 12 large cycles; automation stage 8 large cycles), the training scheme can be dynamically adjusted by comprehensively analyzing the multidimensional data of the patient's last 24 training, including movement accuracy progress curve, pain VAS score trend, physiological index stability, etc.

[0131] As an example, it includes the following:

[0132] If the last training meets the following conditions for 3 consecutive times:

[0133] ① Rehabilitation movement score ≥ 75 points;

[0134] ② Keep basic stability when bearing weight and shifting center of gravity, no obvious shaking or risk of falling. Berg Balance Scale ≥ 45 points;

[0135] ③ The range of motion of the main training joints (such as shoulder joint, hip joint, knee joint) is within the safe range, with no obvious limitation or compensation;

[0136] ④ Training process pain score VAS ≤ 3 points, no pain due to the interruption of training; no joint swelling, sprain and other acute injury performance.

[0137] After meeting the above conditions, the doctor agrees after assessment, then unlock the next difficulty level; once the patient has pain and other conditions, manual intervention to stop this training; if the following one of the three consecutive training:

[0138] ① berg balance scale <40;

[0139] ② The range of motion of the main training joints (such as shoulder joint, hip joint, knee joint) is not within the safe range;

[0140] ③ Training process pain score VAS ≥ 4 points, due to the interruption of training because of pain; joint swelling, sprain and other acute injury performance;

[0141] The scheme can synchronize the report generated by the system to the doctor, and the doctor can intervene in the patient's rehabilitation exercise, conduct clinical reevaluation, and judge to reduce the training level to reduce the training intensity and prevent disease rebound or potential deterioration.

[0142] In order to improve the intuitiveness and flexibility of data interaction, the scheme can automatically generate a medical report containing a three-dimensional motion comparison animation for all evaluation data, support doctors to remotely view rehabilitation progress through a web-based management platform, and manually adjust training parameters to realize a "home training + telemedicine" mode, and expand the application scenarios of the rehabilitation system.

[0143] In combination Figure 8 with the above, the scheme also proposes a knee osteoarthritis rehabilitation training evaluation system based on a pose sensor and a virtual interaction platform, which includes:

[0144] The pose sensor module is multiple, and the multiple pose sensor modules are used to be worn on the user's limbs. The pose sensor module is used to collect three-dimensional motion data of the user's limbs, which includes one or more of joint angle, angular velocity and acceleration.

[0145] The Zigbee communication module is respectively in communication connection with the plurality of pose sensor modules, and is used to upload the pose data collected by the pose sensor module;

[0146] The data processing module is used to receive the pose data, and apply a two-stage extended Kalman filtering algorithm to the data collected by the pose sensor module for error compensation to obtain preprocessed attitude data; also used in combination with individual characteristics of the user, correct the attitude data according to the preset condition, and pathological action compensation and data normalization processing;

[0147] a virtual interaction training module for loading a virtual interaction platform, providing standardized rehabilitation action demonstration, voice and visual prompt, and real-time comparison and guidance with patient action according to training scheme of cognitive stage, contact stage and automation stage;

[0148] a data storage module for collecting posture data generated by the user wearing the posture sensor module during the rehabilitation training process;

[0149] a data evaluation module for evaluating the posture data generated by the user wearing the posture sensor module during the rehabilitation training process, and generating a rehabilitation evaluation report containing action similarity, motion stability and / or VAS score information after each training cycle of the user;

[0150] a medical assistance module connected with the data evaluation module, and transmitting the rehabilitation evaluation report to a remote doctor end and adjusting the training parameters.

[0151] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0152] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0153] The above only describes some embodiments of the present application, and does not limit the protection scope of the present application, and any equivalent device or equivalent process transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A knee osteoarthritis rehabilitation training evaluation method based on a pose sensor and a virtual interaction platform, characterized in that, It comprises: S1, wearing the pose sensor module on the user's limbs according to the preset requirements, collecting the three-dimensional space motion data of the limbs when the user moves through the pose sensor module; S2, applying a two-stage extended Kalman filtering algorithm to the data collected by the pose sensor module to compensate for errors, so as to obtain preprocessed attitude data; S3, correcting the attitude data according to the preset conditions combined with the individual characteristics of the user; S4, collecting the attitude data generated during the rehabilitation training process of the user wearing the pose sensor module, then comparing and analyzing it with the preset reference data, and at the same time, transmitting it to the virtual interaction platform for display, and the virtual interaction platform also guides the user according to the attitude data.

2. The knee osteoarthritis rehabilitation training evaluation method based on the pose sensor and virtual interaction platform according to claim 1, wherein, In S01, the limbs include limbs, waist, head, and the pose sensor module is multiple, which is respectively worn on the user's limbs, waist and head; The three-dimensional space motion data of the limbs includes the joint angle, angular velocity and / or acceleration of the corresponding wearing area of the pose sensor module.

3. The knee osteoarthritis rehabilitation training evaluation method based on the pose sensor and virtual interaction platform according to claim 1 or 2, characterized in that, In S02, the two-stage extended Kalman filtering algorithm includes one-stage filtering processing and two-stage filtering processing. In one-stage filtering processing, the observation noise matrix is adjusted adaptively according to the acceleration observation error covariance to reduce short-term linear acceleration interference, so as to improve the estimation accuracy of the pitch angle and roll angle of the pose sensor module; In two-stage filtering processing, the integral drift of the yaw angle over time is compensated by fusing the magnetometer data to output stable high-precision attitude data.

4. The knee osteoarthritis rehabilitation training evaluation method based on the pose sensor and virtual interaction platform according to claim 1, wherein, S3 includes: correcting the corresponding attitude data of the patient according to the individual characteristics of the patient, which includes: When wearing the pose sensor module for the first time, collect the static standard posture of the user in T standing, sitting posture (knee joint flexion 90°) and affected side single leg support, and use Kabsch algorithm to minimize the deviation between the gravity direction measured by the pose sensor module and the ideal gravity direction in the static posture to calculate the rotation matrix for coordinate alignment, and calculate the limb length standard proportion coefficient based on the height and gender of the patient, and linearly scale the attitude data according to the proportion coefficient.

5. The knee osteoarthritis rehabilitation training evaluation method based on the pose sensor and virtual interaction platform according to claim 4, wherein S3 It comprises: S31, when wearing for the first time, the user completes the static standard posture of T-shaped standing, sitting posture (knee joint flexion 90°) and / or affected side single leg support, and the initial posture data is obtained through the posture sensor module Meanwhile, the basic information and pathological characteristic information of the user are also obtained, wherein the basic information includes the height h, weight w and / or gender s corresponding to the user; the pathological characteristic information includes the information of the affected side of the limb, the subjective pain score VAS and / or the maximum knee joint flexion angle θ max ; the initial posture data includes the limb segment position vector P raw corresponding to the user's body shape and the gravity direction vector S32, construct a homogeneous transformation matrix from the actual coordinate system of the user to the reference coordinate system, which is defined as follows: Wherein, R is the rotation matrix, which is used for attitude direction correction; t is the translation vector, which is used for limb length proportion compensation; The rotation matrix R minimizes the gravity direction error measured in the static posture by Kabsch algorithm to solve the optimal rotation matrix, which is defined as follows: is the ideal gravity vector; is the gravity direction vector measured by the pose sensor module; The translation vector t is determined based on the height h and gender s, which is defined as follows: t = a(s) - L std (h) wherein a(s) is a gender difference coefficient, which is a preset value; L std (h) is a standard bone segment length vector corresponding to the height h, which is obtained by a preset query table. The formula for preliminary correction of the attitude data is defined as follows: P corrected = R · P raw + t where P corrected is the corrected pose data, P raw is the initial limb segment position vector, R is a rotation matrix, and t is a translation vector.

6. The knee osteoarthritis rehabilitation training evaluation method based on the pose sensor and virtual interaction platform according to claim 5, wherein, S3 also includes: S33, pathological correction is introduced to the affected side knee joint action, the compensation time of the knee joint action start delay is calculated according to the subjective pain score VAS of the patient, and the compensation index is calculated according to the ratio of the maximum flexion angle of the hip joint to the knee joint, when the compensation index exceeds the preset threshold, the system triggers voice or visual prompt to correct the abnormal action; The calculation of the compensation time Δt is defined as follows: ​ Wherein, k is an adjustment coefficient, its value range is 0.2~0.4;VAS is subjective pain score, its score range is 0 to 10, indicating the degree of no pain to severe pain; The calculation of the compensation index C is defined as follows: Wherein, θ hip , θ knee are the maximum hip and knee flexion angles of the user during the action period of the action, respectively; when C>1.5, the set threshold is exceeded, triggering voice and visual prompts to guide the user to correct the action to ensure training safety.

7. The knee osteoarthritis rehabilitation training evaluation method based on the pose sensor and virtual interaction platform according to any one of claims 4 to 6, characterized in that, In S3, the corrected posture data is also normalized to the [0, 1] interval; S3 also includes: according to the action data in the posture data and the standard action data in the corresponding preset reference data, the alignment transformation matrix T is updated, which is defined as follows: wherein η t is a time-dependent learning rate, is observation motion data at time t in the pose data, is standard motion data at time t in the preset reference data, T t is a homogeneous transformation matrix at time t, t+1 is a homogeneous transformation matrix at time t+1 after updating.

8. The knee osteoarthritis rehabilitation training evaluation method based on the pose sensor and virtual interaction platform according to claim 7, wherein, In S4, the posture data generated in the rehabilitation training process is preprocessed and corrected, and sent to the three-dimensional virtual interaction platform for display, the virtual interaction platform extracts different training schemes according to the preset cognitive stage, contact stage and automation stage to provide standard taijiquan rehabilitation action demonstration, voice prompt and visual guidance, and generates a rehabilitation evaluation report containing action similarity, motion stability and / or VAS score information after each training cycle.

9. The knee osteoarthritis rehabilitation training evaluation method based on the pose sensor and virtual interaction platform according to claim 8, wherein, When displayed in the virtual interaction platform, different types of scenes are also generated, and audio information is output according to the preset conditions to assist in forming the scene atmosphere.

10. A knee osteoarthritis rehabilitation training evaluation system based on a pose sensor and a virtual interaction platform, characterized in that: It includes: The pose sensor module, the number is multiple, multiple pose sensor modules are used for user to wear on the limbs, the pose sensor module is used for collecting three-dimensional motion data of user's limbs, which includes one or more of joint angle, angular velocity and acceleration; Zigbee communication module, respectively, with multiple pose sensor modules are in communication connection, and used to upload the pose data collected by the pose sensor module; Data processing module, for receiving pose data, and applying double-stage extended Kalman filtering algorithm to the data collected by the pose sensor module to obtain preprocessed posture data;It is also used to correct the posture data according to the preset conditions, pathological action compensation and data normalization processing combined with the individual characteristics of the user; Virtual interaction training module, for loading virtual interaction platform, providing standardized rehabilitation action demonstration, voice and visual prompt, and real-time comparison and guidance according to the training scheme of cognitive stage, contact stage and automation stage with patient action; Data storage module, for collecting the posture data generated in the rehabilitation training process of user wearing pose sensor module; Data evaluation module, for evaluating the posture data generated in the rehabilitation training process of user wearing pose sensor module, generating a rehabilitation evaluation report containing action similarity, motion stability and / or VAS score information after each training cycle of the user; Medical auxiliary module, for connecting with the data evaluation module, and transmitting the rehabilitation evaluation report to the remote doctor end and realizing the adjustment of training parameters.

Images