Somatosensory data driven rehabilitation action real-time evaluation method and system

Abstract

The application relates to the computer technical field and discloses a somatosensory data driven rehabilitation action real-time evaluation method and system, aiming to solve the technical problems of insufficient abnormal mode recognition accuracy, limited data dimension, calculation delay and non-uniform evaluation standard in existing rehabilitation action evaluation. The method comprises the following steps: collecting patient rehabilitation action data in real time by using a somatosensory sensor, constructing a multi-dimensional action feature data set containing three-dimensional coordinates, motion trajectories, joint angles, speeds and accelerations; adaptively carrying out wavelet denoising, energy segmentation and space-time alignment preprocessing on the data; and extracting space-time feature parameters such as action amplitude, trajectory, time sequence and symmetry. The application can improve the abnormal action mode recognition accuracy, comprehensively reflect the kinematic characteristics of rehabilitation actions, meet the real-time evaluation requirement, establish a unified objective evaluation standard, and improve the consistency and comparability of rehabilitation effect evaluation.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, specifically relating to a real-time evaluation method and system for rehabilitation movements driven by somatosensory data. Background Technology

[0002] With the accelerating aging of society and the rising incidence of cardiovascular and cerebrovascular diseases and musculoskeletal disorders, rehabilitation medicine is playing an increasingly prominent role in the modern medical system. Rehabilitation medicine aims to help patients restore motor function and improve their ability to live independently through scientific training methods. Its core objective is to promote the reconstruction and compensation of patients' motor functions. The rehabilitation training process typically includes key stages such as functional assessment, program development, training implementation, and effect evaluation. Among these, accurate evaluation of rehabilitation movements is a crucial prerequisite for developing personalized rehabilitation plans and improving training effectiveness.

[0003] In recent years, the deep integration of rehabilitation medicine and information technology has given rise to interactive functional rehabilitation systems. These systems combine rehabilitation training with engaging interactive activities by constructing virtual training scenarios, effectively enhancing patients' enthusiasm and initiative in training. Among these systems, motion sensors, as key sensing components for acquiring patient movement information, play a crucial role in collecting real-time human motion data. In-depth analysis of the rehabilitation movement data captured by motion sensors allows for the quantitative extraction of characteristic parameters such as patient training posture, range of motion, and movement trajectory, thus providing a data foundation for the objective evaluation of rehabilitation movements.

[0004] However, existing somatosensory sensors still have significant shortcomings in the application of rehabilitation movement evaluation: (1) Rehabilitation movements are complex and diverse, and the movement patterns of different patients vary significantly. Existing systems lack the accuracy to identify abnormal movement patterns, making it difficult to achieve accurate rehabilitation effect evaluation; (2) Rehabilitation movement evaluation usually requires multi-dimensional analysis of the spatiotemporal characteristics of the movement, but the data collected by existing sensors has limited dimensions, making it difficult to fully reflect the kinematic characteristics of rehabilitation movements; (3) Rehabilitation training scenarios have high real-time requirements, and the movement analysis algorithms of existing systems have computational delays, making it difficult to meet the application requirements of real-time evaluation of rehabilitation movements; (4) The evaluation criteria for rehabilitation movements lack unified objective quantitative indicators, and the subjective judgments of different evaluators vary greatly, affecting the consistency and comparability of rehabilitation training effect evaluation. Summary of the Invention

[0005] The purpose of this invention is to provide a real-time evaluation method and system for rehabilitation movements driven by somatosensory data, which can effectively solve the problems in the background art mentioned above.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: a method and system for real-time evaluation of rehabilitation movements driven by somatosensory data, comprising the following steps: S1: The somatosensory sensors in the interactive rehabilitation system collect real-time data on the patient's rehabilitation movements, constructing a multi-dimensional movement feature dataset. This data includes three-dimensional spatial coordinates, motion trajectory, joint angles, movement speed, and acceleration information. The multi-dimensional movement feature dataset integrates this information through spatiotemporal coding. Specifically, the somatosensory sensors continuously monitor the patient's whole-body movement during rehabilitation training, simultaneously acquiring the patient's posture information and motion parameters through multiple sensory channels to construct a comprehensive dataset that fully describes the patient's movement characteristics. This dataset not only contains the precise position information of each joint in three-dimensional space but also records the complete motion trajectory curve of the movement, as well as multi-dimensional kinematic parameters such as angle changes, speed changes, and acceleration changes of each joint during the movement. By using spatiotemporal coding, this information from different dimensions and time points is uniformly integrated to form a structured multi-dimensional movement feature dataset, laying the data foundation for subsequent feature extraction and movement evaluation.

[0007] S2: Preprocessing of the collected rehabilitation movement data includes denoising filtering, motion segmentation, and spatiotemporal alignment to obtain a standardized movement sequence. Denoising filtering employs an adaptive threshold wavelet transform method. Motion segmentation is based on the rate of change of movement energy. Spatiotemporal alignment maps movement data from different sampling times to a unified time reference. Due to various interference factors in the rehabilitation training environment, noise signals inevitably enter the collected raw movement data, affecting the accuracy of subsequent analysis. Therefore, denoising filtering is necessary to purify the raw data. This invention uses an adaptive threshold wavelet transform method for denoising. This method can automatically adjust the threshold parameters according to the statistical characteristics of wavelet coefficients, effectively removing noise while preserving the original characteristics of the movement signal to the maximum extent. The purpose of motion segmentation is to divide the continuous rehabilitation movement sequence into several independent movement units, facilitating individual evaluation of each movement. This invention performs motion segmentation based on the rate of change of movement energy. By calculating the energy function of the movement sequence and detecting the peak and trough values ​​of the energy curve, the starting and ending points of each movement are identified, and the energy trough values ​​between adjacent movements serve as the segmentation boundaries. Spatiotemporal alignment maps motion data from different sampling times to a unified time base, solving the problem of time axis deviation caused by sampling rate fluctuations or clock asynchrony, and ensuring that motion data collected at different times can be compared and analyzed on a unified time scale.

[0008] S3: Based on standardized movement sequences, spatiotemporal feature parameters of rehabilitation movements are extracted, including movement amplitude features, movement trajectory features, movement temporal features, and movement symmetry features. Movement amplitude features are calculated using joint displacement vectors; movement trajectory features are extracted using a dynamic time warping algorithm; movement temporal features are described based on the state transition probability distribution of a Hidden Markov Model; and movement symmetry features are reflected by the ratio of the motion parameters of the left and right joints. Movement amplitude features reflect the range and intensity of the rehabilitation movement. By calculating the displacement vector amplitude of joints between adjacent frames and statistically analyzing the maximum, minimum, and average displacements within a movement cycle as feature values, the amplitude characteristics of the movement can be comprehensively described. Movement trajectory features characterize the spatial movement path of the rehabilitation movement. The similarity distance between the test movement and the standard movement is calculated using a dynamic time warping algorithm, and the distance value is used as the trajectory feature. This method can effectively handle the problem of inconsistent movement execution speeds. Movement temporal features describe the evolution of rehabilitation movements in the time dimension. A Hidden Markov Model is used to model the temporal dynamics of the movement, extracting feature parameters such as state duration and state transition probability. Movement symmetry reflects the body's balance in rehabilitation movements. It is characterized by calculating the motion parameters of the corresponding joints on the left and right sides respectively, and calculating the ratio or difference of the parameters on the left and right sides. This is of great significance for evaluating rehabilitation effects.

[0009] S4: Construct a rehabilitation movement evaluation model, employing a multi-task neural network architecture to achieve movement classification and quantitative scoring. The multi-task neural network includes a feature encoding module, task-specific branches, and a scoring output layer. The feature encoding module uses a spatiotemporal graph convolutional network to extract global and local features of the movements. The task-specific branches are used for abnormal movement pattern recognition and rehabilitation effect quantitative scoring, respectively. The scoring output layer outputs standardized scores. The rehabilitation movement evaluation model, designed with a multi-task neural network architecture, can simultaneously achieve two tasks: abnormal movement pattern recognition and rehabilitation effect quantitative scoring. The feature encoding module uses a spatiotemporal graph convolutional network, which constructs a graph structure based on the topological relationships of the human skeleton. Each joint in the human body is used as a graph node, and the skeletal connections between joints are used as graph edges. Convolution operations are performed on this graph structure to extract global and local features of the movements. The task-specific branches include an abnormal movement pattern recognition branch and a rehabilitation effect quantitative scoring branch, each responsible for different evaluation tasks. The output of the abnormal movement pattern recognition branch is used to determine whether the patient's movements are abnormal, while the output of the rehabilitation effect quantitative scoring branch provides a quantitative score of the rehabilitation effect. The scoring output layer outputs the scoring results in a standardized form, facilitating subsequent feedback presentation and effect comparison analysis.

[0010] S5: Based on the output of the rehabilitation movement evaluation model, real-time evaluation feedback information is generated and presented to the patient through the feedback module of the interactive rehabilitation system. Real-time evaluation feedback information includes movement correctness judgment, quantitative scoring, and improvement suggestions. The feedback module presents feedback information through at least one of the visual, auditory, and tactile channels. The generation frequency of real-time evaluation feedback information matches the patient's movement rhythm, ensuring that the patient receives timely evaluation feedback during movement execution. Movement correctness judgment is determined based on the output of the abnormal movement pattern recognition branch, with judgment results including three levels: correct, slight deviation, and severe deviation, helping the patient understand the quality of their current movement execution. Quantitative scoring is determined based on the output of the rehabilitation effect quantitative scoring branch, presenting the rehabilitation effect in a standardized score format. Improvement suggestions are automatically generated based on the specific type of abnormal movement pattern, including suggestions for adjusting movement amplitude, correcting movement trajectory, and optimizing movement timing, providing targeted improvement guidance for the patient. The feedback module delivers feedback information to the patient through a multi-channel presentation method, selecting one or more combinations of visual, auditory, or tactile channels to ensure that feedback information is effectively conveyed to the patient.

[0011] In step S1, the motion sensor is a fusion sensing device combining a depth camera and an inertial measurement unit (IMU). The depth camera has a predetermined resolution and a predetermined frame rate. The IMU includes a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer, with a preset sampling rate. The two sensors are fused using a Kalman filter algorithm to obtain a high-precision 3D human pose estimation result. This fusion sensing device can acquire human motion information from different angles and dimensions, forming a complementary advantage. The depth camera provides rich spatial visual information, while the IMU provides precise motion inertial information. The fusion of these two sensors using a Kalman filter algorithm effectively compensates for the shortcomings of a single sensor in terms of accuracy and stability, resulting in a more accurate and reliable 3D human pose estimation result, providing high-quality data input for subsequent rehabilitation movement evaluation.

[0012] In step S2, the adaptive threshold wavelet transform uses a preset wavelet basis function, the number of decomposition layers is a preset number, and the threshold calculation uses a soft thresholding method. The threshold coefficient is adaptively determined based on the statistical characteristics of the wavelet coefficients at each layer. During motion segmentation, the energy function of the action sequence is first calculated, and then the start and end points of the action are identified using a peak detection algorithm. The energy valley values ​​between adjacent actions are used as the segmentation boundaries. Spatiotemporal alignment uses an interpolation method to resample the original action sequence to a uniform time length. In the wavelet transform denoising method, the selection of the wavelet basis function and the determination of the number of decomposition layers have a significant impact on the denoising effect. This invention selects a preset wavelet basis function and a preset number of decomposition layers based on the characteristics of rehabilitation action signals, and uses a soft thresholding method to calculate the threshold. The threshold coefficient is adaptively determined based on the statistical characteristics of the wavelet coefficients at each layer, achieving a good balance between denoising effect and signal preservation. Motion segmentation is achieved through energy function analysis and peak detection algorithms, which can accurately identify the boundaries of each independent action. Spatiotemporal alignment uses an interpolation method to unify action sequences of different lengths to the same time length, facilitating subsequent comparative analysis.

[0013] In step S3, the calculation of the motion amplitude feature specifically involves: calculating the displacement vector amplitude of joints between adjacent frames, and statistically analyzing the maximum, minimum, and average displacements within a motion cycle as feature values; calculating the similarity distance between the test motion and the standard motion using a dynamic time warping algorithm, with the distance value serving as the trajectory feature; using a Gaussian mixture hidden Markov model for the motion temporal feature, which contains a preset number of hidden states, each corresponding to a typical motion phase, and extracting state duration and state transition probability as temporal features; and calculating the motion parameters of the corresponding joints on the left and right sides for the motion symmetry feature, and calculating the ratio or difference between the left and right side parameters. The motion amplitude feature, calculated through joint displacement vectors, accurately reflects the range of motion of the rehabilitation motion. The motion trajectory feature employs a dynamic time warping algorithm, which effectively addresses the problem of inconsistent motion execution speeds, and evaluates the quality of the motion trajectory by calculating the similarity distance between the test motion and the standard motion. The motion temporal feature uses a Gaussian mixture hidden Markov model, which captures the complex dynamic characteristics of motion in the time dimension, and comprehensively describes the temporal characteristics of the motion by extracting feature parameters such as state duration and state transition probability. Movement symmetry characteristics reflect the body's balance by comparing the motion parameters of the left and right joints, and have important reference value for evaluating rehabilitation effects.

[0014] In step S4, the spatiotemporal graph convolutional network's graph structure is constructed based on the topological relationships of the human skeleton, including a preset number of joint nodes and bone connection edges. Graph convolution uses a preset number of convolution kernels with a preset kernel size, and the activation function is a modified linear unit (MRU). The output dimension of the feature encoding module is a preset dimension. In the task-specific branches, the output dimension of the abnormal action pattern recognition branch is the number of abnormal types, using a softmax classifier; the output dimension of the rehabilitation effect quantification scoring branch is a preset dimension, using a regression predictor. The two branches share the parameters of the feature encoding module, and the total loss function during joint training is a weighted sum of classification loss and regression loss. The spatiotemporal graph convolutional network constructs a graph model based on the natural topological structure of the human skeleton, effectively utilizing prior knowledge of human movement to improve the accuracy of feature extraction. Graph convolution operations use multi-channel convolution kernels for feature extraction, and the MRU is used as the activation function to introduce nonlinear factors, enhancing the network's expressive power. The high-dimensional feature vector output by the feature encoding module is shared by the two task-specific branches, realizing feature reuse and parameter sharing, reducing model complexity and computational overhead. The abnormal action pattern recognition branch uses a softmax classifier to achieve multi-classification, while the rehabilitation effect quantification scoring branch uses a regression predictor to achieve continuous value prediction. The total loss function when the two are trained together is a weighted sum of classification loss and regression loss, which can optimize the performance of both tasks at the same time.

[0015] In step S5, the frequency of real-time evaluation feedback information generation matches the patient's movement rhythm, ensuring synchronization with the patient's movements. Movement correctness is determined based on the output of the abnormal movement pattern recognition branch in step S4, with results including three levels: correct, slight deviation, and severe deviation. Quantitative scoring is determined based on the output of the rehabilitation effect quantitative scoring branch in step S4, with a preset scoring range. Improvement suggestions are automatically generated based on the specific type of abnormal movement pattern, including suggestions for adjusting movement amplitude, correcting movement trajectory, and optimizing movement timing. The timeliness of real-time evaluation feedback is crucial for rehabilitation training. This invention ensures that the frequency of feedback information generation matches the patient's movement rhythm, enabling the patient to receive timely feedback during movement execution. Movement correctness is determined using a three-level classification method, distinguishing between different degrees of movement deviation. Quantitative scoring uses standardized scores within a preset scoring range, facilitating horizontal comparisons between different patients and different training stages. Improvement suggestions are automatically generated based on specific abnormality types, providing targeted and effective guidance for patients to quickly correct incorrect movement habits and improve rehabilitation training outcomes.

[0016] Compared with the prior art, the present invention has the following beneficial effects: By constructing a multi-dimensional motion feature dataset and applying a spatiotemporal graph convolutional network, this invention achieves accurate identification of complex and diverse rehabilitation movement patterns, effectively solving the problem of insufficient accuracy in identifying abnormal movement patterns in existing systems. The multi-dimensional motion feature dataset constructed in this invention integrates various information such as three-dimensional spatial coordinates, motion trajectory, joint angles, movement speed, and acceleration, comprehensively describing the characteristics of rehabilitation movements from multiple perspectives including spatial, temporal, and frequency domains. The spatiotemporal graph convolutional network can utilize the topological structure information of the human skeleton to extract global and local features of movements, demonstrating a significant advantage in distinguishing different abnormal movement patterns. Experimental results show that the method of this invention can significantly improve the accuracy of abnormal movement pattern recognition.

[0017] By fusing data from depth cameras and inertial measurement units (IMUs) and combining various spatiotemporal feature parameter extraction methods, a multi-dimensional analysis of the spatiotemporal characteristics of rehabilitation movements was achieved, effectively addressing the technical limitation of existing sensor data dimensions. The fusion sensing device of depth cameras and IMUs can acquire human motion information from both visual and inertial dimensions. These two types of information complement and verify each other, effectively improving data quality and reliability. The extraction methods for various spatiotemporal feature parameters cover multiple aspects such as movement amplitude, movement trajectory, movement sequence, and movement symmetry, comprehensively reflecting the kinematic characteristics of rehabilitation movements and providing rich feature evidence for evaluating rehabilitation effects.

[0018] By optimizing the algorithm flow and computational architecture, this invention achieves rapid processing of motion data and instant output of evaluation results, effectively solving the problem that existing systems' computational latency cannot meet real-time evaluation requirements. The algorithm design fully considers real-time requirements, employing efficient preprocessing methods, optimized feature extraction strategies, and a streamlined neural network structure to ensure the system can respond quickly with limited computational resources. The generation frequency of real-time evaluation feedback information matches the patient's movement rhythm, enabling the patient to receive timely evaluation feedback during movement execution, thus improving the efficiency and effectiveness of rehabilitation training.

[0019] By establishing a standardized quantitative score through a multi-task neural network, a unified objective evaluation index is created, effectively eliminating the influence of subjective judgment differences among different assessors. This invention employs a multi-task neural network architecture to organically combine abnormal movement pattern recognition and rehabilitation effect quantitative scoring, outputting standardized scoring results. These results do not rely on the assessor's subjective judgment, exhibiting good consistency and comparability. This facilitates horizontal comparison and longitudinal tracking of rehabilitation effects across different patients and at different training stages, providing an objective basis for adjusting and optimizing rehabilitation programs. Attached Figure Description

[0020] Figure 1This is a schematic diagram of the overall technical solution architecture proposed in this invention; Figure 2 This is a schematic diagram of the core principle framework of the multi-task neural network architecture in this invention; Figure 3 This is a logical flowchart of the motion sensor data acquisition and fusion in this invention; Figure 4 This is a flowchart illustrating the spatiotemporal feature extraction and quantitative scoring process for rehabilitation movements in this invention. Figure 5 This is a schematic diagram illustrating the real-time feedback interaction between the scenario-based interactive rehabilitation system and the patient in this invention. Detailed Implementation

[0021] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0022] Example 1

[0023] Please refer to Figures 1 to 5 This embodiment relates to a real-time evaluation system for somatosensory data-driven rehabilitation movements applied in a scenario-based interactive functional rehabilitation system in the Department of Rehabilitation Medicine. The core sensing module of this system is composed of a depth camera and an inertial measurement unit, and the two sensors work together through a predefined communication interface and data fusion algorithm.

[0024] As the front-end for acquiring spatial visual information, the depth camera uses a charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) image sensor as its photosensitive element. The effective pixel count is preset to a specific resolution level, typically 640 x 480 pixels or 1280 x 720 pixels. The depth camera's frame rate is set to a predetermined rate, typically 30 or 60 frames per second, and can be adaptively adjusted according to the speed of rehabilitation training movements. The depth camera is equipped with an infrared emitter that projects a structured light pattern. By analyzing the distortion of reflected light, it calculates the depth information of each pixel in the scene, generating a depth map corresponding to the visible light image. In the rehabilitation training scenario, the depth camera is installed directly above or to the side of the training area. The installation height is determined based on the patient's height and the type of training movement, typically ranging from 1.5 meters to 2.5 meters. The installation angle ensures complete capture of the patient's full-body movement.

[0025] The inertial measurement unit (IMU), serving as the front-end for acquiring motion inertial information, comprises three core sensor components: a triaxial accelerometer, a triaxial gyroscope, and a triaxial magnetometer. The triaxial accelerometer measures linear acceleration along the three orthogonal axes of the body coordinate system, with a preset range of ±16 times or ±8 times the gravitational acceleration. The sampling rate is set to a preset value, typically 200 Hz or 500 Hz. The triaxial gyroscope measures angular velocity around the three orthogonal axes of the body coordinate system, with a preset range of ±2000 degrees per second or ±1000 degrees per second. The sampling rate is consistent with that of the accelerometer. The triaxial magnetometer measures the Earth's magnetic field component, providing absolute heading reference information. The IMU is manufactured using microelectromechanical systems (MEMS) technology, with a typical package size of 3-5 mm in length, 3-5 mm in width, and 1-1.5 mm in height, allowing it to be attached to joints on the patient's body.

[0026] The depth camera and inertial measurement unit (IMU) are fused using a Kalman filter algorithm to obtain high-precision and low-latency 3D human pose estimation results. The Kalman filter algorithm employs a recursive Bayesian estimation framework, using sensor measurements as observation inputs and combining the system state transition model and the observation model to perform optimal state estimation.

[0027] In its implementation, the system maintains a state vector containing the three-dimensional coordinates and velocities of the human joints. The state transition model is built upon the constant velocity assumption, meaning that the human joints are assumed to move at approximately uniform speeds between adjacent sampling times. The state transition equation can be expressed as the current state equal to the state transition matrix multiplied by the previous state plus process noise. The observation model correlates sensor measurements with the system state. The two-dimensional image coordinates and depth values ​​of the joints provided by the depth camera serve as visual observation inputs, while the acceleration and angular velocity provided by the inertial measurement unit serve as inertial observation inputs. The Kalman filter prediction step predicts the current state and state covariance matrix based on the state transition model, and the correction step calculates the Kalman gain based on the observation model and updates the state estimate by fusing sensor measurements.

[0028] Data fusion from two sensors can effectively compensate for the inherent limitations of a single sensor. While a depth camera provides accurate spatial position information under good lighting conditions, measurement errors may occur in low light, motion blur, or occlusion. An inertial measurement unit (IMU) exhibits high short-term dynamic response characteristics, maintaining attitude tracking continuity even when visual information is lost, but it suffers from cumulative drift. Through collaborative fusion using Kalman filtering, the system can maintain stable and reliable 3D attitude estimation performance under various operating conditions.

[0029] The patient rehabilitation movement data collected in real time by motion sensors is fused and processed to construct a multi-dimensional movement feature dataset. This dataset integrates various types of information, such as three-dimensional spatial coordinates, movement trajectory, joint angles, movement speed, and acceleration, through spatiotemporal coding.

[0030] Three-dimensional spatial coordinate information describes the three-dimensional position of each joint of the patient in the camera coordinate system. The system-defined human skeleton model contains a pre-set number of joints, typically twenty-five or nineteen. The spatial coordinates of each joint are stored as three-dimensional coordinate vectors in the time-series structure of the dataset. Motion trajectory information records the spatial position change path of each joint over continuous time, forming a trajectory curve by tracking the position changes of the joints in consecutive frames. Joint angle information is obtained by calculating the angle formed by adjacent joints. Typical human joint angles include shoulder flexion-extension angle, elbow flexion-extension angle, hip flexion-extension angle, knee flexion-extension angle, and ankle dorsiflexion-plantarflexion angle. Movement velocity information is obtained by calculating the time derivative of the joint position, representing the movement rate of the joint in three-dimensional space. Movement acceleration information is obtained by calculating the time derivative of the velocity, representing the rate of change of the movement velocity.

[0031] Spatiotemporal encoding integrates information from different dimensions and time points into a structured dataset. The encoding uses a time-axis-centric storage structure, with each timestamp corresponding to a data frame containing all feature information from relevant nodes at that moment. This encoding method facilitates subsequent time series analysis and feature extraction while preserving the complete spatiotemporal information of the action data.

[0032] Example 2

[0033] The raw motion data inevitably contains noise signals, which may originate from factors such as sensor measurement noise, environmental electromagnetic interference, and reflections from the patient's clothing. To ensure the accuracy of subsequent analysis, the raw data needs to be denoised and filtered.

[0034] This embodiment employs an adaptive threshold wavelet transform method for denoising. This method automatically adjusts the threshold parameter based on the statistical characteristics of the wavelet coefficients, effectively removing noise while preserving the original features of the motion signal to the maximum extent. The basic principle of wavelet transform is to decompose the original signal into components of different frequency scales. Noise signals are usually distributed in high-frequency components, and denoising can be achieved by thresholding the high-frequency components.

[0035] In the specific implementation, a preset wavelet basis function is first selected, typically the DB4 or DB6 wavelet from the Daubechies wavelet family. This wavelet basis function has tight support characteristics and good regularity, making it suitable for processing smooth and continuous signals such as human motion. The number of wavelet decomposition levels is set to a preset number, typically three to five. Too few decomposition levels will result in ineffective denoising, while too many decomposition levels may cause signal distortion.

[0036] The threshold calculation employs a soft thresholding method. The mathematical expression for the soft thresholding function is: when the absolute value of a wavelet coefficient is greater than the threshold, the value is the sign multiplied by the absolute value in parentheses minus the threshold; when the absolute value of a wavelet coefficient is less than or equal to the threshold, the value is zero. The threshold coefficient is adaptively determined based on the statistical characteristics of the wavelet coefficients at each layer. Specifically, the median of the absolute values ​​of the wavelet coefficients at the current layer is calculated, and this median is divided by 0.675 to obtain the threshold estimate. This median-based threshold estimation method exhibits good robustness and can adapt to denoising requirements under different noise levels.

[0037] After wavelet transform denoising, random noise in the original signal is effectively suppressed, while the edge features and motion trends of the action signal are preserved, creating favorable conditions for subsequent motion segmentation and feature extraction.

[0038] During rehabilitation training, patients' movements are usually continuous, consisting of multiple independent movement units. The purpose of motor segmentation is to divide the continuous sequence of movements into several independent movement units, making it easier to evaluate each movement individually.

[0039] This embodiment performs motion segmentation based on the rate of change of motion energy. The motion energy function is calculated by summing the squares of the velocity vectors at each joint, as expressed in the following formula:

[0040] Where E(t) represents the motion energy at time t, and N represents the total number of joints. This represents the velocity vector of the i-th joint at time t. The motion energy function reflects the distribution of motion intensity throughout the entire motion sequence; the energy value is higher when the patient performs active movements and lower when in a static or transitional state.

[0041] Peak detection algorithms are used to identify extreme points in the motion energy curve, thereby determining the start and end points of the motion. The algorithm first smooths the energy curve to eliminate the influence of minor fluctuations, then sets two parameters: an energy threshold and a peak threshold. The energy threshold determines whether the current moment is in a valid motion state, and the peak threshold determines whether the current extreme point constitutes a valid peak. When the energy curve rises from low to high and breaks through the energy threshold, the motion is considered to have started; when the energy curve falls from high to low and falls below the energy threshold, the motion is considered to have ended. Energy valleys between adjacent motions serve as dividing boundaries, separating the continuous motion sequence into several independent motion segments.

[0042] Due to fluctuations in sampling rate or clock asynchrony, motion data from different sampling times may exhibit time axis deviation. Spatiotemporal alignment maps motion data from different sampling times to a unified time base, ensuring that motion data collected at different times can be compared and analyzed on a uniform time scale.

[0043] This embodiment employs an interpolation method for spatiotemporal alignment. Specifically, the target time length is first determined, which can be set to the duration of a standard action or the average duration of all action segments. Then, the original action sequence is resampled, mapping the original discrete time points onto a unified time grid. The resampling uses cubic spline interpolation, a method that can accurately reconstruct the continuous characteristics of the original signal while maintaining signal smoothness.

[0044] After spatiotemporal alignment, all action sequences are standardized to the same time length, facilitating subsequent feature extraction and action comparison analysis. Simultaneously, spatiotemporal alignment eliminates timeline jitter caused by unstable sampling rates, improving data temporal consistency.

[0045] Example 3 The amplitude of movement reflects the range and intensity of rehabilitation movements and is an important indicator for evaluating rehabilitation effectiveness. This embodiment obtains the amplitude of movement characteristics by calculating the joint displacement vectors.

[0046] The specific calculation method is as follows: First, calculate the displacement vector of each joint point between adjacent frames. The displacement vector is equal to the joint point position in the current frame minus the joint point position in the previous frame. Then, calculate the amplitude of the displacement vector, i.e., the Euclidean norm of the displacement vector. This amplitude represents the displacement distance of the joint point per unit time. Finally, calculate the maximum, minimum, and average displacement within a motion cycle as the motion amplitude feature values.

[0047] Maximum displacement reflects the limit of the movement's range of motion and is important for assessing joint mobility recovery. Minimum displacement reflects the stability of the movement; an excessively small minimum displacement may indicate an overly static movement, which is detrimental to training effectiveness. Average displacement reflects the overall intensity of the movement and is an important indicator for measuring training volume. These three characteristics together constitute a complete description of the movement's amplitude characteristics.

[0048] Movement trajectory features are used to characterize the spatial movement path of rehabilitation movements, reflecting the accuracy and standardization of movement execution. This embodiment uses a dynamic time warping algorithm to extract movement trajectory features.

[0049] Dynamic Time Warping (RTW) is an algorithm used to compare the similarity of two time series. Its core idea is to align the two series in the time dimension through nonlinear time warping, and then calculate the aligned distance. The algorithm first constructs a local distance matrix, where the elements are the feature distances between corresponding time points in the two series. Then, it uses dynamic programming to find the optimal alignment path, which satisfies boundary conditions, monotonicity conditions, and continuity conditions.

[0050] In rehabilitation movement evaluation applications, the Dynamic Time Warping (VTW) algorithm is used to calculate the similarity distance between the test movement and a standard movement reference template. The standard movement template, formed by collecting and processing data from standard movements demonstrated by rehabilitation therapists, represents the correct movement execution pattern. The VTW distance serves as a trajectory feature; a smaller distance value indicates a closer similarity between the test movement and the standard movement, resulting in a more standardized movement trajectory. This algorithm effectively addresses the issue of inconsistent movement execution speeds, accurately assessing the accuracy of the trajectory even when patients perform movements at different speeds.

[0051] Movement temporal features describe the evolution of rehabilitation movements over time, reflecting the rhythmic and phase characteristics of the movements. This embodiment uses a Gaussian mixture hidden Markov model to extract movement temporal features.

[0052] A Gaussian mixture hidden Markov model is a statistical time series model containing a predetermined number of hidden states, each corresponding to a typical action phase. The model's state transition probabilities describe the probability distribution of switching from one action phase to another, while the state emission probabilities use a Gaussian mixture distribution to describe the distribution characteristics of observed features in a given state.

[0053] In the implementation, a Gaussian Mixture Hidden Markov Model (GMMM) is first trained using a large number of standard action samples. The Baum-Welch algorithm is then used to optimize the model parameters to maximize the likelihood function. After training, the model can compute the corresponding state sequence for any input action sequence. Then, state duration and state transition probabilities are extracted from the input action sequence as temporal features. The state duration feature reflects the dwell time of each action phase, and the state transition probability feature reflects the switching pattern of the action phase. These features together describe the temporal dynamics of the action.

[0054] Movement symmetry reflects the body's balance during rehabilitation movements and is of great significance for assessing the rehabilitation effect in patients with hemiplegia or fractures. This embodiment obtains movement symmetry characteristics by calculating the ratio or difference of motion parameters of corresponding joints on the left and right sides.

[0055] The specific calculation method involves first extracting the motion parameters of the corresponding joints on the left and right sides, including joint angles, range of motion, and movement speed. Then, the relative ratio or absolute difference between the parameters on the left and right sides is calculated as a symmetry feature. Quantitative parameters are calculated using ratios, such as the maximum flexion angle of the left knee divided by the maximum flexion angle of the right knee; qualitative parameters are calculated using differences, such as the difference in flexion angles between the left and right knees.

[0056] The closer the symmetry characteristic value is to one or zero, the more symmetrical the left and right movements are, and the better the rehabilitation effect. A value that deviates significantly from one or zero indicates an imbalance in left and right movements, which needs to be addressed in subsequent training. By continuously monitoring changes in symmetry characteristics, the improvement in body balance during the rehabilitation process can be objectively quantified.

[0057] Example 4 The rehabilitation movement evaluation model is implemented using a multi-task neural network architecture, which consists of two main parts: a feature encoding module and task-specific branches. The feature encoding module uses a spatiotemporal graph convolutional network to extract global and local features of the movement, providing a unified feature representation for the subsequent task-specific branches.

[0058] The graph structure of spatiotemporal graph convolutional networks is constructed based on the topological relationships of the human skeleton. In this graph structure, each joint of the human body serves as a graph node, and the skeletal connections between joints are graph edges. Assuming the human skeleton model contains a predetermined number of joints, the graph structure contains a corresponding number of nodes and edges connecting these nodes. The adjacency matrix of the graph is defined based on the natural connectivity of the human skeleton, and the elements in the adjacency matrix indicate whether there is a skeletal connection between two corresponding nodes.

[0059] Graph convolution operations are performed on the graph structure described above. The kernel size and the number of kernels are preset. The convolution operation employs a spatial domain graph convolution method, where the kernel weights are weighted and summed with the features of neighboring nodes. Each graph convolutional layer is followed by a modified linear unit activation function, introducing nonlinearity to enhance the network's expressive power. By stacking multiple graph convolutional layers, the network can extract multi-level features ranging from local details to global overview.

[0060] Spatiotemporal graph convolutional networks perform graph convolutions in both the spatial and temporal dimensions to capture the joint spatiotemporal features of actions. Temporal convolution uses a one-dimensional kernel to perform sliding window convolutions on features from consecutive frames along the time axis. This combination of spatial and temporal convolutions enables the network to simultaneously learn the spatial structural features and temporal dynamic features of actions.

[0061] The output dimension of the feature encoding module is a preset dimension, outputting a high-dimensional feature vector that contains global and local spatiotemporal feature representations of the input action.

[0062] The task-specific branches consist of two parts: an abnormal movement pattern recognition branch and a rehabilitation effect quantification scoring branch, each responsible for different evaluation tasks. The two branches share the parameters of the feature encoding module and achieve multi-task learning through joint training.

[0063] The abnormal action pattern recognition branch takes a high-dimensional feature vector as input from the feature encoding module. The branch's network structure contains several fully connected layers, with the output dimension being the number of abnormal types. The output layer uses a softmax classifier to map the input features to probability distributions for each category. Abnormal types include correct actions and various typical erroneous action patterns, such as insufficient action amplitude, excessive action amplitude, action trajectory deviation, and disordered action sequence. The branch outputs the category with the highest probability as the recognition result, and also outputs probability values ​​for each category for subsequent confidence assessment.

[0064] The input to the quantitative scoring branch for rehabilitation effectiveness is also a high-dimensional feature vector output from the feature encoding module. The branch's network structure contains several fully connected layers, with a preset output dimension. The output layer uses a regression predictor to output continuous score values. The scoring range is set to a preset range, such as zero to one hundred or zero to ten. The regression predictor learns the mapping relationship between the feature vector and the score value to achieve a quantitative assessment of rehabilitation effectiveness.

[0065] The two branches share the parameters of the feature encoding module, optimizing all network parameters simultaneously during joint training. The total loss function is a weighted sum of the classification loss and the regression loss, expressed by the following formula:

[0066] in, The cross-entropy loss represents the loss for a classification task. This represents the mean squared error loss for the regression task, where α and β are the weighting coefficients for the two types of losses. These weighting coefficients can be adjusted according to task priority, typically configured to make the classification and regression losses roughly equivalent in magnitude.

[0067] The advantage of joint training lies in the mutual enhancement between the two tasks. The feature representations learned by the feature encoding module serve both classification and regression tasks simultaneously. This multi-task learning paradigm improves the generalization ability of features and avoids overfitting. Experiments show that models trained jointly typically outperform models trained separately on both tasks.

[0068] Example 5 Based on the output of the rehabilitation movement evaluation model, the system generates real-time evaluation feedback information. The frequency of feedback generation matches the patient's movement rhythm, ensuring that the patient receives timely evaluation feedback during movement execution.

[0069] The correctness of an action is determined based on the output of the abnormal action pattern recognition branch. The recognition branch outputs the probability distribution of each category, and the system selects the category with the highest probability as the judgment result. The judgment result is divided into three levels: correct, slightly biased, and severely biased. When the recognition result is a correct action and the probability is higher than a preset threshold, it is judged as correct; when the recognition result is an incorrect action but the probability is low, or the identified error type is slightly biased, it is judged as slightly biased; when the recognition result is an incorrect action with a high probability and the error type is severely biased, it is judged as severely biased.

[0070] The quantitative score is determined based on the output of the quantitative score branch for rehabilitation effectiveness. Continuous score values ​​output by the regression predictor are mapped to a preset score range, such as a score interval of 0 to 100. The score results are presented in numerical form and can be supplemented with a grading system, for example, a score of 90 or above is classified as excellent, 80 to 90 as good, 60 to 80 as passable, and below 60 as failable.

[0071] Improvement suggestions are automatically generated based on the specific type of abnormal movement pattern. The system maintains a suggestion knowledge base, recording the improvement suggestions corresponding to various abnormal movement patterns. When the judgment result is a slight deviation or a serious deviation, the system retrieves the corresponding improvement suggestions from the knowledge base according to the identified abnormality type. Suggestion types include suggestions for adjusting the range of motion, suggestions for correcting the movement trajectory, and suggestions for optimizing the movement timing. For example, for insufficient range of motion, the system generates suggestions to prompt the patient to increase the range of motion; for deviation in movement trajectory, the system generates suggestions to prompt the patient to adjust the spatial path of the movement.

[0072] The feedback module presents feedback information through at least one of the visual, auditory, and tactile channels. This multi-channel presentation ensures that feedback information can be effectively conveyed to patients with different sensory abilities.

[0073] The visual feedback is presented through a graphical user interface (GUI). Interface elements include a correctness indicator icon for the current movement, a quantitative score, improvement suggestions text, and a demonstration animation of the standard movement. The correctness indicator icon uses a color-coded system: green for correct, yellow for slight deviation, and red for serious deviation. The quantitative score is presented as a digital dashboard or progress bar, visually displaying the current score's position within the score range. Improvement suggestions text is presented as a pop-up or sidebar for easy reading by the patient during training.

[0074] Auditory feedback utilizes either voice prompts or sound effects. Voice prompts are used to announce quantitative scores and improvement suggestions; a typical application scenario is when patients cannot easily view the visual interface, providing feedback through voice announcements. Sound effects are used to indicate the correctness of actions; for example, a pleasant sound is played for correct actions, and a warning sound is played for incorrect actions. The advantage of sound effects feedback is its extremely low response latency, providing feedback to the patient almost instantly upon execution of the action.

[0075] Tactile feedback is achieved using vibration motors or force feedback devices. The intensity and pattern of the vibration feedback are correlated with the correctness of the movement; for example, a slight, short vibration is given for correct movements, while a strong, continuous vibration is given for incorrect movements. Force feedback devices can be used in wearable rehabilitation devices to indicate the direction of movement deviation by applying resistance or assistance to the patient's limbs.

[0076] In practical applications, single-channel or multi-channel feedback methods can be selected based on the specific scenario and patient needs. For scenarios requiring simultaneous reception of multiple feedback information, a multi-channel combination approach can be used, such as simultaneously presenting visual scores and auditory cues, to achieve redundant transmission of feedback information and improve the reliability of information delivery.

[0077] Example 6 When the system is applied to upper limb rehabilitation training scenarios, the joint configuration of the human skeletal model focuses on the upper limb and shoulder girdle regions. A typical configuration includes approximately eleven main joints, such as the head joint, neck joint, bilateral shoulder joints, bilateral elbow joints, bilateral wrist joints, and finger joints. The types of movements in upper limb rehabilitation training mainly include shoulder flexion and extension, abduction and adduction, elbow flexion and extension, wrist flexion and extension, and forearm pronation and supination.

[0078] In this scenario, the depth camera is positioned with a focus on lateral perspective to ensure complete capture of the patient's upper limb range of motion. The inertial measurement unit (IMU) configuration prioritizes major upper limb joints, with one IMU at each joint to obtain precise local motion information. The feature extraction module focuses on extracting upper limb joint angles and motion trajectories, while the abnormal pattern library for evaluating the model focuses on common erroneous movement patterns of the upper limb, such as elbow hyperextension and shoulder compensatory movements.

[0079] When the system is applied to lower limb rehabilitation training scenarios, the joint configuration of the human skeletal model focuses on the lower limbs and pelvic region. A typical configuration includes pelvic joints, bilateral hip joints, bilateral knee joints, bilateral ankle joints, and foot joints, totaling approximately eleven major joints. The types of movements in lower limb rehabilitation training mainly include hip flexion and extension, abduction and adduction, knee flexion and extension, ankle dorsiflexion and plantarflexion, and gait cycle training.

[0080] In this scenario, the depth camera is positioned to capture a frontal or rear viewpoint, ensuring complete capture of the patient's lower limb range of motion, particularly the foot's contact with the ground. The inertial measurement unit (IMU) is configured to focus on the major joints and large muscle surfaces of the lower limbs, such as the quadriceps and hamstrings. The feature extraction module focuses on extracting lower limb joint angles, weight-bearing status, and gait spatiotemporal parameters. The abnormal pattern library for evaluating the model focuses on common erroneous movement patterns of the lower limbs, such as knee hyperextension, gait asymmetry, and foot drop.

[0081] When the system is applied to trunk rehabilitation training scenarios, the joint configuration of the human skeletal model focuses on the spine and core muscle groups. A typical configuration includes head joints, cervical joints, thoracic joints, lumbar joints, sacroiliac joints, and pelvic joints, totaling approximately seven to nine major joints. The types of movements in trunk rehabilitation training mainly include cervical flexion-extension and rotation, thoracic lateral flexion-rotation, lumbar flexion-extension and lateral flexion, and core stability training.

[0082] In this scenario, the depth camera needs to be positioned to capture the patient's torso from the front, side, and back; a multi-camera array is recommended for omnidirectional capture. The inertial measurement unit (IMU) configuration focuses on each vertebral segment of the spine, using a flexible IMU attached to the body surface. The feature extraction module focuses on extracting changes in spinal curvature, core muscle activation sequence, and trunk stability. The abnormal pattern library for evaluating the model focuses on common erroneous movement patterns of the trunk, such as spinal compensatory movements and delayed core muscle activation.

[0083] During actual operation, the system may encounter various abnormal situations, and it is necessary to establish a sound abnormality handling and security degradation mechanism to ensure the stable and reliable operation of the system.

[0084] When visual tracking fails due to occlusion of the depth camera or severely poor lighting conditions, the system automatically switches to pure inertial navigation mode. Although the inertial measurement unit (IMU) suffers from cumulative drift, its relative motion measurement accuracy is high over short periods, maintaining the continuity of motion tracking. When the IMU experiences data anomalies or communication interruptions, the system switches to pure vision mode, using depth information from the depth camera for attitude estimation.

[0085] When the evaluation model's output becomes unreliable due to data anomalies, the system employs conservative strategies for feedback. These strategies include delaying feedback, reducing feedback confidence, or pausing feedback. The delayed feedback strategy smooths the evaluation results over multiple consecutive sampling periods before outputting them; the reduced feedback confidence strategy adds uncertainty warnings to the feedback information; and the paused feedback strategy temporarily suppresses feedback output and prompts the system to enter a self-checking state.

[0086] The system also sets training safety boundaries, triggering an emergency stop mechanism immediately when a patient's movements exceed the safe range. These safety boundaries are preset based on the specific type of rehabilitation training and the patient's condition, such as boundaries for joint range of motion, amplitude of movement, and speed of motion. When movement parameters reach these safety boundaries, the system simultaneously sends an emergency stop signal through all feedback channels and controls the interactive rehabilitation system to pause the training process.

[0087] Example 7 The entire system consists of three layers: the perception layer, the processing layer, and the application layer. The perception layer includes a depth camera, an inertial measurement unit, and corresponding interface circuits. The processing layer includes an image processing computer, an inertial data fusion processor, and a neural network inference accelerator. The application layer includes evaluation software, a feedback presentation device, and a context-interactive rehabilitation system.

[0088] Depth cameras connect to the image processing computer via a Universal Serial Bus (USB) or Gigabit Ethernet interface. A typical version of the USB interface is version 3.0, with a theoretical bandwidth of up to five gigabits per second, sufficient for real-time transmission of high-resolution depth images. Gigabit Ethernet interfaces are suitable for scenarios requiring long-distance transmission or distributed deployment.

[0089] The inertial measurement unit (IMU) connects to the embedded processor via a serial peripheral interface or an integrated circuit's built-in bus. The serial peripheral interface uses a four-wire synchronous serial communication protocol with a communication rate of up to ten megabits per second. The integrated circuit's built-in bus uses a two-wire serial communication protocol, saving wiring resources and making it suitable for multi-sensor cascading scenarios.

[0090] Neural network inference accelerators employ either graphics processing units (GPUs) or dedicated neural network acceleration chips. GPUs feature massively parallel computing architectures, making them suitable for handling neural network inference tasks with intensive convolution operations. Dedicated neural network acceleration chips are optimized for neural network operators, offering significantly better energy efficiency than general-purpose GPUs.

[0091] The software system adopts a modular design, including a data acquisition module, a data preprocessing module, a feature extraction module, a model inference module, a feedback generation module, and a user interface module.

[0092] The data acquisition module is responsible for communicating with the sensor hardware to acquire raw data in real time. Internally, the module includes a depth camera driver, an inertial measurement unit driver, and a data synchronization mechanism. The driver encapsulates the hardware operation details and provides a unified data interface to the upper layer. The data synchronization mechanism coordinates the sampling timestamps of the two sensors to ensure data alignment in the time dimension.

[0093] The data preprocessing module implements three sub-functions: noise reduction filtering, motion segmentation, and spatiotemporal alignment. Each sub-function is encapsulated as an independent processing unit, supporting parameter configuration and algorithm replacement. The module design follows a pipelined architecture, with data flowing between sub-functions to minimize memory usage and processing latency.

[0094] The feature extraction module extracts features related to motion amplitude, motion trajectory, motion timing, and motion symmetry. The module receives a preprocessed, standardized motion sequence and outputs a multi-dimensional feature vector. The feature extraction algorithm has been optimized to complete computation within milliseconds, meeting real-time requirements.

[0095] The model inference module loads a pre-trained multi-task neural network model and performs forward inference calculations. The module supports both CPU and neural network accelerator operating modes, automatically selecting the optimal mode based on hardware configuration. Inference results include probability distributions of abnormal movement patterns and quantitative scores for rehabilitation effectiveness.

[0096] The feedback generation module generates evaluation feedback information based on the model's inference results. The module includes a suggestion knowledge base, retrieving corresponding improvement suggestions based on the identified anomaly types. The format and content of the feedback information are adapted according to user preferences.

[0097] The user interface module is responsible for presenting feedback information and receiving user operation commands. The interface design follows the operating habits of rehabilitation medicine physicians and therapists, providing intuitive visual displays and convenient operation access.

[0098] Before deployment, the system needs to undergo sensor calibration and model performance verification to ensure that the system meets the requirements for clinical application.

[0099] Depth camera calibration consists of two parts: intrinsic parameter calibration and extrinsic parameter calibration. Intrinsic parameter calibration determines the camera's internal parameters such as focal length, principal point coordinates, and distortion coefficients. Typically, the Zhang Zhengyou calibration method is used with a checkerboard calibration board. Extrinsic parameter calibration determines the camera's pose relative to the world coordinate system, establishing its precise spatial position within the rehabilitation training environment. After calibration, the system saves the calibration parameters to a configuration file for subsequent data processing.

[0100] The calibration of an inertial measurement unit (IMU) consists of two parts: zero-bias calibration and sensitivity calibration. Zero-bias calibration determines the sensor's output deviation in a stationary state, which needs to be compensated for in subsequent data processing. Sensitivity calibration determines the proportionality coefficient between the sensor output and the actual physical quantity, ensuring the accuracy of the measurement. The calibration process requires fixing the sensor in a standard position with a known attitude and calculating the sensitivity correction factor based on the difference between the output value and the theoretical value.

[0101] Model performance validation employs hold-out or cross-validation methods, using pre-labeled test datasets to evaluate the model's performance on two tasks: abnormal movement pattern recognition and rehabilitation effect quantification. Evaluation metrics include accuracy, precision, recall, root mean square error, and mean absolute error. The system can only be deployed clinically after validation results meet preset thresholds.

Claims

1. A real-time evaluation method for rehabilitation movements driven by somatosensory data, characterized in that, include: The rehabilitation system utilizes motion sensors in the contextual interactive function to collect patients' rehabilitation movement data in real time, and constructs a multi-dimensional movement feature dataset. The rehabilitation movement data includes three-dimensional spatial coordinates, movement trajectory, joint angles, movement speed and acceleration information. The multi-dimensional movement feature dataset integrates the information through spatiotemporal coding. The collected rehabilitation movement data is preprocessed, including noise reduction filtering, motion segmentation, and spatiotemporal alignment, to obtain a standardized movement sequence. Based on the standardized movement sequence, spatiotemporal feature parameters of rehabilitation movements are extracted, including movement amplitude features, movement trajectory features, movement temporal features, and movement symmetry features. A rehabilitation movement evaluation model was constructed, and a multi-task neural network architecture was used to achieve movement classification and quantitative scoring. Based on the output of the rehabilitation action evaluation model, real-time evaluation feedback information is generated and presented to the patient through the feedback module of the scenario-interactive rehabilitation system.

2. The real-time evaluation method for rehabilitation movements driven by somatosensory data according to claim 1, characterized in that, The motion sensor is a fusion sensing device combining a depth camera and an inertial measurement unit. The depth camera is used to acquire spatial visual information, and the inertial measurement unit includes a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer. The depth camera and the inertial measurement unit fuse data through a filtering fusion algorithm to obtain a three-dimensional human pose estimation result.

3. The real-time evaluation method for rehabilitation movements driven by somatosensory data according to claim 1, characterized in that, The preprocessing of the collected rehabilitation movement data includes: An adaptive threshold wavelet transform method is used for noise reduction filtering. Motion segmentation based on the rate of change of motion energy; An interpolation method is used to map motion data from different sampling times to a unified time base.

4. The real-time evaluation method for rehabilitation movements driven by somatosensory data according to claim 1, characterized in that, The motion amplitude characteristics are calculated through joint displacement vectors, including statistically analyzing the maximum, minimum, and average displacements within a motion cycle as feature values. The motion trajectory features are extracted using a dynamic time warping algorithm to determine the similarity distance between the test motion and the standard motion. The action timing features are described based on the state transition probability distribution of a hidden Markov model; The symmetry of the movement is reflected by the ratio or difference of the motion parameters of the left and right joints.

5. The real-time evaluation method for rehabilitation movements driven by somatosensory data according to claim 4, characterized in that, The action timing features use a Gaussian mixture hidden Markov model, which contains a preset number of hidden states. Each state corresponds to a typical action phase, and the state duration and state transition probability are extracted as timing features.

6. The real-time evaluation method for rehabilitation movements driven by somatosensory data according to claim 1, characterized in that, The multi-task neural network architecture includes a feature encoding module, task-specific branches, and a scoring output layer; The feature encoding module uses a spatiotemporal graph convolutional network to extract global and local features of the action; The task-specific branches include an abnormal movement pattern recognition branch and a rehabilitation effect quantitative scoring branch; The scoring output layer outputs standardized scores.

7. The real-time evaluation method for rehabilitation movements driven by somatosensory data according to claim 6, characterized in that, The graph structure of the spatiotemporal graph convolutional network is constructed based on the topological relationship of the human skeleton, and includes a preset number of key points and bone connection edges; The abnormal action pattern recognition branch is implemented using a classifier; The quantitative scoring branch for rehabilitation effectiveness is implemented using a regression predictor. The two task-specific branches share the parameters of the feature encoding module, and the total loss function during joint training is a weighted sum of the classification loss and the regression loss.

8. The real-time evaluation method for rehabilitation movements driven by somatosensory data according to claim 1, characterized in that, The real-time evaluation feedback information includes judgment of action correctness, quantitative score and improvement suggestions; The determination of the correctness of the action includes three levels: correct, slight deviation, and serious deviation. The feedback module presents the feedback information through at least one of the visual, auditory, and tactile channels.

9. A real-time evaluation system for rehabilitation movements driven by somatosensory data, characterized in that, include: The motion sensor data acquisition module is used to collect patients' rehabilitation movement data in real time using motion sensors and to construct a multi-dimensional movement feature dataset. The data preprocessing module is used to perform noise filtering, motion segmentation, and spatiotemporal alignment operations on the collected rehabilitation action data to obtain a standardized action sequence. The spatiotemporal feature extraction module is used to extract the amplitude features, trajectory features, temporal features, and symmetry features of the rehabilitation movements based on the standardized movement sequence. The rehabilitation movement evaluation model module is used to implement movement classification and quantitative scoring using a multi-task neural network architecture. The real-time feedback presentation module is used to generate real-time evaluation feedback information based on the output results of the rehabilitation action evaluation model, and present it to the patient through the feedback module.

10. The real-time evaluation system for rehabilitation movements driven by somatosensory data according to claim 9, characterized in that, The motion sensor data acquisition module includes a fusion sensing device of a depth camera and an inertial measurement unit; The depth camera and the inertial measurement unit fuse data through a filtering fusion algorithm to obtain a three-dimensional human pose estimation result; The spatiotemporal feature extraction module includes: The amplitude feature extraction unit is used to calculate the motion amplitude features through the joint displacement vector; The trajectory feature extraction unit is used to extract motion trajectory features using a dynamic time warping algorithm. The temporal feature extraction unit is used to describe the temporal features of actions based on the state transition probability distribution of a hidden Markov model. The symmetry feature extraction unit is used to reflect the symmetry features of the movement through the ratio or difference of the motion parameters of the left and right joints.

Images