A stroke rehabilitation training action standard degree real-time monitoring device

Abstract

The application discloses a kind of cerebral apoplexy rehabilitation training action standard degree real-time monitoring device, the device is by integrating inertial measurement, surface myoelectricity and visual capture module, the multimodal data of patient movement is synchronously collected, and by the real-time fusion analysis of deep learning model in edge computing unit, dynamic compare for preset standard action template, to accurately quantify joint range of motion, motion trajectory, muscle activation timing and posture stability Key indicators, such as, and provide immediate error correction guidance and compliance score through audiovisual feedback module;The application realizes high-precision monitoring of rehabilitation action full-dimension and objectification by the real-time closed loop constructed, effectively solves the problem that traditional method relies on subjective experience and feedback delay, greatly improves the standardization level of rehabilitation training and neural remodeling efficiency, provides intelligent technical support for clinical and home rehabilitation.

Description

Technical Field

[0001] This invention relates to the field of medical rehabilitation monitoring technology, and in particular to a real-time monitoring device for the standardization of stroke rehabilitation training movements. Background Technology

[0002] After a stroke, patients often suffer from residual limb motor dysfunction. Currently, in clinical and home rehabilitation settings, the assessment of patients' motor standardization mainly relies on the on-site visual observation and experience judgment of rehabilitation therapists.

[0003] Current technologies rely on therapists' subjective observation and lack objective, quantifiable evaluation standards. Assessment results are easily influenced by personal experience and fatigue, and it is difficult to achieve accurate recording and traceability of the entire training process. At the same time, devices based on single sensors (using only inertial measurement units or only electromyography sensors) have limited information dimensions and cannot comprehensively and synergistically reflect the overall state of nerve, muscle, and skeletal structures when performing complex functional movements. For example, focusing only on joint angles may overlook muscle compensation or abnormal coordination patterns; analyzing only electromyography signals may not be able to accurately determine the spatial position and movement trajectory of joints.

[0004] Therefore, in response to the problems mentioned above, this invention proposes a real-time monitoring device for the standardization of stroke rehabilitation training movements. Summary of the Invention

[0005] To overcome the problems of subjective bias and limited information dimensions in existing technologies, this invention proposes a real-time monitoring device for the standardization of stroke rehabilitation training movements. This device integrates multiple sensors and edge intelligence analysis to achieve high-precision, low-latency, fully automated quantitative assessment and real-time guidance of key indicators of rehabilitation movements.

[0006] The technical solution of this invention is: a real-time monitoring device for the standardization of stroke rehabilitation training movements, comprising: The data acquisition module is used to collect multimodal physiological and motor data of patients in real time when performing rehabilitation training movements. The data acquisition module includes: At least one inertial measurement unit is used to collect kinematic data of the limbs, including acceleration, angular velocity and attitude angle, with a sampling rate of not less than 100Hz; At least one surface electromyography sensor is used to acquire electromyographic signals of the target muscle group, with a sampling rate of not less than 1000 Hz; A visual capture module is used to acquire image data of the patient's body posture; An edge computing unit, electrically connected to the data acquisition module, is used to receive and process the multimodal physiological and motion data locally in real time. This edge computing unit is pre-installed with a deep learning model and a standard movement template database. The deep learning model is used to perform multimodal feature fusion and spatiotemporal alignment analysis on the input kinematic data, electromyographic signals, and posture image data. The edge computing unit also includes a data synchronization module, which assigns a unified high-precision timestamp to the data streams from different sensors to ensure temporal consistency of the kinematic data, electromyographic signals, and posture image data during multimodal fusion. The edge computing unit then compares the fused real-time data with a standard movement template retrieved from the standard movement template database that corresponds to the current training movement. The analysis and evaluation module, integrated into the edge computing unit, calculates and outputs evaluation values ​​of the following key indicators in real time based on the comparison results: joint range of motion, limb movement trajectory, muscle activation sequence of target muscle groups, and body posture stability. The audiovisual feedback module communicates with the edge computing unit to receive the evaluation values ​​output by the analysis and evaluation module, and generates real-time error correction guidance information and action compliance scores according to preset threshold rules, which are then presented to the patient and therapist through at least one audiovisual format.

[0007] Preferably, the inertial measurement unit and the surface electromyography sensor are integrated into a multi-parameter sensing node that can be worn on the target segment of the patient's limb. Each multi-parameter sensing node simultaneously outputs acceleration, angular velocity, magnetic field strength data and analog or digital electromyography signals. The wearing position of the multi-parameter sensing node is pre-set according to the main joints and active muscle groups involved in the standard rehabilitation movements, including but not limited to the shoulder, elbow, wrist, hip, knee and ankle.

[0008] Preferably, the visual capture module includes one or more depth cameras for constructing a three-dimensional skeleton model of the patient's body, and the posture image data is a time-series data stream containing three-dimensional coordinate information of at least 15 joints, with an output frequency of not less than 30Hz.

[0009] Preferably, the deep learning model is a hybrid neural network model based on the fusion of spatiotemporal graph convolutional network and long short-term memory network. Its input layer receives nine-axis data sequences from the inertial measurement unit, signal envelope sequences from the surface electromyography sensor, and joint coordinate sequences from the visual capture module. The data are fused through parallel processing and multi-head attention mechanism by the feature extraction layer. Finally, the output layer generates an embedding vector in the same feature space as the standard action template. The model is trained and optimized using a multimodal dataset containing the actions of healthy subjects and stroke patients before deployment.

[0010] Preferably, the audiovisual feedback module includes a display terminal and an audio output device. The display terminal is used to display in real time a comparison animation between the patient's current movement and a standard movement template, the numerical deviation curves of the joint range of motion and movement trajectory, a muscle activation time sequence diagram, a posture stability heatmap, and a real-time updated compliance score. The audio output device is used to output personalized voice correction prompts when key indicators are detected to deviate from the preset range.

[0011] Preferably, the standard movement template database stores multimodal data templates generated from standard movements performed by professional therapists. Each template is associated with a specific type and difficulty level of rehabilitation training movement and includes standard values ​​or allowable ranges for joint range of motion, movement trajectory, muscle activation sequence, and postural stability. Before comparing real-time data with the standard movement templates, the edge computing unit first performs coordinate system alignment and gravity compensation preprocessing on the kinematic data, and performs bandpass filtering, full-wave rectification, and amplitude standardization preprocessing on the electromyographic signals.

[0012] Preferably, the evaluation values, error correction guidance information, and compliance scores output by the device are synchronously transmitted to a remote server via a wireless communication module to generate long-term rehabilitation training reports and efficacy evaluation charts.

[0013] The beneficial effects of this invention are: 1. This invention constructs a comprehensive evaluation system covering kinematics, neuromuscular and postural stability by fusing multimodal data from inertial measurement, electromyography and visual capture. It overcomes the shortcomings of single sensor technology, which provides limited information and cannot reveal abnormal compensation patterns, and achieves a comprehensive and accurate quantitative analysis of the standardization of complex rehabilitation movements.

[0014] 2. By deploying a dedicated deep learning model using embedded edge computing units for real-time processing, an end-to-end low-latency closed loop is achieved, from multi-source data acquisition and fusion analysis to feedback generation. This allows patients to receive precise audiovisual error correction guidance at the moment of action execution, thus solving the problems of delayed feedback and missed optimal correction opportunities in traditional cloud processing, and significantly improving training efficiency and the reshaping effect of neuroplasticity.

[0015] 3. By dynamically linking the abstract features output by the deep learning model with specific clinical indicators such as joint range of motion, movement trajectory and muscle activation sequence, this invention achieves highly objective and data-driven real-time performance evaluation in rehabilitation training scenarios, providing therapists with decision support that goes beyond subjective experience, and making the tracking and management of rehabilitation progress measurable and comparable.

[0016] 4. By providing automated real-time standard comparison and error correction feedback, this device can significantly reduce the physical and mental burden on therapists who need to continuously conduct manual observation and verbal guidance during training. This allows them to supervise multiple patients simultaneously or focus on more complex treatment decisions, thereby effectively optimizing the allocation of clinical human resources and improving the overall efficiency of rehabilitation services. Attached Figure Description

[0017] Figure 1 The diagram shown is a schematic representation of the system framework of the present invention. Figure 2 The diagram shown is a schematic of the hybrid neural network model structure based on the fusion of spatiotemporal graph convolutional network and long short-term memory network of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figure 1 This invention provides an embodiment: a real-time monitoring device for the standardization of stroke rehabilitation training movements. In this embodiment, the data acquisition module will be described in detail: This example illustrates shoulder abduction in upper limb rehabilitation training: The multi-parameter sensing node arrangement of the present invention includes: (1) An inertial measurement unit was worn on the surface of the middle deltoid muscle of the upper arm to measure the upper arm posture and surface electromyography sensors were used to collect electromyography of the middle deltoid muscle.

[0020] (2) Wear an inertial measurement unit on the forearm near the wrist to measure the forearm and indirectly calculate the elbow joint angle.

[0021] (3) Wear an inertial measurement unit near the scapula to monitor whether there is abnormal compensatory activity in the scapular girdle.

[0022] The visual capture module has a single depth camera facing the patient's side to ensure complete capture of the side profile from shoulder to hand, which is used to obtain the absolute spatial position of the joints and to complement and verify the data from the inertial measurement unit.

[0023] The above configuration can accurately capture the active joint angle, possible compensation, and activation quality of the agonist muscles in shoulder abduction movements, thereby forming a three-dimensional assessment of upper limb functional movements.

[0024] This example illustrates the sit-to-stand transfer technique in lower limb and trunk rehabilitation training: The multi-parameter sensing node arrangement of the present invention includes: (1) An inertial measurement unit and a surface electromyography sensor were worn on the surface of the quadriceps femoris muscle on the front of both thighs to monitor the main muscle groups exerting force in the standing phase.

[0025] (2) Inertial measurement units were worn on both lower legs to monitor the angles of the knee and ankle joints.

[0026] (3) An inertial measurement unit was worn near the L5 vertebra in the lumbar region to monitor the forward tilt angle and stability of the trunk.

[0027] The visual capture module's single depth camera is positioned at a 45° angle above and in front of the patient, allowing observation of both leg movements, as well as trunk leaning and weight shifting.

[0028] This configuration allows for a comprehensive assessment of the biomechanical symmetry, joint coordination, and core stability of sit-to-stand transitions, which is crucial for preventing falls and correcting abnormal patterns.

[0029] In this embodiment, the edge computing unit is described in detail: (1) The data of the inertial measurement unit is preprocessed, the raw data of the accelerometer and gyroscope are denoised and calibrated, the IMU data from the same node or adjacent nodes are fused by the sensor fusion algorithm, the stable posture of the limb segment in the global coordinate system is calculated, and finally the coordinate system is aligned and the sensor local coordinate system is moved to the global coordinate system defined by human anatomy.

[0030] The surface electromyography (EMG) sensor data is preprocessed. The raw EMG signal is first subjected to a hardware or software bandpass filter (20-500Hz) to remove motion artifacts and power frequency interference. Then, full-wave rectification and linear envelope extraction are performed to obtain an envelope signal that reflects the level of muscle activation. Finally, to eliminate individual differences and the influence of electrode position, amplitude normalization is performed, which is divided by the peak value of the surface EMG sensor data of the patient in the maximal voluntary contraction (MVC) test.

[0031] For the preprocessing of visual data, the pose image data output by the depth camera is used to extract the three-dimensional coordinates (x, y, z) of more than 15 key joints of the human body (head, neck, shoulder, elbow, wrist, hip, knee, ankle, etc.) in real time through human pose estimation algorithms built into or running on the edge computing unit.

[0032] Finally, a combination of hardware and software synchronization was adopted. All wireless sensor nodes were equipped with high-precision clock crystals, and clock synchronization was performed through a unified wireless protocol. Visual data was aligned with the sensor timestamps based on its frame timestamps. The data synchronization module ensured that at any given moment, the data from the inertial measurement unit used for analysis, the envelope data from the surface electromyography sensor, and the visual keypoint data corresponded to the same physical time, with an error controlled within 10 milliseconds.

[0033] (2) Please refer to Figure 2 This unit includes a deep learning model, which is a hybrid neural network model based on the fusion of spatiotemporal graph convolutional networks and long short-term memory networks. Specifically: Its input stream includes: Pose Stream takes as input a sequence of visual joint coordinates from T consecutive frames, with dimensions [T, J, 3], where J is the number of joints. This stream primarily captures the macroscopic spatial pose and temporal evolution of the action.

[0034] The kinematic flow takes as input T frames of nine-axis data (three-axis acceleration, three-axis angular velocity, and three-axis magnetometer) from N inertial measurement units, with dimensions [T, N, 9]. This flow provides high-frequency, high-precision details of local limb motion.

[0035] The myoelectric currents are input as T-frame muscle envelope data from M surface electromyography (EMG) sensors, with dimensions [T, M, 1]. This flow reflects the intrinsic drive of neuromuscular control.

[0036] For attitude flow and kinematic flow, due to the obvious graph structure characteristics of the data (joint / sensor nodes connected through the human body), this model uses a spatiotemporal graph convolutional network as its backbone. The spatiotemporal graph convolutional network can perform convolutions simultaneously in the spatial dimension (inter-joint connections) and the temporal dimension (inter-frame connections), efficiently extracting the local and global spatiotemporal patterns of motion. For electromyography (EMG), since the relationships between its channels focus more on functional coordination than physical connections, a one-dimensional convolutional neural network combined with a multi-head self-attention mechanism is used to extract the activation patterns of each muscle and their synergistic relationships.

[0037] The features from the three streams are extracted into high-dimensional feature vectors in the intermediate layer. Then, a cross-modal attention fusion strategy is adopted to calculate the importance weight of each modal feature for the final action representation. For example, when performing action evaluation that "emphasizes fine muscle control", the attention weight of the myoelectric current is dynamically increased; while when "evaluating overall posture stability", the weight of the posture stream may be higher. This adaptive fusion method is more effective than simple concatenation or weighted averaging.

[0038] The fused feature sequence is input into a bidirectional long short-term memory network to capture long-term temporal dependencies. Finally, the features are mapped to the same common feature space as the standard template through a fully connected layer, and a fixed-dimensional action embedding vector is output.

[0039] The model was trained on a large multimodal rehabilitation movement dataset. The dataset contains synchronous inertial measurement unit data, surface electromyography sensor data, and motion capture data of healthy subjects and stroke patients at different stages of recovery performing standard movements. The training objective is to use contrastive learning or a triplet loss function to ensure that samples of the same standard movement are close together in the embedding space, while samples of different movements or incorrect execution methods are far apart.

[0040] (3) This unit includes a standard action template database. The templates in this database are not a single "gold standard," but a hierarchical reference system, specifically: 1) Expert-level templates are ideal data generated by senior rehabilitation therapists.

[0041] 2) Personalized baseline templates are the best movement data that patients try their best to complete in the early stages of rehabilitation with the assistance of therapists, serving as their initial personalized benchmark.

[0042] 3) Progressive target templates are action templates with higher standards set according to the patient's recovery progress.

[0043] Each template contains time-aligned synchronous inertial measurement unit attitude data, surface electromyography sensor envelope data, visual keypoint data, and standard values ​​or ranges for each indicator annotated by experts.

[0044] In this embodiment, the analysis and evaluation module will be described in detail: This module combines the features output by deep learning models with clinically interpretable metrics for evaluation. Specifically: (1) For the calculation of joint range of motion, this module mainly uses the joint angles calculated by the inertial measurement unit. For example, the elbow flexion and extension angles can be calculated by the posture quaternions of the upper arm and forearm inertial measurement units. Its output joint range of motion score = (actual angle range / standard angle range) · 100%. At the same time, the system will determine whether the movement reaches the minimum angle threshold required for functional activity.

[0045] (2) For the analysis of motion trajectory, this module takes the trajectory of the limb end as an example. Its three-dimensional coordinates are obtained by fusing the wrist inertial measurement unit and the visual wrist joint point through Kalman filtering. The dynamic time warping distance between the current trajectory and the standard template trajectory is calculated. This distance is not sensitive to the difference in movement speed and focuses on the similarity of trajectory shape. Its output trajectory similarity = 1 / (1 + dynamic time warping distance), normalized to a score of 0-1, and can also intuitively show which stage of the movement the maximum deviation point occurs in.

[0046] (3) For the analysis of muscle activation timing, this module uses the double standard deviation threshold method to detect muscle activation intervals (the starting point is when the signal continuously exceeds the resting mean + 2 standard deviations; the ending point is when it falls back below the threshold) for the standardized surface electromyography sensor envelope, and calculates the delay time of each muscle activation starting point relative to the action trigger signal. The output timing deviation and activation sequence correctness are also considered. The timing deviation is the difference (ms) between the actual activation time of each muscle and the standard time. The activation sequence correctness is determined by whether the agonist muscles are activated before the antagonist muscles, and whether the core stabilizing muscles are activated before the limb motor muscles.

[0047] (4) For the calculation of attitude stability, this module uses the angular velocity variance of the inertial measurement unit at the waist, or the center of gravity swing area calculated visually. Its output stability index = K / (angular velocity variance + ε), where K is the normalization constant and ε is a small amount to prevent division by zero. The higher the index, the better the stability.

[0048] In this embodiment, the audiovisual feedback module will be described in detail: The module's display terminal shows a real-time animation comparing the patient's current movement with a standard movement template, numerical deviation curves of joint range of motion and limb movement trajectory, muscle activation timing diagrams, postural stability heatmaps, and a real-time updated compliance score. The main view area displays the patient's virtual skeleton and the standard movement skeleton side-by-side in real-time, with erroneous areas flashing red pulses.

[0049] The auditory feedback logic of the audio output device in this module is a hierarchical voice prompt rule base. The first level is a preventive prompt, such as "Ready, keep your torso stable." The second level is a corrective prompt, such as "Straighten your elbows a little more." The third level is a warning prompt, such as "Stop! Your knees are buckling inward!"

[0050] This example provides a comparative example: This study invited 20 stroke patients with hemiplegia in the subacute phase (3-6 months after onset) to be randomly divided into an experimental group and a control group. Both groups underwent upper limb object-reaching training for 4 weeks, 3 times a week, for 30 minutes each time. The control group received traditional on-site guidance and verbal feedback from a therapist, while the experimental group used the present invention. Before and after the training, assessors conducted blind assessments using an optical motion capture system to objectively measure the shoulder flexion angle error, the root mean square error of the hand distal trajectory, and the correlation between the activation sequence of related muscles. The Fugl-Meyer upper limb motor function score was also used for evaluation. The specific results are shown in the table below.

[0051] Table 1 Comparison of clinical function and motor standardization indices before and after training.

[0052] As shown in Table 1, after using this invention, the experimental group showed a significantly greater improvement in the upper limb Fugl-Meyer motor function score (13.1 points) than the control group (6.6 points), demonstrating its advantages in promoting motor function recovery. Furthermore, the experimental group showed significantly lower errors in shoulder joint angle control (7.2°-12.1°) and hand movement trajectory (45.6mm-70.3mm) compared to the control group, indicating that this invention has a significant effect on improving the accuracy of movement execution. Simultaneously, the correlation of abnormal synergistic activation of related muscles in the experimental group decreased to 0.31, significantly lower than the 0.58 in the control group, indicating that the patients' motor control patterns are becoming more normalized. During the experiment, the direct guidance time of therapists in the experimental group was reduced by approximately 70%, demonstrating the significant role of this invention in improving rehabilitation training efficiency and saving professional human resources.

[0053] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A real-time monitoring device for the standardization of stroke rehabilitation training movements, characterized in that, Including: The data acquisition module is used to collect multimodal physiological and motor data of patients in real time when performing rehabilitation training movements. This data acquisition module includes: At least one inertial measurement unit is used to collect kinematic data of the limbs; At least one surface electromyography sensor is used to acquire electromyographic signals of the target muscle group; A visual capture module is used to acquire image data of the patient's body posture; The edge computing unit, electrically connected to the data acquisition module, is used to receive and process multimodal physiological and motion data locally in real time. The edge computing unit is pre-loaded with a deep learning model and a standard action template database. The deep learning model is used to perform multimodal feature fusion and spatiotemporal alignment analysis on the input kinematic data, electromyographic signals and posture image data. Then, the edge computing unit compares the fused and analyzed real-time data with the standard action template retrieved from the standard action template database that corresponds to the current training action. The analysis and evaluation module, integrated into the edge computing unit, calculates and outputs evaluation values ​​of key indicators in real time based on the comparison results. Key indicators include joint range of motion, limb movement trajectory, muscle activation sequence of target muscle groups, and body posture stability. The audiovisual feedback module communicates with the edge computing unit to receive the evaluation values ​​output by the analysis and evaluation module, and generates real-time error correction guidance information and action compliance scores according to preset threshold rules, which are then presented to the patient and therapist through at least one audiovisual format.

2. The real-time monitoring device for the standardization of stroke rehabilitation training movements according to claim 1, characterized in that: The inertial measurement unit and surface electromyography sensor are wearable multi-parameter sensing nodes on the target segment of the patient's limb. Each multi-parameter sensing node simultaneously outputs acceleration, angular velocity, magnetic field strength data and analog / digital electromyography signals.

3. The real-time monitoring device for the standardization of stroke rehabilitation training movements according to claim 2, characterized in that: The wearing position of the multi-parameter sensing node is pre-set according to the main joints and active muscle groups involved in the standard rehabilitation movements, including but not limited to the shoulder, elbow, wrist, hip, knee and ankle.

4. The real-time monitoring device for the standardization of stroke rehabilitation training movements according to claim 1, characterized in that: The visual capture module includes one or more depth cameras for constructing a three-dimensional skeleton model of the patient's body, wherein the posture image data is time-series data containing three-dimensional coordinate information of joint points.

5. The real-time monitoring device for the standardization of stroke rehabilitation training movements according to claim 1, characterized in that: The deep learning model is a hybrid neural network model based on the fusion of spatiotemporal graph convolutional network and long short-term memory network. Its input layer receives nine-axis data sequences from the inertial measurement unit, signal envelope sequences from the surface electromyography sensor, and joint coordinate sequences from the visual capture module. The data is then fused through parallel processing by the feature extraction layer and a multi-head attention mechanism. Finally, the output layer generates an embedding vector that is in the same feature space as the standard action template.

6. The real-time monitoring device for the standardization of stroke rehabilitation training movements according to claim 5, characterized in that: The edge computing unit also includes a data synchronization module, which is used to assign a unified timestamp to data from different sensors, thereby ensuring the temporal consistency of kinematic data, electromyographic signals and posture image data during multimodal fusion.

7. The real-time monitoring device for the standardization of stroke rehabilitation training movements according to claim 1, characterized in that: The audiovisual feedback module includes a display terminal and an audio output device. The display terminal is used to display in real time a comparison animation between the patient's current action and the standard action template, numerical deviation curves of joint range of motion and limb movement trajectory, muscle activation timing diagram, postural stability heat map, and real-time updated compliance score. The audio output device is used to output personalized voice correction prompts when key indicators are detected to deviate from preset ranges.

8. The real-time monitoring device for the standardization of stroke rehabilitation training movements according to claim 1, characterized in that: The standard movement template database stores multimodal data templates generated from standard movements performed by professional therapists. Each template is associated with a specific type and difficulty level of rehabilitation training movement and includes standard values ​​or allowable ranges for joint range of motion, limb movement trajectory, muscle activation sequence, and postural stability.

9. The real-time monitoring device for the standardization of stroke rehabilitation training movements according to claim 1, characterized in that: Before comparing real-time data with standard action templates, the edge computing unit first performs coordinate system alignment and gravity compensation preprocessing on the kinematic data, and performs filtering, rectification and standardization preprocessing on the electromyographic signals.

10. A real-time monitoring device for the standardization of stroke rehabilitation training movements according to any one of claims 1-9, characterized in that: The device outputs assessment values, error correction guidance information, and compliance scores, which are synchronously transmitted to a remote server via a wireless communication module to generate long-term rehabilitation training reports and efficacy assessment charts.

Images