A stroke hand rehabilitation training method based on multi-modal data fusion

Abstract

The application discloses a stroke hand rehabilitation training method based on multi-modal data fusion, and particularly relates to the field of hand rehabilitation training, and is used for solving the problem that the motion solidification risk cannot be recognized and the training scheme lacks individual dynamic adjustment in the existing rehabilitation process. The application obtains hand joint motion characteristics by constructing a standardized hand function micro-evaluation task, and establishes a personal motion mode baseline in combination with dynamic time warping and multi-scale permutation entropy analysis. A neural motion solidification index is extracted based on the difference change of the current and optimal motion mode, and a function recovery momentum feature is generated in combination with rehabilitation progress gradient and acceleration inflection point analysis. A rehabilitation risk score is generated by fusing an electromyographic signal coupling coordination index, and when the risk exceeds a dynamic threshold, a training strategy adaptive reorganization is triggered, precise early warning and individualized training regulation are realized. The application can effectively improve the rehabilitation monitoring precision and training pertinence, and promote high-quality recovery of the hand function of stroke.

Description

Technical Field

[0001] This invention relates to the field of hand rehabilitation training technology, and more specifically, to a method for hand rehabilitation training in stroke based on multimodal data fusion. Background Technology

[0002] Stroke is a leading cause of adult disability, with hand dysfunction being the most common and requiring a long recovery period. Patients often experience decreased finger dexterity, uncoordinated grasping movements, and rigid motor patterns, significantly limiting their daily living abilities. Current rehabilitation training largely relies on therapists' experience to develop programs, and assessments primarily depend on subjective scales or static indicators, making it difficult to track subtle changes in hand motor function in real time and quantitatively. Furthermore, stroke recovery exhibits significant stages and individual differences; the levels of neuroplasticity and motor learning abilities vary considerably among patients, making single-dimensional assessments insufficient to accurately reflect the true process of neurological recovery. With repeated rehabilitation training, patients are prone to developing maladaptive compensatory movements and motor rigidity—that is, although they complete the movement, the manner of movement is incorrect, leading to a long-term inability to regain normal hand control patterns. However, current rehabilitation systems generally lack mechanisms for identifying and warning of the risk of motor rigidity.

[0003] Furthermore, while existing intelligent rehabilitation equipment is gradually incorporating technologies such as motion capture and electromyography (EMG) monitoring, it still suffers from limitations such as relying on single evaluation indicators and failing to integrate the correlation between kinematics and neural activation signals. This results in an inability to respond promptly to rehabilitation risks and adjust training strategies. This is especially true in home-based or semi-supervised rehabilitation settings, where there is a lack of systems capable of real-time reconfiguration of training task difficulty and feedback methods based on changes in the patient's own movement patterns.

[0004] Therefore, there is an urgent need for a rehabilitation training method that can comprehensively interpret changes in neuroplasticity, functional recovery trends, and the risk of consolidation from multimodal dynamic data, so as to achieve precise tracking and personalized adaptive intervention of the hand rehabilitation process after stroke, thereby improving rehabilitation efficiency and reducing the continuous reinforcement of incorrect movement patterns. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a stroke hand rehabilitation training method based on multimodal data fusion to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for hand rehabilitation training in stroke based on multimodal data fusion includes the following steps:

[0008] S1. Construct a standardized micro-assessment task for hand function, collect motor characteristics of the hemiplegic hand in stroke patients, and generate a digital assessment vector for hand function.

[0009] S2. Establish a baseline database of hand movement patterns for stroke patients, perform dynamic time normalization on continuously collected hand function assessment vectors, extract the entropy value of motor control, and calculate the range of motor coordination fluctuations during the rehabilitation period.

[0010] S3. Compare the similarity distance decay curve between the current hand movement pattern and the individual's optimal movement pattern, and calculate the neuromotor solidification index in combination with the motor learning rate.

[0011] S4. Analyze the momentum characteristics of functional recovery, track the rehabilitation progress gradient of each dimension index in the digital assessment vector of hand function, and detect the inflection point of the acceleration change in the process of neurological function recovery.

[0012] S5. Multimodal fusion of neuromotor consolidation index and functional recovery momentum characteristics, and introduction of surface electromyography and kinematic coupling coordination degree as a verification index to generate a hand rehabilitation risk fusion score;

[0013] S6. Implement rehabilitation interventions. When the hand rehabilitation risk score exceeds the dynamic threshold, reconfigure the task difficulty parameters.

[0014] In a preferred embodiment, step S1, which involves constructing a standardized hand function micro-assessment task, collecting hand movement characteristics of the hemiplegic side of a stroke patient, and generating a digital assessment vector for hand function, specifically includes:

[0015] The design incorporates standard hand gesture sequences with different grip patterns and finger separation degrees, and uses computer vision to track hand joints and collect motion parameters of the hemiplegic hand.

[0016] During the execution of standard hand gestures, the angle data of each joint and the speed of movement are recorded. At the same time, multi-dimensional motion features, including the smoothness of the motion trajectory and the symmetry of the range of motion, are extracted. The above data and time stamp information are integrated into a digital evaluation vector of hand function.

[0017] In a preferred embodiment, step S2 involves establishing a baseline database of hand movement patterns for stroke patients, dynamically time-normalizing continuously acquired hand function assessment vectors, extracting the patient's motor control entropy value, and calculating the range of motor coordination fluctuations during the rehabilitation period. Specifically, this includes:

[0018] Continuously collect digital assessment vectors of hand function in stroke patients at different rehabilitation cycles;

[0019] A dynamic time warping algorithm is used to align digital assessment vectors from different rehabilitation cycles, and the minimum cumulative distance between sequences is calculated as a time warping metric.

[0020] Multi-scale permutation entropy analysis was performed on the regularized joint angle change trajectory to calculate the motor control entropy value, which reflects the complexity of neuromuscular control.

[0021] Analyze the phase synchronization relationship between the trajectories of joint angle changes, and calculate the range of motion coordination fluctuations during the rehabilitation period;

[0022] Establish a baseline database of hand movement patterns in stroke patients that includes motor control entropy values ​​and coordination fluctuation ranges.

[0023] In a preferred embodiment, calculating the motor control entropy value, which reflects the complexity of neuromuscular control, specifically includes:

[0024] Using the dynamic time-normalized trajectory of hand joint angle changes as input data, the trajectory is broken down into multiple motion trend segments at different scales;

[0025] Based on the size order of data points in the motion trend segment, an arrangement pattern is constructed, the frequency of each arrangement pattern in the sequence is counted, and the entropy value at the corresponding scale is calculated based on the dispersion of the frequency distribution.

[0026] In a preferred embodiment, step S3, comparing the similarity distance decay curve between the current hand movement pattern and the individual's optimal movement pattern, and calculating the neuromotor fixation index in conjunction with the motor learning rate, specifically includes:

[0027] The optimal hand function digital assessment vector with the best motor control entropy value was selected from the baseline database of hand movement patterns of stroke patients as the individual optimal movement pattern;

[0028] Based on the similarity comparison of joint angle change trajectory and motion speed distribution characteristics, the dynamic time bending algorithm is used to calculate the similarity distance between the current hand movement pattern and the individual's optimal movement pattern.

[0029] A similarity distance decay curve was constructed based on the similarity distance of continuous rehabilitation cycles, and the trend of curve slope change was analyzed.

[0030] The motor learning rate is calculated by combining the improvement magnitude of the motor control entropy value of the current hand movement pattern. The slope of the similarity distance decay curve is weighted and fused with the motor learning rate to generate the neuromotor solidification index.

[0031] In a preferred embodiment, step S4, analyzing the momentum characteristics of functional recovery, tracking the rehabilitation progress gradient of each dimension of the hand function digital assessment vector, and detecting the inflection point of the acceleration change in the neurological function recovery process, specifically includes:

[0032] The rehabilitation progress gradient of each dimension index is calculated based on the digital assessment vector of hand function in continuous rehabilitation cycles. The rehabilitation progress gradient is characterized by the degree of improvement of the joint angle change trajectory and the optimization magnitude of the movement speed distribution characteristics in adjacent rehabilitation cycles.

[0033] The gradient sequence of rehabilitation progress was smoothed to extract the characteristics of functional recovery acceleration changes.

[0034] By analyzing the distribution of extreme points in the acceleration change characteristics of functional recovery, the inflection point of acceleration change in the process of neural function recovery can be detected.

[0035] Functional recovery momentum feature vectors are generated based on the gradient of rehabilitation progress and the inflection point of acceleration change.

[0036] In a preferred embodiment, step S5 involves multimodal fusion of the neuromotor consolidation index and functional recovery momentum characteristics, introducing surface electromyography and kinematic coupling coordination as a verification indicator, and generating a hand rehabilitation risk fusion score, specifically including:

[0037] A multimodal fusion matrix of neuromotor consolidation index and functional recovery momentum features is established. This matrix includes the slope of the similarity distance decay curve and the feature dimension of the inflection point of functional recovery acceleration change.

[0038] The mutual information between surface electromyography (EMG) signals and joint angle change trajectories is calculated to obtain the degree of coordination between surface EMG and kinematics.

[0039] The coupling coordination degree is used as a verification metric to weight the credibility of the output results of the multimodal fusion matrix.

[0040] Based on the weighted multimodal fusion matrix output and the coupling coordination degree verification results, a hand rehabilitation risk fusion score is generated.

[0041] In a preferred embodiment, in step S6, performing rehabilitation intervention, when the hand rehabilitation risk score exceeds a dynamic threshold, specifically reconfiguring the task difficulty parameters includes:

[0042] Based on the historical hand rehabilitation risk fusion score data in the baseline of hand movement patterns in stroke, its statistical distribution characteristics were calculated and a dynamic threshold judgment standard was established.

[0043] When the real-time hand rehabilitation risk fusion score exceeds the dynamic threshold, the task difficulty parameters are reconfigured, and the accuracy requirements of the joint angle change trajectory and the target range of the movement speed are linearly adjusted according to the proportion exceeding the dynamic threshold.

[0044] The technical effects and advantages of the stroke hand rehabilitation training method based on multimodal data fusion proposed in this invention are as follows:

[0045] By integrating hand kinematics data and surface electromyography signals, this invention achieves multi-dimensional and precise quantification of motor control ability in the hemiplegic hand of stroke patients. It no longer relies on traditional subjective scoring and single-indicator evaluation, but can reflect the comprehensive recovery degree of joint coordination, movement smoothness, and neural activation in real time. By introducing dynamic time warping, multi-scale permutation entropy analysis, and movement pattern similarity comparison, this invention can accurately extract the changing trends of neuromotor solidification risk and motor learning ability, promptly identifying compensatory movements or error patterns that occur during training, and avoiding rehabilitation stagnation caused by functional solidification. Simultaneously, the functional recovery momentum features constructed in this invention can capture the inflection point of acceleration changes during the recovery process, making rehabilitation stage goals clearer and training program adjustments more targeted. Through a multimodal fusion scoring mechanism, this invention not only improves the reliability of assessment results but also automatically triggers warnings and reorganizes personalized training tasks when the risk exceeds a dynamic threshold, achieving adaptive adjustment of rehabilitation difficulty and biofeedback type, effectively improving training effects and patient participation.

[0046] It is applicable to various scenarios such as hospital and home rehabilitation, and can significantly improve the accuracy of rehabilitation monitoring and decision-making efficiency, shorten the rehabilitation cycle, and improve the continuity and quality of hand movement recovery in stroke patients. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of a stroke hand rehabilitation training method based on multimodal data fusion according to the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0049] Example 1

[0050] Figure 1 This invention presents a method for hand rehabilitation training in stroke based on multimodal data fusion, which includes the following steps:

[0051] S1. Construct a standardized micro-assessment task for hand function, collect motor characteristics of the hemiplegic hand in stroke patients, and generate a digital assessment vector for hand function.

[0052] S2. Establish a baseline database of hand movement patterns for stroke patients, perform dynamic time normalization on continuously collected hand function assessment vectors, extract the entropy value of motor control, and calculate the range of motor coordination fluctuations during the rehabilitation period.

[0053] S3. Compare the similarity distance decay curve between the current hand movement pattern and the individual's optimal movement pattern, and calculate the neuromotor solidification index in combination with the motor learning rate.

[0054] S4. Analyze the momentum characteristics of functional recovery, track the rehabilitation progress gradient of each dimension index in the digital assessment vector of hand function, and detect the inflection point of the acceleration change in the process of neurological function recovery.

[0055] S5. Multimodal fusion of neuromotor consolidation index and functional recovery momentum characteristics, and introduction of surface electromyography and kinematic coupling coordination degree as a verification index to generate a hand rehabilitation risk fusion score;

[0056] S6. Implement rehabilitation interventions. When the hand rehabilitation risk score exceeds the dynamic threshold, reconfigure the task difficulty parameters.

[0057] In S1, a standardized hand function micro-assessment task is constructed, and the hand movement characteristics of the hemiplegic side of stroke patients are collected to generate a digital assessment vector of hand function.

[0058] To address the objective quantitative need for assessing the motor characteristics of the hemiplegic hand in stroke patients, a standardized hand gesture sequence was established as the foundational action set for a micro-assessment task of hand function. This sequence explicitly includes movement types such as fist clenching, pinching, opening the hand, finger separation, fine grasping within a small range, and rapid finger flexion and extension. The action design encompasses different grasping patterns, finger coordination levels, and differences in finger separation to cover the main fine motor control performance of the hand. During the action acquisition phase, a computer vision-based hand joint tracking method was used for dynamic monitoring. The acquisition device was fixed at a suitable distance in front of the patient, ensuring complete coverage of the hemiplegic hand region. The locations of major hand joints, including the metacarpophalangeal joints, middle phalangeal joints, distal phalangeal joints, and wrist joints, were identified using pre-set depth imaging or a two-dimensional skeletal keypoint detection model. To avoid external noise and background interference, an action execution area restriction strategy was employed to ensure that no other moving objects were present in the acquired image. During the data acquisition process, time-series data of each joint position are continuously recorded at a constant frame rate. A synchronization mechanism between action command triggering and data acquisition start and end is established to ensure data recording begins and ends within the correct action execution segment, avoiding the inclusion of invalid segments. Before performing each standard gesture, patients are provided with clear and intuitive demonstration prompts, using voice commands and visual illustrations to ensure consistency. The number of repetitions for each action is clearly defined during the acquisition process, for example, five repetitions per action. Multiple sets of data are collected through repeated actions to enhance the stability and representativeness of the assessment data.

[0059] After completing the tracking of the joints in the hemiplegic hand, this embodiment further analyzes and processes the collected motion data to extract key features that characterize the quality of hand movement from multiple dimensions and construct a unified digital evaluation vector. During the execution of the movement, the changes in the angles of each joint are calculated from the joint position data, and the flexion and extension angles of the five fingers, the interphalangeal expansion angle, and the wrist joint deflection angle are continuously recorded. To ensure that the angle information has true kinematic significance for the hand, the joint coordinates are topologically calibrated according to the human hand structure, so that the angle sequence reflects the true flexion and extension state of the corresponding physiological structure. In addition, for velocity characteristics, the angle change rate of adjacent frames is calculated at each frame interval to form a complete motion velocity distribution sequence, which is used to characterize the timeliness and smoothness of hand movement execution. The smoothness of the motion trajectory and the symmetry of the motion range are introduced as supplementary evaluation dimensions. Smoothness is determined by analyzing the trend of angle time series changes in consecutive frames to identify unnecessary tremors or pauses in movement execution; range of motion symmetry is determined by comparing the differences in angle distribution and range of change of corresponding joints on the hemiplegic side and the healthy side. If there is no control side, a mapping comparison is performed based on the health baseline trajectory preset in the rehabilitation database to reflect the degree of recovery and coordination of motor ability.

[0060] All motion features, including joint angle values, velocity sequences, trajectory change trends, and left-right difference indices, are assembled in a fixed order into a structurally standardized digital evaluation vector for hand function. The internal feature arrangement of the vector maintains consistency and supports repeated acquisition results for longitudinal comparative analysis. All feature data are dimensionless standardized.

[0061] In S2, a baseline database of hand movement patterns for stroke patients is established, the continuously collected hand function assessment vectors are dynamically time-normalized, the entropy value of the patient's motor control is extracted, and the range of motor coordination fluctuations during the rehabilitation period is calculated.

[0062] Digital assessment vectors of hand function were collected from patients who repeatedly performed standardized hand movements across multiple rehabilitation cycles. Each acquisition maintained the same movement execution requirements and rhythm, and a uniform time stamp was used for opening and closing control during data recording to ensure the validity of the movement segments. After acquisition, multiple digital assessment vectors with varying durations were obtained. Because patients' motor abilities change continuously during rehabilitation, the speed, amplitude, and details of each movement differ. Directly comparing different cycle sequences would be affected by differences in execution rhythm and would not accurately reflect changes in movement pattern similarity. Therefore, a dynamic time warping method was used to align the time dimensions of different cycle trajectories. A segment of digital assessment vectors considered as the current cycle was selected as the matching sequence according to chronological order, and the assessment vector of a certain cycle in the early stages of rehabilitation was set as the reference trajectory. Then, the two trajectories were compared point-by-point based on the temporal proximity and similarity of movement trends. By constructing a cumulative matching path, points on subsequent cycle trajectories corresponded to the points on the reference trajectory with the most consistent movement trends. This path reflects the correspondence between the sequence and the motion execution phase. Its traversal structure ensures that all action phases can be effectively matched, and no action phase will be skipped or compressed due to differences in execution speed.

[0063] After matching is completed, two data segments of consistent length and phase correspondence are obtained, resulting in a normalized sequence. Rules are set to ensure the continuity of the normalized path and limit the step length of single-step matching to prevent non-physiological jumps or unreasonable stretching of the path. Simultaneously, the minimum cumulative matching cost during the normalization process is recorded and used as an evaluation index of temporal normalization for subsequent assessment of changes in movement execution difficulty. By performing the above normalization process on the evaluation vectors of multiple rehabilitation cycles of the patient, a standardized set of hand movement trajectories with unified temporal dimension and consistent trajectory phase is finally obtained.

[0064] After obtaining the hand joint angle change trajectory after dynamic time warping, a multi-scale permutation entropy analysis method was used to extract the motor control entropy value to reflect the dynamic complexity of neuromuscular control. First, the warped trajectory was reconstructed into a sequence according to different scale factors. Continuous angle change data were sampled at fixed intervals to form multiple sub-sequences reflecting the characteristics of motor changes at different time resolutions. The sub-sequences at each scale exhibited different fine-grained motor trends, comprehensively characterizing the details of the patient's hand movement control. Each scale's sub-sequence was divided into local segments, each containing angle change data from several adjacent points. By sorting the size order of the points in the segments, each segment was converted into a permutation pattern with consistent length and structure. These permutation patterns reflected the structural characteristics of the motor trend, including different signal change types such as acceleration, deceleration, tremor, and stagnation. After the permutation process was completed, the frequency of different permutation patterns in the sequence was statistically analyzed, and the randomness and complexity of that scale were evaluated based on the dispersion of the frequency distribution. When all permutation patterns had a relatively balanced frequency of occurrence, it indicated rich motor control changes and good dynamic regulation ability; conversely, it indicated that the movement tended to be fixed and lacked physiological regulatory changes. After the complexity measurement results obtained at all scales are organized into a unified format, they are normalized and converted into motion control entropy values ​​within a unified range.

[0065] The phase synchronization relationship between joint angle change trajectories was analyzed, and the motor coordination status was quantified by comparing the relative rhythmicity of multiple joints in the hand over time. The specific process included: first, selecting representative joint combinations, such as thumb and index finger, index finger and middle finger, etc.; extracting the phase information of each joint over time from the regularized trajectory; and using a sliding time window to statistically analyze the synchronization degree between each pair of joints. If the change trends of different joints are consistent within the same time segment, the synchronization degree is improved; if the movement rhythm is inconsistent, it reflects a decline in coordination. After continuous monitoring of the phase synchronization degree over multiple rehabilitation cycles, a complete curve of its change over time was obtained. By statistically analyzing the numerical span of the fluctuations in this curve, the range of motor coordination fluctuations was calculated. A larger fluctuation range indicates significant room for improvement in coordination or a prominent stage of regression; a smaller range indicates stable motor coordination performance. To ensure that the fluctuation range has medical support, a baseline interval was set based on the phase synchronization degree data from the initial rehabilitation stage. During long-term tracking, whenever the synchronization degree exceeded the boundary of this range, the abnormal segment was recorded and used as the basis for judging the statistical analysis of the coordination fluctuation range.

[0066] Finally, the motor control entropy value and coordination fluctuation range were written into the stroke patient hand movement pattern baseline database using a fixed-structure coding method and indexed according to the rehabilitation cycle. This baseline database contains the motor complexity and coordination status of patients in each historical cycle.

[0067] In step S3, the similarity distance decay curve between the current hand movement pattern and the individual's optimal movement pattern is compared, and the neuromotor solidification index is calculated in combination with the motor learning rate.

[0068] Using multi-cycle data recorded in a baseline database of hand movement patterns in stroke patients, this study sorts the motor control entropy values ​​corresponding to all rehabilitation cycles. The assessment vector with the highest motor control entropy value and a stable entropy trend is identified as the optimal neuromuscular control state achieved by the patient during rehabilitation, thus obtaining the individual's optimal movement pattern. The highest motor control entropy value indicates the richest dynamic regulation and higher complexity of the hand motor control signal within that cycle, making it a valid and feasible candidate criterion. The specific screening process includes: dividing the motor control entropy values ​​in the baseline database, arranged chronologically according to rehabilitation cycle, into different time periods based on rehabilitation stages to avoid misleading analysis from abnormal data in a single cycle. Within each time period, the top three assessment vectors with the highest entropy values ​​are extracted to form a candidate set. These are then further screened based on the stability of their multi-joint synergy indicators and whether the differences in continuous cycle changes remain within a reasonable range. If a candidate data point exhibits a sudden change that does not conform to the continuity of recovery, it is removed. After screening, the assessment vector that satisfies both the highest entropy value and stage-specific stability is retained as the final optimal pattern data. The optimal pattern includes a complete trajectory of joint angle changes, a sequence of movement velocity distributions, and execution cycle time labels. The optimal pattern data obtained based on the above screening process will serve as a reference for comparing hand movement patterns in all subsequent stages of the rehabilitation process and will be labeled in the baseline database.

[0069] Based on the dynamic time-bending algorithm, the digital assessment vector of hand function in any rehabilitation cycle of the patient is matched and compared with the optimal pattern to calculate the similarity distance between the two movement patterns. The dynamic time-bending algorithm aligns the current pattern trajectory with the optimal pattern trajectory point by point, so that they produce a consistent action execution phase in the time dimension, and then accumulates the similarity distance based on the differences in the corresponding trajectories. The smaller the value, the closer the current pattern is to the optimal pattern. The same comparison procedure is performed for each rehabilitation cycle to obtain multiple continuous similarity distance values ​​for the patient throughout the entire rehabilitation process. After obtaining the continuous cycle similarity distance, it is plotted as a similarity distance decay curve according to the rehabilitation time series. As rehabilitation progresses, the patient's hand movement pattern gradually approaches the optimal pattern, so the curve shows a gradual downward trend over time. In order to extract the convergence speed of movement patterns in the rehabilitation process, this embodiment performs segmented analysis on the similarity distance decay curve at different time periods and calculates the trend of curve slope change by sliding time window. Among them, the increase in the absolute value of the slope indicates that the movement pattern is approaching the optimal state faster, while the decrease in slope indicates that the rehabilitation progress is slowing down or tending to plateau. Before plotting the similarity distance data, a time consistency check is performed, and abnormal data segments that deviate significantly from the previous and subsequent trends are labeled.

[0070] The change in motor control entropy and the slope of the similarity distance decay curve are combined to calculate the motor learning rate and generate a neuromotor consolidation index. The improvement in motor control entropy is measured by the change in the difference between the entropy value and the optimal mode entropy value: that is, when the entropy value of a rehabilitation cycle significantly increases compared to the previous cycle and converges towards the optimal mode entropy value, the motor learning rate is considered to have improved in that cycle. To make the improvement identifiable, the motor control entropy value at the initial stage of rehabilitation is set as a zero-point reference. The increase in each cycle is normalized by the ratio of the convergence of the difference with the optimal mode, ensuring that the entropy changes in different rehabilitation stages reflect the continuity of the motor learning process. Subsequently, this improvement and the slope of the similarity distance decay curve are correlated with each cycle and fused using a fixed-ratio weighting method to form a single and stable motor learning rate index. The weighting rule is set based on the importance ratio of the two indicators in the rehabilitation behavior, with importance dynamically set based on a comparison of the patient's degree of loss in motor execution ability and neuromodulation ability. After weighting, a motor learning rate sequence with temporal continuity is obtained.

[0071] In S4, the momentum characteristics of functional recovery are analyzed, the rehabilitation progress gradient of each dimension index in the digital assessment vector of hand function is tracked, and the inflection point of the acceleration change in the process of neurological function recovery is detected.

[0072] Unlike the neuromotor consolidation index, which focuses on assessing the patient's consolidation of the flexibility and stability of their hand movements, the rehabilitation progress gradient quantifies the rate of improvement in hand function of stroke patients over the course of rehabilitation. It uses the hand movement status of healthy individuals as a benchmark and calculates the degree of improvement in movement health during the cycle based on multiple kinematic dimensions in the hand function digital assessment vector. To ensure the rehabilitation progress gradient's accurate representation, this embodiment simultaneously focuses on changes in both the quality of movement trajectory execution and the dynamics of movement, namely the degree of improvement in the trajectory of joint angle changes and the extent of optimization of movement speed distribution characteristics. The digital assessment vectors of hand function from consecutive rehabilitation cycles are compared for inter-cycle differences. By analyzing the reduction in irregular fluctuations in the trajectory of joint angle changes over time and the increase in the range of motion completed, channel-by-channel analysis is used as a quantitative basis for trajectory improvement. The so-called improvement in the trajectory of joint angle change is the reduction in the difference between the trajectory deviation of the same movement stage between adjacent rehabilitation cycles and the optimal trajectory, and the trend of indicators such as the maximum extension value and retraction value of the movement approaching the preset normal hand movement control parameter range. The above-mentioned optimal trajectory and normal hand movement control parameter range are set based on data generated by sampling multiple healthy hand personnel performing standard gesture movement sequences with different grip patterns and finger separation degrees.

[0073] The dynamic quality changes of movements are evaluated from the perspective of movement speed distribution characteristics. The optimization magnitude of movement speed distribution is defined as the ratio by which the peak range of the speed distribution in key movement stages such as finger flexion and extension, and opening and closing, tends to a preset stable range, and the number of abnormally high and low speed segments decreases during a continuous rehabilitation cycle. For example, when the peak of the speed curve is detected to be closer to the statistical range of healthy movement speed in a later cycle, and the duration and number of low-speed dragging segments are shortened, the speed distribution is considered to be optimized. The aforementioned healthy movement speed is also based on data generated from sampling standard hand gesture sequences performed by multiple healthy individuals. To improve data consistency, the statistical range of speed recorded in the early stage of rehabilitation is used as a reference interval for normalization mapping, enabling direct comparison of data from different cycles. Finally, the changes in the above two dimensions are combined in a structured manner to form a rehabilitation progress gradient vector. Each movement and its corresponding joint in the vector maintains an independent gradient record value, ensuring that the rehabilitation progress analysis has fine-grained resolution.

[0074] To identify the phased characteristics of functional changes, a temporal continuity analysis was further conducted, and acceleration information of gradient changes was obtained through smoothing. First, a continuous periodic curve smoothing strategy was employed to soften local abrupt changes in the gradient vector caused by single-execution anomalies, making the gradient change curve reflect the overall change pattern of the patient's recovery trend. The smoothed gradient change trend can more accurately reflect the true increase or slowdown in motor ability with the increase in training repetitions. Second-order change features of gradient changes, namely functional recovery acceleration change features, were extracted from the smoothed data. Acceleration change features are defined as the change in the rate of increase of the rehabilitation progress gradient within a continuous period, reflecting whether the patient's motor learning ability is in an acceleration, deceleration, or plateau phase. By detecting extreme points in the acceleration change curve, key turning points in the rehabilitation process can be identified, such as points where a significant increase turns into a slow increase or decrease, to determine whether the patient is experiencing important dynamic changes such as slowed recovery or decreased training compliance. Subsequently, based on the information of rehabilitation progress gradient and acceleration change extreme points, a functional recovery momentum feature vector was constructed. This vector contains not only gradient values ​​and time markers, but also markers indicating the direction and turning points of acceleration changes, enabling it to reflect both accumulated ability and propulsive speed during motor skill learning. To ensure consistent interpretability of the momentum feature vector across different patients, all numerical results were normalized to a uniform interval and assigned to a fixed order based on movement type and joint region.

[0075] In S5, the neuromotor consolidation index and functional recovery momentum characteristics are fused in a multimodal manner, and the surface electromyography and kinematic coupling coordination degree are introduced as a verification index to generate a hand rehabilitation risk fusion score.

[0076] After obtaining the neuromotor consolidation index and the momentum feature vector for functional recovery, the two evaluation indicators are integrated into a single data structure to construct a multimodal fusion matrix. The main purpose of the fusion matrix is ​​to simultaneously reflect the dual characteristics of the convergence trend of movement patterns and the improvement trend of motor ability during hand rehabilitation, making the subsequent risk identification process universal and scalable. The fusion matrix sets a fixed structural dimension, including the slope information of the similarity distance decay curve and the inflection point information of the acceleration change in functional recovery. Among them, the slope of the similarity distance decay curve comes from the trend of the convergence degree of the patient's continuous periodic hand movement patterns to the optimal movement pattern, which can illustrate the pattern consolidation in the rehabilitation stage. The inflection point of acceleration change comes from the precise marking of the inflection point of the gradient curve, which is used to indicate the turning direction of the momentum of functional improvement. The fusion operation process includes: first, aligning the neuromotor consolidation index and the momentum feature vector with time indexes to ensure that the consolidation trend and momentum feature corresponding to the same rehabilitation cycle are represented in the same column in the matrix. Then, the numerical interval is unified, and indicators of different dimensions are converted to a standardized unified interval to ensure that there is no bias between the features in the matrix due to scale differences.

[0077] Surface electromyography (SEMG) signals are introduced as another data source for neural drive, and their synergy with joint angle change trajectories is coupled and analyzed to verify whether the motor output is truly initiated and regulated by the neural control system. This analysis is achieved through mutual information calculation, the goal of which is to clarify the correlation between the SEMG change trend and the hand movement trajectory change trend in the same movement phase, describing the degree of coupling coordination between the two types of signals. The specific process includes: acquiring SEMG signals synchronized with joint movement, and performing noise suppression and amplitude range standardization on the signals; then aligning the SEMG signals and angle trajectories into segments at the same time, and statistically analyzing the correspondence of signal change trends in each segment, using the amount of information sharing between data as the coupling criterion. If the SEMG signal change is detected as leading or synchronized with the movement trend, the coupling coordination is high; if the correlation between movement and SEMG is weak or there is a significant lag, the coupling coordination decreases. The entire calculation process is executed in segments according to the rehabilitation cycle, thereby obtaining a coupling coordination sequence in the rehabilitation process, which is used to support the credibility judgment of the multimodal fusion results.

[0078] Coupling coordination is used as a credibility weighting factor in the output of the multimodal fusion matrix. The weighting mechanism is based on the following logic: when the coupling coordination between electromyography and motor functions is high, it indicates that the improvement in motor quality and neural activation are simultaneously enhanced. In this case, each indicator in the fusion matrix represents a credible behavior and its weight should be increased. If the coupling coordination decreases, it indicates that there is rigidity in movement execution or the inclusion of compensatory movements. In this case, the indicators representing the rehabilitation trend in the fusion matrix should have their weights reduced. The specific weight adjustment ratio is set based on the actual calculation results of the coupling coordination. All weighted and corrected fusion feature outputs are converted into a single quantitative output according to a preset scoring rule. The scoring rule uses a weighted summation method. This output serves as a hand rehabilitation risk fusion score, which intuitively reflects the risk level within the same rehabilitation cycle and is continuously compared with historical cycle data to form a trend analysis.

[0079] In S6, rehabilitation intervention is performed, and when the hand rehabilitation risk score exceeds the dynamic threshold, the task difficulty parameters are reconfigured.

[0080] A dynamic threshold determination criterion is constructed based on patients' historical rehabilitation score data. The hand rehabilitation risk fusion score accumulates continuously with each rehabilitation cycle and is written into the hand movement pattern baseline database. This score sequence reflects the long-term trend of the patient's neuromotor recovery status. First, historical score data is arranged chronologically, and its statistical distribution characteristics are analyzed in depth, including the concentrated distribution interval of scores, the range of score changes, and the structure of score fluctuations, thereby extracting the median trend representing the stable level of the patient's inherent rehabilitation status. Simultaneously, to ensure the threshold's timely responsiveness, the scores from the most recent cycles are further weighted, allowing the latest status to contribute more significantly to the threshold determination. Subsequently, a dynamic threshold range is formed based on the statistical analysis results. The threshold represents the reference risk level for the current rehabilitation stage, and its upper limit is used to define the sensitive triggering conditions for the rehabilitation status to deviate from the normal recovery trajectory. This dynamic threshold is automatically updated after each new cycle score is written, enabling rehabilitation monitoring to adaptively adjust the risk determination criteria according to the actual changes in the patient's abilities, avoiding misjudgments or delayed responses caused by fixed thresholds.

[0081] Once the real-time rehabilitation risk fusion score is generated, it is compared instantly with the aforementioned dynamic threshold, and the rehabilitation training program is automatically adjusted based on the comparison results. When the real-time score exceeds the upper limit of the dynamic threshold, it is considered that the patient's current rehabilitation trend is at risk of deviation. At this time, the system will trigger a training task difficulty reorganization mechanism, dynamically adjusting the precision requirements and speed control targets of movement execution to make the training content more suitable for the patient's real-time rehabilitation state. The adjustment process includes: determining the difficulty adjustment range according to the proportion of the score exceeding the threshold; if the score is only slightly higher than the threshold, a mild adjustment is performed, such as appropriately reducing the target precision of the joint angle change range, or correcting and limiting excessively high or low areas of the speed curve; if the score significantly exceeds the threshold, a higher level of intervention is performed, such as replanning the task type, replacing fine movements with basic movement training, or limiting the duration of high-risk movements and strengthening the steady-state training phase. All adjustments are performed linearly with the same proportion of change, making the difficulty changes controllable and continuous, avoiding abrupt changes in training intensity that the patient cannot adapt to. Through the above processing flow, dynamic adaptive adjustment of movement control difficulty and neural activation requirements during rehabilitation training is achieved, completing the closed-loop intervention execution under risk exceeding the limit conditions.

[0082] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0083] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0084] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0085] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0086] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0087] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0088] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0089] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0090] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0091] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A stroke hand rehabilitation training method based on multi-modal data fusion, characterized in that, It comprises the following steps: S1, constructing a standardized hand function micro-evaluation task, collecting hemiplegic hand movement characteristics of stroke patients, and generating a hand function digital evaluation vector; S2, dynamic time warping of the continuously collected hand function evaluation vector, extraction of patient movement control entropy value, calculation of movement coordination fluctuation range during rehabilitation period, and establishment of a baseline library of stroke patient hand movement patterns containing movement control entropy value and coordination fluctuation range; S3, screening the hand function digital evaluation vector with the optimal movement control entropy value from the baseline library of stroke patient hand movement patterns as the individual optimal movement pattern, comparing the similarity distance decay curve of the current hand movement pattern and the individual optimal movement pattern, and calculating the neural movement solidification index combined with the movement learning rate; S4, calculating the rehabilitation progress gradient of each dimension index based on the hand function digital evaluation vector of the continuous rehabilitation period, the rehabilitation progress gradient being characterized by the improvement degree of the joint angle change trajectory and the optimization amplitude of the movement speed distribution characteristics between adjacent rehabilitation periods, smoothing the rehabilitation progress gradient sequence, extracting the functional recovery acceleration change characteristics, detecting the acceleration change inflection point of the neural function recovery process by analyzing the extreme value point distribution of the functional recovery acceleration change characteristics, and generating a functional recovery momentum feature vector based on the rehabilitation progress gradient and the acceleration change inflection point; S5, establishing a multi-modal fusion matrix of neural movement solidification index and functional recovery momentum characteristics, the matrix containing the slope of the similarity distance decay curve and the characteristic dimension of the functional recovery acceleration change inflection point, introducing surface electromyography and kinematic coupling coordination degree as a verification index, and generating a hand rehabilitation risk fusion score; S6, performing rehabilitation intervention, and reconfiguring the task difficulty parameters when the hand rehabilitation risk score exceeds the dynamic threshold. 2.The stroke hand rehabilitation training method based on multi-modal data fusion according to claim 1, characterized in that, In S1, the standardized hand function micro-evaluation task is constructed, the hemiplegic hand movement characteristics of stroke patients are collected, and the hand function digital evaluation vector is generated, which specifically includes: Designing a standard gesture action sequence containing different grip patterns and finger separation degrees, tracking the hand joint based on computer vision, and collecting hemiplegic hand movement parameters; Recording joint angle data and movement speed during the execution of the standard gesture action, and extracting multi-dimensional movement characteristics including movement trajectory smoothness and movement range symmetry, and integrating the above data and time marker information into a hand function digital evaluation vector. 3.The stroke hand rehabilitation training method based on multi-modal data fusion according to claim 1, characterized in that, In S2, the continuously collected hand function evaluation vector is subjected to dynamic time warping, the patient movement control entropy value is extracted, the movement coordination fluctuation range during the rehabilitation period is calculated, and the baseline library of stroke patient hand movement patterns containing movement control entropy value and coordination fluctuation range is established, which specifically includes: Continuous collection of hand function digital evaluation vectors of stroke patients at different rehabilitation periods; Aligning the digital evaluation vectors at different rehabilitation periods using the dynamic time warping algorithm, and calculating the minimum cumulative distance between the sequences as the time warping measure; Multi-scale permutation entropy analysis of the joint angle change trajectory after warping, calculation of the movement control entropy value reflecting the complexity of neuromuscular control; Analysis of the phase synchronization relationship between joint angle change trajectories, and calculation of the movement coordination fluctuation range during the rehabilitation period.

4. The stroke hand rehabilitation training method based on multi-modal data fusion according to claim 3, characterized in that, The motion control entropy value reflecting the complexity of neuromuscular control specifically comprises: The hand joint angle change trajectory after dynamic time warping is taken as input data, and the change trajectory is divided into multiple motion trend segments at different scales; Arrangement patterns are constructed based on the size order of data points in the motion trend segments, the frequency of each arrangement pattern in the sequence is counted, and the entropy value at the corresponding scale is calculated according to the discrete degree of the frequency distribution. 5.The stroke hand rehabilitation training method based on multi-modal data fusion according to claim 1, characterized in that, In S3, the hand function digital evaluation vector with the optimal motion control entropy value is selected from the baseline library of hand movement patterns of stroke patients as the individual optimal movement pattern, the similarity distance decay curve of the current hand movement pattern and the individual optimal movement pattern is compared, and the neuromotor consolidation index is calculated based on the motion learning rate, specifically comprising: Based on the similarity comparison of the joint angle change trajectory and the motion speed distribution characteristics, the dynamic time warping algorithm is used to calculate the similarity distance of the current hand movement pattern and the individual optimal movement pattern; The similarity distance decay curve is constructed based on the similarity distance of the continuous rehabilitation period, and the change trend of the curve slope is analyzed; The motion learning rate is calculated by combining the improvement amplitude of the motion control entropy value of the current hand movement pattern, and the neuromotor consolidation index is generated by weighting and fusing the similarity distance decay curve slope and the motion learning rate. 6.The stroke hand rehabilitation training method based on multi-modal data fusion according to claim 1, characterized in that, In S5, a multi-modal fusion matrix of the neuromotor consolidation index and the functional recovery momentum characteristics is established, the matrix includes the similarity distance decay curve slope and the functional recovery acceleration change inflection point feature dimension, the surface electromyography and kinematics coupling coordination degree is introduced as a verification index, and a hand rehabilitation risk fusion score is generated, specifically comprising: The mutual information of the surface electromyography signal and the joint angle change trajectory is calculated to obtain the surface electromyography and kinematics coupling coordination degree; The coupling coordination degree is used as a verification index to weight the output results of the multi-modal fusion matrix; Based on the weighted multi-modal fusion matrix output and the coupling coordination degree verification result, a hand rehabilitation risk fusion score is generated. 7.The stroke hand rehabilitation training method based on multi-modal data fusion according to claim 1, characterized in that, In S6, the rehabilitation intervention is performed, and when the hand rehabilitation risk score exceeds the dynamic threshold, the task difficulty parameters are reconfigured, specifically comprising: Based on the historical hand rehabilitation risk fusion score data in the baseline of stroke hand movement patterns, the statistical distribution characteristics are calculated and the dynamic threshold judgment standard is established; When the real-time hand rehabilitation risk fusion score exceeds the dynamic threshold, the task difficulty parameters are reconfigured, and the accuracy requirement of the joint angle change trajectory and the target range of the motion speed are linearly adjusted according to the proportion exceeding the dynamic threshold.

Images