Hand function rehabilitation evaluation system suitable for stroke patients and training method thereof

Abstract

The application belongs to the technical field of medical rehabilitation, and discloses a hand function rehabilitation evaluation system suitable for stroke patients and a training method thereof, which comprises a data receiving module, a feature fusion module, a muscle coordination quantification module, a motion coordination quantification module and an input feature construction module, and a neural network prediction module.The data receiving module is used for receiving upper limb multi-modal time sequence data of the patient when performing the upper limb motor function evaluation setting action.The feature fusion module is used for fusing the data after feature extraction to obtain a space-time fusion feature matrix.The muscle coordination quantification module is used for extracting a coordination mode similarity, a muscle coordination coupling degree and an activation time sequence synchronism based on the arm electromyography data.The motion coordination quantification module is used for extracting a motion coordination coupling degree based on the kinematics data.The input feature construction module is used for splicing the space-time fusion feature matrix, the coordination mode similarity, the muscle coordination coupling degree, the activation time sequence synchronism and the motion coordination coupling degree to obtain an input feature matrix.The neural network prediction module is used for realizing the prediction of the hand function rehabilitation degree of the patient according to the input feature matrix.Based on the above system, the automatic evaluation of the hand function rehabilitation of the stroke patient can be realized.

Description

Technical Field

[0001] This invention belongs to the field of medical rehabilitation technology, and more specifically, relates to a hand function rehabilitation assessment system and training method suitable for stroke patients. Background Technology

[0002] Stroke is a serious cerebrovascular disease that severely endangers life, health, and quality of life. After onset, it often leads to sequelae such as hand motor dysfunction, sensory abnormalities, and muscle spasms, directly impacting patients' ability to manage daily life and reintegrate into society. Seizing the critical window of early rehabilitation and conducting precise and efficient hand function assessments and rehabilitation interventions are key to improving stroke rehabilitation outcomes and patient prognosis.

[0003] Currently, assessing hand function rehabilitation in patients requires evaluating the stroke motor function rating scale score and / or assessing the patient's current stage of motor function rehabilitation.

[0004] The Fugl-Meyer Assessment (FMA) scale is considered the gold standard for assessing motor function in stroke patients. It covers five dimensions, including motor and sensory functions, with hand function assessment being the core module. The scale uses an ordered scoring system of 0 to 2 points and reflects the degree of hand function impairment and rehabilitation level through the quality of completion of specified actions such as grasping, pinching, and postural maintenance. Its clinical effectiveness and reliability have been fully verified.

[0005] The assessment of the motor function rehabilitation stage is actually an assessment of the patient's rehabilitation stage according to the Brunnstrom classification. The Brunnstrom classification is divided into six rehabilitation stages based on the recovery pattern of hand movement, covering the complete rehabilitation process from the flaccid phase with no voluntary movement to near-normal function, which can provide a staged basis for the development of personalized rehabilitation plans.

[0006] However, traditional hand function rehabilitation assessments mainly rely on the clinical experience and subjective judgment of rehabilitation therapists, which is time-consuming, laborious, and prone to human bias.

[0007] Therefore, how to automate the assessment of hand function rehabilitation in stroke patients is a technical problem that urgently needs to be solved. Summary of the Invention

[0008] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a hand function rehabilitation assessment system and training method suitable for stroke patients, the purpose of which is to realize the automated assessment of hand function rehabilitation of stroke patients.

[0009] According to a first aspect of the present invention, a hand function rehabilitation assessment system suitable for stroke patients is provided, comprising: The data receiving module is used to receive multimodal temporal data of the upper limb when the patient performs the upper limb motor function assessment setting action, including arm electromyography data and kinematic data. The kinematic data includes finger joint posture data, back of hand posture data, arm posture data, and mechanical data resisting external forces. The feature fusion module is used to extract features from the upper limb multimodal temporal data and then fuse the features based on the cross-attention mechanism to obtain a spatiotemporal fusion feature matrix. The muscle coordination quantification module is used to synthesize the arm electromyography data. The electromyographic data of each muscle corresponding to the acquisition channel are constructed into an electromyographic signal matrix, and non-negative matrix decomposition is performed to obtain the muscle synergistic basis matrix. With activation coefficient matrix , Let K be the temporal step size of the electromyography (EMG) data, K be the number of coordinating patterns, and W be the different elements in the muscle coordinating basis matrix representing the activation weights of different muscles under different coordinating patterns. Coordinating pattern similarity is extracted. Muscle synergistic coupling degree And activation timing synchronization The similarity of the collaborative patterns Response muscle synergy basis matrix W and standard muscle synergy basis matrix of healthy individuals The similarity between them, the muscle synergistic coupling degree reflects the degree of abnormal activation coupling of different muscles in the muscle synergistic basis matrix W under each synergistic mode, the activation timing synchronization It reflects the synchronicity of activation time of different muscles in the same synergistic mode; The motion coordination quantization module is used to integrate different kinematic data and construct a kinematic feature matrix. , For feature dimension, The time step size of the kinematic data; the kinematic feature matrix is ​​determined based on principal component analysis. Calculate the kinematic feature matrix based on the first to rth principal components. The ratio of the variance of the first principal component feature to the sum of the variances of the r principal component features is used to construct the motion cooperative coupling degree. ; The input feature construction module is used to combine features under the same action. , , , The feature matrix is ​​concatenated with the spatiotemporal fusion feature matrix to obtain the input feature matrix of the corresponding action; The neural network prediction module is used to predict the functional assessment score of the patient's upper limb performing the corresponding action based on the input feature matrix of different actions, and / or predict the rehabilitation stage of the patient's upper limb motor function.

[0010] According to a second aspect of the present invention, a training method for a hand function rehabilitation assessment system suitable for stroke patients is provided, wherein the above-mentioned hand function rehabilitation assessment system suitable for stroke patients is trained using a training sample set. Each training sample in the training sample set includes an input sample and a sample label. The input sample is the upper limb multimodal temporal data of the patient when performing the upper limb motor function assessment setting action. The sample label is the actual functional assessment score of the patient's upper limb performing the corresponding action and / or the actual rehabilitation stage of the patient's upper limb motor function.

[0011] Compared with existing technologies, the present invention has the following main advantages: 1. In this invention, the hand function rehabilitation assessment system has multiple core modules. The data receiving module receives multimodal temporal data of the upper limbs when the patient performs actions and performs feature fusion through the feature fusion module to obtain a spatiotemporal fusion feature matrix. This spatiotemporal feature matrix carries information related to the patient's upper limb function when performing actions. The muscle coordination quantification module extracts the similarity of the patient's coordination patterns based on the patient's arm electromyography data. Muscle synergistic coupling degree And activation timing synchronization By extracting the above three muscle-level indicators, we can quantify the patient's neuromuscular control ability from three dimensions: the degree of similarity between the patient's muscle coordination pattern and the healthy baseline, the degree of co-activation of abnormal muscle groups, and the sequential relationship of activation of multiple muscles. This provides objective electromyographic evidence for the upper limb function assessment of stroke patients. The motor coordination quantification module extracts the degree of motor coordination coupling based on the patient's kinematic data. By extracting the above-mentioned motor indicators, we can characterize whether a patient's upper limb movements excessively rely on a single dominant motor pattern from the perspective of multi-joint motor performance, quantify their inter-joint separation control ability and the quality of movement completion, and overcome the problem that simple electromyography indicators cannot directly reflect external motor performance. Finally, by integrating the spatiotemporal fusion feature matrix and the core performance indicators extracted above, we construct an information-rich input feature matrix and input it into the neural network prediction module. This allows us to directly obtain the prediction results of hand function rehabilitation assessment, thereby realizing the automated assessment of hand function rehabilitation for stroke patients. Furthermore, because the input feature matrix covers information from multiple core dimensions, it can effectively ensure the accuracy of the prediction results.

[0012] 2. Furthermore, the system is also equipped with a data acquisition device, which includes a flexible bending sensing glove with a sensing module and an electromyography loop with a sensing module. This device can automate data acquisition, transforming the FMA prescribed movements from traditional therapist observation and scoring into measurable, reproducible, and traceable objective data input, thereby reducing experience differences and scoring biases among different assessors.

[0013] 3. Furthermore, the system is also equipped with an external force application device. Based on this device, a controllable resistance detection process can be constructed. The single detection process is standardized through stopping mechanisms such as increasing motor output torque, displacement triggering, and time triggering. This allows for the stable acquisition of mechanical and kinematic responses under different gripping actions. Combined with electromyographic, bending, and posture data from other non-resistance actions, the system can cover multiple movement dimensions of FMA except for neural reflexes, solving the problem that traditional scales cannot quantitatively reflect gripping strength, range of motion, postural compensation, and the standardization of action completion.

[0014] 4. Furthermore, the system also includes a data preprocessing module, which can solve the problems of temporal misalignment, noise interference and information redundancy in clinical acquisition of multi-source heterogeneous sensor data by performing data temporal alignment processing and data denoising processing on the original multimodal time series data.

[0015] 5. Furthermore, the muscle synergy quantification module dynamically iterates based on the variance contribution rate to determine the optimal number of synergy patterns K. Optimizing the number of synergy patterns in the above way can match the number of synergy patterns with the actual muscle activation complexity of the patient, avoiding under- or over-decomposition of muscle synergy features due to a fixed number of synergy patterns, thereby improving the stability and individual adaptability of synergy pattern extraction.

[0016] 6. Furthermore, regarding muscle synergistic coupling... A specific calculation formula is provided, based on which the muscle synergy coupling degree can be accurately and quickly extracted. .

[0017] 7. Furthermore, regarding activation timing synchronization... A specific calculation formula is provided, based on which activation timing synchronization can be accurately and quickly extracted. .

[0018] 8. Furthermore, a spatiotemporal graph neural network is used for prediction. When predicting the action-level representation vector, compensatory inhibition weights are introduced. This reduces the weighting of temporal segments corresponding to compensatory behaviors such as trunk movement, abnormal wrist displacement, and overactivation of non-target muscle groups, minimizing the interference of compensatory actions on the action-level representation vector and making it more accurately reflect the patient's true upper limb functional state. 9. Furthermore, the spatiotemporal graph neural network is used for prediction. Its motor function staging prediction head performs a weighted summation of the action-level representation vectors of all actions based on the attention weights of the actions, obtaining a global representation vector z for the patient's overall upper limb motor function. This global representation vector z is then input into the classification prediction head to predict the patient's upper limb motor recovery stage. During this process, by introducing attention weights to perform a weighted summation of the action-level representation vectors of all actions, the action-level representation vectors of multiple actions can be integrated, enhancing the global representation vector's ability to express the patient's overall upper limb motor function state. This avoids the randomness caused by judging the rehabilitation stage based solely on a single action, thereby improving the stability and reliability of the rehabilitation stage prediction results. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the structure of a hand function rehabilitation assessment system according to an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the external force application device in one embodiment of the present invention.

[0021] Figure 3 This is a visualization of arm electromyography data collected by an 8-channel dry electrode electromyography loop in one embodiment of the present invention.

[0022] Figure 4 This is a visualization of hand IMU pose data in one embodiment of the present invention.

[0023] Figure 5 This is a visualization of finger bending data in one embodiment of the present invention.

[0024] Figure 6 This is a visualization of the velocity-displacement-force curve in one embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0026] Example 1 This invention provides a hand function rehabilitation assessment system suitable for stroke patients, such as... Figure 1 The diagram shown is a structural schematic of a hand function rehabilitation assessment system according to an embodiment of the present invention.

[0027] The data receiving module is used to receive multimodal temporal data of the upper limb when the patient performs the upper limb motor function assessment setting action, including arm electromyography data and kinematic data. The kinematic data includes finger joint posture data, back of hand posture data, arm posture data, and mechanical data of resistance to external forces.

[0028] Specifically, the FMA scale is the gold standard for assessing motor function in stroke patients. When assessing a patient's upper limb motor function based on the FMA scale, the patient needs to complete a series of actions set by the scale corresponding to the upper limb motor function assessment and the upper limb data of the patient performing the actions are collected, including finger joint posture data, arm electromyography data, back of hand posture data, arm posture data, and mechanical data of resistance to external forces.

[0029] In one embodiment, the hand function rehabilitation assessment system further includes a data acquisition device, which includes a flexible bending sensing glove with a sensing module and an electromyography loop with a sensing module.

[0030] The flexible bending sensing glove with a sensing module is used to collect finger joint posture data, back-of-hand posture data, and mechanical data of resistance to external forces during patient movements. Specifically, the flexible bending sensing glove uses medical-grade breathable elastic knitted fabric as the base material, and one piezoresistive flexible bending sensing unit is sewn on the ventral side of the interphalangeal joint (PIP) and metacarpophalangeal joint (MCP) of the thumb, index finger, middle finger, ring finger, and little finger, for a total of 5 basic sensing channels. More specifically, the sensing glove is connected to an NIUSB6100 data acquisition card with a 50Hz sampling rate to collect mechanical data of resistance to external forces. An inertial measurement unit (IMU) (such as an MTWAwinda IMU) is installed in the flexible sensing glove (such as at the Velcro on the back of the hand) to collect hand posture information at a frequency of 50Hz.

[0031] An electromyographic (EMG) loop with a sensing module is deployed on the patient's forearm and upper arm. Specifically, the EMG loop can be an 8-channel dry electrode EMG loop with a sampling rate of 500 Hz, used to capture surface electromyographic signals, reflecting muscle activation intensity and synergistic characteristics. The EMG loop is equipped with an IMU sensing module with a sampling rate of 50 Hz, used to simultaneously acquire the movement posture of the forearm and upper arm. Both provide electromyographic and kinematic data support for the assessment of upper limb motor function in stroke patients.

[0032] In one embodiment, the hand function rehabilitation assessment system further includes an external force application device, which includes a support base, a servo motor 3, a linkage component, and a gripping carrier mounting end. The gripping carrier mounting end is used to install a gripping carrier, which may include cylindrical, spherical, paper-shaped, hook-shaped, or pen-shaped gripping carriers. The servo motor is fixed to the support base and controls the movement of the gripping carrier through the drive linkage component to perform upper limb resistance training on the patient holding the gripping carrier. The external force application device is also used to collect mechanical data during the upper limb resistance training, including servo motor displacement data, output torque, and the force applied to the patient.

[0033] like Figure 2 The diagram shows a schematic of the external force application device in one embodiment of the present invention. Specifically, it includes a locking desktop component 1, a computer interface 2, a tree-shaped motor 3 (4 being the back of the motor), a plastic sleeve 5, a Bowden wire 6, a tension sensor 7, a computer interface 8, a spring 9, and a 3D-printed gripper 10. This device can use a Unitree GO-M8010-6 DC brushless servo motor, and a 3D-printed and metal-machined fixing device to securely mount the motor to the desktop. Five gripping carriers—cylindrical, spherical, paper-like, hook-like, and pen-like—are designed and 3D-printed to serve as resistance gripping carriers for stroke patients. These five models can be connected to the motor output in batches via a Bowden wire, spring, and tension sensor linkage mechanism to achieve stable force transmission.

[0034] The system relies on the high-precision torque control characteristics of servo motors, and can control the motor output torque according to the upper limb hand function assessment requirements of the FMA scale. It starts to increase continuously, and the maximum output torque is limited to... The system is configured with dual stop trigger conditions: when the motor displacement reaches... Or the motor continuously outputs torque for a period of time reaching When the motor stops, it immediately stops running; after the motor stops, there is a delay. The system will automatically reset to its initial position, completing a single resistance test. It can accurately collect and quantify mechanical parameters such as force, displacement, and torque throughout the entire resistance process, providing precise and reproducible quantitative mechanical evidence for the FMA scale assessment of upper limb motor function in stroke patients.

[0035] In general, upper limb motor function assessment training for patients includes two main categories: one is resistance training, in which finger joint posture data, back of hand posture data, arm electromyography data, arm posture data, and mechanical data of resistance to external forces can be collected; the other is other non-resistance training, in which finger joint posture data, back of hand posture data, arm electromyography data, and arm posture data can be collected.

[0036] like Figure 3The image shown is a visualization of the arm electromyography data collected by the 8-channel dry electrode electromyography loop in one embodiment of the present invention.

[0037] like Figure 4 The image shown is a visualization of hand IMU pose data in one embodiment of the present invention, which is the back of the hand pose.

[0038] like Figure 5 The image shown is a visualization of finger bending data in one embodiment of the present invention, which represents the finger joint pose.

[0039] like Figure 6 The figure shown is a visualization of the velocity-displacement-force curve in one embodiment of the present invention, which is mechanical data for resisting external forces.

[0040] The feature fusion module is used to extract features from the multimodal temporal data of the upper limbs and then fuse the features based on the cross-attention mechanism to obtain the spatiotemporal feature matrix.

[0041] Specifically, before feature extraction, the acquired upper limb multimodal time-series data is preprocessed to improve data quality. Therefore, in one embodiment, the system further includes a data preprocessing module, which performs data preprocessing on the received upper limb multimodal time-series data. The data preprocessing includes data temporal alignment and data denoising.

[0042] In one embodiment, considering the inconsistency between the sampling rates of electromyography (EMG) signals and those of other sensors, such as the temporal consistency issue between a 500Hz high sampling rate EMG signal and a 50Hz baseline sampling rate signal, temporal alignment processing is performed on the acquired upper limb multimodal temporal data before feature extraction. The data temporal alignment preprocessing includes sequential downsampling and dynamic time warping. The downsampling operation uses the lowest sampling rate as a baseline and downsamples the upper limb data at other sampling rates to reduce their sampling rates to the baseline. The dynamic time warping operation uses a dynamic time warping algorithm to temporally align the upper limb data with consistent sampling rates.

[0043] For example, the electromyography (EMG) signal is first downsampled to 50Hz to achieve preliminary sampling rate alignment, thereby reducing the risk of timing misalignment while preserving the core features of the EMG signal and obtaining a downsampled EMG envelope signal. Subsequently, the Dynamic Time Warping (DTW) algorithm is used, with the 50Hz reference sampling rate as the time axis, to correct the millisecond-level phase shift caused by differences in sensor startup timing.

[0044] The cost function for performing Dynamic Time Warping (DTW) on any time series data x and y to align time series data y to time series data x is: ; In the formula, i is the timing index of timing data x (50Hz reference signal), and j is the timing index of timing data y (downsampled electromyographic envelope signal). This represents the weighting coefficients dynamically set based on the signal-to-noise ratio of each sensor channel. These are the signal values ​​at their respective indices. This is the cost of timing alignment at this point.

[0045] After calculating the overall cost matrix, the optimal normalization path with the minimum cost is obtained through dynamic programming algorithm. Based on this path, the downsampled electromyographic envelope signal is time-series rearranged and interpolated to achieve millisecond-level time-series closed-loop alignment with the 50Hz reference signal.

[0046] In one embodiment, before feature extraction, the acquired upper limb multimodal time-series data undergoes denoising preprocessing. The denoising preprocessing includes outlier removal and smoothing filtering. The outlier removal operation includes... The criteria remove extreme outliers from the signal to eliminate the influence of sensor malfunctions or external interference, ensuring signal validity. The smoothing filter operation employs a moving average filtering algorithm to suppress environmental noise by dynamically adjusting the filter window length. To balance signal smoothness and motion detail, the calculation formula is as follows: ; ; in, For the signal in The instantaneous rate of change after normalization of the absolute value of the first difference at time step; Limited to between 3 and 10; The filtered output value at time 1; For the first in the filter window The signal values ​​at each time point.

[0047] Furthermore, if the subsequent muscle synergy quantification module still needs to use some of the original 500Hz high-frequency electromyographic signal extraction, it can be preprocessed in advance. For example, a cascaded 50Hz power frequency notch filter and a 20-240Hz bandpass filter can be used to remove baseline drift, power frequency interference and high-frequency noise. At the same time, a sliding window detection is combined to remove peak outliers, and finally, a preprocessed electromyographic signal that retains the fine features of muscle activation is obtained.

[0048] After performing the above data preprocessing, feature extraction is then performed on each time-aligned modal data to obtain the corresponding time-series features, such as the 50Hz multimodal time-series features.

[0049] Subsequently, feature fusion of multimodal temporal features was performed based on the cross-attention mechanism to obtain the spatiotemporal feature matrix. This involves a cross-attention mechanism, which can perform the above fusion operation by setting cross-attention.

[0050] The following is a brief explanation of the cross-attention mechanism.

[0051] Cross-attention mechanism involves query vector Key vector AND value vector : Query vector The temporal features based on inertial measurement units are encoded by a one-dimensional convolutional neural network (1D-CNN) and defined as the query vector. This allows it to carry information about the intent of the action and a phase timestamp; Query vector The temporal features of upper limb posture collected by inertial measurement units of the back of the hand, forearm, and upper arm are temporally aligned and concatenated, and then encoded by a one-dimensional convolutional neural network (1D-CNN) to be defined as a query vector. This allows it to carry information about the intent of the action and a phase timestamp; key vector AND value vector The preprocessed temporally aligned electromyographic features of the arm, finger joint pose features, and mechanical features resisting external forces are cascaded and used together as the reference modality for the cross-attention mechanism, which is then mapped to a key vector. AND value vector And superimpose one-dimensional absolute position encoding to preserve the temporal phase information of action evolution.

[0052] We utilize a cross-attention mechanism to learn an attention weight matrix that combines modality and time dimensions. The weight formula is as follows: ; in, For feature dimensions. This step involves querying the vector. With each modal bond vector The dot product operation enables forced alignment of cross-modal features in the same action logic stage, eliminating asynchronous errors caused by sensor hardware delays.

[0053] Based on the attention weight matrix generated by the above cross-attention mechanism, the temporal features are dynamically reconstructed and optimized for output. Specifically, this is achieved by combining the generated attention weight matrix with the value vector. Perform matrix multiplication. This process relies on the query vector. The model is guided by high-relevance motion phases, resulting in higher feature gains. Simultaneously, it implicitly suppresses baseline wandering and environmental noise in irrelevant phases through lower attention weights. Leveraging the temporal attention distribution, the model automatically captures the activation cycles of actions defined by the Functional Motion Model (FMA), reducing the proportion of redundant data during action intervals and resting states, thus focusing features within the functional motion region. The final output is a fused spatiotemporal feature matrix. ,in To define the feature dimensions, The reference timing step is 50Hz.

[0054] Furthermore, the spatiotemporal feature matrix can also be analyzed. After simplification, the simplified spatiotemporal fusion feature matrix is ​​obtained. .

[0055] The muscle synergy quantification module is used to integrate arm electromyography data. The electromyographic data of each muscle corresponding to the acquisition channel are constructed into an electromyographic signal matrix, and non-negative matrix decomposition is performed to obtain the muscle synergistic basis matrix. With activation coefficient matrix , Let K be the temporal step size of the electromyography (EMG) data, K be the number of coordinating patterns, and different elements in the muscle coordinating basis matrix W represent the activation weights of different muscles under different coordinating patterns; extract the coordinating pattern similarity. Muscle synergistic coupling degree And activation timing synchronization Similarity of collaborative patterns Response muscle synergy basis matrix W and standard muscle synergy basis matrix of healthy individuals The similarity between them, the muscle synergistic coupling degree reflects the degree of abnormal activation coupling of different muscles in the muscle synergistic basis matrix W under each synergistic mode, and the activation temporal synchronization. This module reflects the synchronicity of activation time of different muscles within the same synergistic pattern. Specifically, targeting the core pathological characteristics of abnormal synergistic movements in stroke patients, this muscle synergy quantification module primarily analyzes the patient's arm electromyography data, extracts fine muscle synergistic patterns, and combines them with joint kinematic characteristics to quantitatively identify the rehabilitation stage of the patient's transition from "abnormal synergy" to "dissociated movement." This module aims to provide objective quantitative evidence for FMA scoring and Brunnstrom staging, effectively addressing the technical pain points of traditional assessments that cannot quantify the fineness of neural control and heavily rely on subjective judgment.

[0056] Specifically, during the initial data acquisition phase, electromyographic (EMG) timing data of different muscles in the arm are collected. Each muscle corresponds to one or more acquisition channels. By combining the EMG timing data from different channels, an EMG signal matrix E can be constructed. For example, the preprocessed 500Hz high-resolution EMG signal can be reconstructed into an EMG signal matrix. ,in Number of acquisition channels The time step size can be denoted as, for example, the time step size of a 500Hz high-resolution electromyography signal. This provides a standardized data foundation for subsequent matrix decomposition.

[0057] First, nonnegative matrix decomposition is performed on the electromyographic signal matrix E to accurately capture the minute time-series differences in muscle activation.

[0058] The matrix factorization formula is: ; in, For muscle coordination basis matrix, is a high-resolution temporal activation coefficient matrix, and K is the number of cooperative modes.

[0059] Muscle Synergy Basis Matrix The distribution of non-zero elements in the data directly maps to the typical abnormal muscle group co-activation pattern after stroke, and its elements... This reflects the muscle activation weight of the i-th muscle in the k-th synergistic mode.

[0060] In one embodiment, the optimal number of cooperative patterns K can be determined dynamically and iteratively based on the variance accounted for (VAF). The specific process is as follows: Initialization: Initialize the number of cooperative modes K to K=1; Matrix decomposition: The electromyographic signal matrix E is decomposed into nonnegative matrix components. and and Closest to E; Iterative judgment: Calculate the electromyographic signal matrix E and the current decomposition result. The variance contribution rate (VAF) between the cooperative patterns is used to determine whether it reaches a preset threshold. If it does, the iteration ends and the final K value is output as the optimal number of cooperative patterns; otherwise, the iteration ends. The value is incremented by 1 and the process jumps to perform matrix decomposition until the variance contribution rate (VAF) reaches the preset threshold. Then the iteration ends and the final K value is output as the optimal number of collaborative modes.

[0061] Among them, the electromyographic signal matrix E is calculated and compared with the current decomposition result. The formula for calculating the variance contribution rate (VAF) between the variances is: ; in, It is the square of the F-norm.

[0062] In one embodiment, the preset threshold for variance contribution rate can be set to 90%.

[0063] In one embodiment, a time coherence constraint can also be introduced to perform median filtering on the values ​​within a continuous action cycle, aiming to balance decomposition accuracy and physical meaning, and prevent abnormal parameter fluctuations caused by instantaneous noise.

[0064] Subsequently, based on the matrix factorization results, the following three core indicators were extracted to quantify neural control function.

[0065] Indicator 1: Similarity of Collaboration Patterns .

[0066] Specifically, the muscle synergy basis matrix W is calculated and compared with the standard muscle synergy basis matrix of a healthy person. Similarity between them to construct collaborative pattern similarity Similarity of collaborative patterns This value characterizes how closely the patient's synergistic pattern resembles a healthy level; the closer the value is to 1, the closer the patient's synergistic pattern is to a healthy level. The standard muscle synergistic basis matrix for healthy individuals is also included. It can be the mean benchmark matrix obtained by pre-collecting a healthy sample set and performing offline matrix factorization.

[0067] In one embodiment, the muscle synergy basis matrix W can be calculated along with the standard muscle synergy basis matrix of a healthy person. The mean cosine similarity. Its calculation formula is: ; Where K is the number of collaborative modes; For the patient's muscle synergistic basis matrix The first in One collaborative pattern vector; Standard muscle synergy basis matrix for healthy individuals The first in A cooperative pattern vector, It represents the L2 norm of a vector.

[0068] Indicator 2: Muscle Coordination .

[0069] Specifically, the degree of aberrant activation coupling of different muscles in the muscle synergy basis matrix W under each synergy mode is statistically analyzed to construct the muscle synergy coupling degree. Muscle synergistic coupling degree This indicator represents the degree of coupling in a patient's abnormal comorbidity pattern of movement. The higher the score of this indicator, the deeper the coupling in the patient's abnormal comorbidity pattern of movement.

[0070] In one embodiment, the proportion of abnormally coupled muscle pairs can be statistically analyzed to visually quantify the recovery phase of the separated movement. The calculation formula is as follows: ; In the formula, This refers to the number of muscle pieces collected. Let m be the total number of muscle pairs formed by combining two or more muscles. This represents the activation coefficient of the i-th muscle in the k-th coordination mode within the muscle coordination basis matrix W. This represents the activation weight of the j-th muscle in the k-th coordination mode within the muscle coordination basis matrix W. The preset abnormal co-activation weight threshold, This is an indicator function; the value when the condition is true is greater than the value when the condition is false. For example, the value is 1 when the condition is true, and 0 otherwise.

[0071] Indicator 3: Activation of timing synchronization .

[0072] Specifically, the activation temporal synchronicity of different muscles under the same synergistic mode is calculated to construct activation temporal synchronicity. Activate timing synchronization This indicates the synchronicity of activation of multiple muscles in the same synergistic mode. The higher the score, the more synchronized the activation of multiple muscles in the same synergistic mode, the more stable the patient's nerve coordination and control of multiple muscles, and the better the smoothness of the movement execution.

[0073] In one embodiment, the activation coefficient matrix can be obtained based on the preprocessed electromyographic envelope signal and matrix decomposition. The weighted average activation time of each muscle under different synergistic modes is calculated to characterize the differences in the activation sequence of multiple muscles in the same synergistic mode. ; in, Let be the weighted average activation time of the i-th muscle in the k-th synergistic mode; This represents the average weighted average activation time of all muscles in the k-th synergistic mode. First, the activation intensity of the i-th muscle at each time step is determined based on the preprocessed electromyographic envelope signal. Then, combined with the activation coefficient corresponding to the k-th synergistic mode, a weighted average is calculated for each time step to obtain the weighted average activation time of the i-th muscle in the k-th synergistic mode. Then, average the weighted average activation times of m muscles under the same synergistic mode to obtain the average activation time under that synergistic mode. This formula uses a negative exponential mapping to transform the dispersion of different muscle activation moments into a synchronicity score. .

[0074] The motion coordination quantization module is used to integrate different kinematic data and construct a kinematic feature matrix. , For feature dimension, The time step size of the kinematic data; the kinematic feature matrix is ​​determined based on principal component analysis. Calculate the kinematic feature matrix based on the first to rth principal components. The ratio of the variance of the first principal component feature to the sum of the variances of the r principal component features is used to construct the motion cooperative coupling degree. .

[0075] First, we synthesize different kinematic data and construct a kinematic feature matrix. .

[0076] For example, data collected by a 50Hz inertial measurement unit, bending sensor, and force sensor are preprocessed and reconstructed into a kinematic feature matrix. ,in For feature dimension, This is the timing step size.

[0077] Subsequently, the matrix was extracted using principal component analysis. The principal motion components are identified, and the first r principal components are extracted. The variance contribution rate of the first principal component is calculated to characterize the degree of motion cooperative coupling. , The higher the value, the more concentrated the patient's multi-joint movements are on a single primary movement pattern, the higher the degree of abnormal synergistic coupling, and the weaker the ability to separate movements.

[0078] The calculation formula is as follows: ; in, The variance of the data features of the first principal component; Let be the data variance of the i-th principal component feature.

[0079] The input feature building module is used to combine features under the same action. , , , The feature matrix is ​​concatenated with the spatiotemporal fusion feature matrix to obtain the input feature matrix of the corresponding action.

[0080] Specifically, first, the similarity of the collaborative patterns obtained above... Muscle synergistic coupling degree Activate timing synchronization and motion coordination coupling degree Constructing a collaborative feature vector This provides multi-dimensional, high-order feature support for subsequent accurate scoring models. Then, the collaborative feature vectors under the same action... The spatiotemporal fusion feature matrix output by the feature fusion module is concatenated to obtain the input feature matrix for this action. The subscript g indicates the action number.

[0081] The neural network prediction module is used to predict the functional assessment score of the patient's upper limb performing the corresponding action based on the input characteristics of different actions, and / or to predict the patient's upper limb rehabilitation stage based on the input characteristics of different actions.

[0082] Specifically, the neural network prediction module typically has a feature mapping layer and a prediction head. The neural network prediction module first maps the input feature matrix of the action to the corresponding action-level representation vector based on the feature mapping layer, and then predicts the FMA scale score corresponding to different actions based on the prediction head, or predicts the patient's Brunnstrom recovery stage, or both predicts the FMA scale score corresponding to different actions and predicts the patient's Brunnstrom recovery stage.

[0083] In one embodiment, the neural network prediction module may employ a spatiotemporal graph neural network.

[0084] For the g-th action, construct a multimodal spatiotemporal graph. .in, Represents a set of multimodal sensing nodes. This indicates the connection relationships between nodes. This represents the graph adjacency matrix. The adjacency matrix consists of a fixed physical prior adjacency matrix and a trainable dynamic adjacency matrix.

[0085] in, The anatomical structure of the human body, the kinematic chain of the fingers, the muscle-joint innervation relationship and the mechanical constraints are predetermined; A_phy is a predetermined physical prior adjacency matrix that conforms to the normal upper limb / hand movement law.

[0086] λ is a dynamic adjacency matrix used to learn individualized motor associations of different patients at different stages of rehabilitation; λ is a weighting coefficient.

[0087] feature matrix and adjacency matrix Input a spatiotemporal graph neural network. The network extracts the spatial cooperative relationships between joints, muscles, and mechanical nodes through graph convolution, and extracts dynamic change features during action execution through temporal convolution or temporal attention modules, obtaining action-level deep representations: ; in, The mapping function represents the spatiotemporal graph neural network, which extracts the spatial cooperative relationship between joints, muscles and mechanical nodes through graph convolution, and extracts the temporal dynamic features during the action execution process through temporal convolution or temporal attention module; This represents the deep spatiotemporal representation of the g-th action.

[0088] To reduce the impact of compensatory behaviors such as trunk-driven movements, abnormal wrist displacement, and overactivation of non-target muscle groups on the scoring results, the system generates compensatory inhibition weights based on joint range of motion, postural deviation, consistency of mechanical response, and degree of abnormal co-activation of electromyography, and then deweights abnormal segments: ; In the formula, The compensatory inhibition weight for the g-th action, This represents element-wise multiplication. This indicates a time-series pooling operation. This is the action-level representation vector after removing the compensatory effects.

[0089] After obtaining the action-level representation vector Then, it is input into the prediction head for prediction.

[0090] In one embodiment, a motion score regression prediction head is set up to predict the stroke motor function rating scale score for a single action of the patient.

[0091] Action-level representation vectors As input, it outputs a continuous sub-score for each FMA-defined action, which is used to capture subtle functional improvements that are difficult to reflect by traditional discrete scores of 0, 1, and 2.

[0092] Specifically, the representation vector of the g-th action Input the action rating regression prediction head to obtain the continuous predicted score for the action: ; In the formula, This indicates that the action score regression predicts the head. This represents the continuous prediction score of the stroke motor function assessment scale for the g-th action. To ensure that it conforms to the single-action scoring range of the FMA, the output can be constrained to the interval [0, 2].

[0093] In one embodiment, a motor function staging prediction head can also be set to predict the recovery stage of the patient's upper limb motor function.

[0094] The Brunnstrom staging system (motor function staging) reflects the overall stage of a patient's motor function recovery and is both global and discrete. In this embodiment, global aggregation is performed based on motor representation, and the patient's Brunnstrom stage is output.

[0095] Specifically, set This represents the action-level representation vector of the g-th FMA action. This indicates the number of actions involved in the evaluation. To highlight key actions that contribute more significantly to the overall staging judgment, an attention weight is introduced. The calculation formula is as follows: ; In the formula, For trainable parameter vectors, This represents the contribution weight of the g-th action to the Brunnstrom staging assessment. The overall upper limb motor function representation vector of the patient is obtained by weighting and summing the representations of each action according to the attention weights. ; Here, z represents the global representation after fusing multiple action information, which is used to describe the patient's current overall motor recovery status.

[0096] The global representation vector z is input into the classification prediction head, and the Softmax function outputs the probability distribution of the patient belonging to each Brunnstrom stage: ; in, denoted as the classification prediction head, and p represents the probability distribution of each period category.

[0097] The category with the highest probability is ultimately selected as the predicted staging result: ; in, This represents the probability that the patient belongs to the u-th stage category. This indicates the final predicted Brunnstrom stage result.

[0098] Example 2 The present invention also relates to a training method for a hand function rehabilitation assessment system suitable for stroke patients, which includes: training the hand function rehabilitation assessment system for stroke patients described above using a training sample set; Each training sample in the training sample set includes an input sample and a sample label. The input sample is the upper limb multimodal temporal data of the patient when performing the upper limb motor function assessment setting action. The sample label is the actual functional assessment score of the patient's upper limb performing the corresponding action, and / or the actual rehabilitation stage of the patient's upper limb motor function.

[0099] In one embodiment, considering the technical challenge of scarce labeled samples in rehabilitation clinical settings, targeted data augmentation can be performed on the upper limb multimodal temporal signals after performing time-series alignment and denoising processing on existing clinical samples. This process aims to effectively alleviate model overfitting by expanding the sample set at low cost, thereby improving the generalization ability of subsequent algorithms in small sample environments.

[0100] You can choose any one or more of the following methods to augment data.

[0101] Time-warping enhancement: Local time-scale transformation is performed on the 50Hz reference timing signal and the 50Hz electromyography (EMG) signal, i.e., randomly stretching or compressing timing segments within a ±20% time range. This simulates the speed differences in patient-performed movements while preserving the core temporal relationships of the action, effectively improving the model's adaptability to different movement rhythms.

[0102] Modal Gaussian noise injection: Differentiated noise injection strategies are adopted for different modal physical characteristics: Gaussian noise with a signal-to-noise ratio of 30dB is injected into the 500Hz electromyography signal to simulate the sensor background noise and human physiological noise in complex clinical environments; at the same time, Gaussian noise with a standard deviation of 0.05 is injected into the 50Hz kinematic and mechanical signals to simulate minor environmental interferences without destroying the core motion trajectory, thereby comprehensively enhancing the noise robustness of the model.

[0103] Channel Masking: A masking strategy is constructed to fit the occasional failure scenarios of clinical sensors. For 8-channel EMG signals, 1 to 2 non-core channels are randomly masked each time to simulate poor electrode contact or local acquisition anomalies. Local masking is performed on inertial measurement unit signals to simulate unilateral or local sensor data loss. This strategy forces the model to mine redundant correlation features across channels, significantly improving its predictive stability and practicality under conditions of partial data loss.

[0104] Training also involves a training loss function.

[0105] In one embodiment, if the system predicts a functional evaluation score for a single action, the training loss function includes a local regression loss function. Local regression loss function The bias used to calculate the predicted results of functional assessment scores for a single action can be expressed as: ; In the formula, This is the predicted functional assessment score for the g-th action. Let g be the standard score of the g-th action given by the therapist, and Huber() be the Huber loss function. The Huber loss can take into account the sensitivity of mean square error to small biases and the robustness of absolute error to outlier annotations.

[0106] In one embodiment, if the system predicts the patient's recovery stage, the training loss function includes a classification loss function. Classification loss function To calculate the bias in the prediction results during the rehabilitation phase, the cross-entropy loss function can be used to calculate the discrete classification loss. : ; in, This is the unique hot code for the u-th real Brunnstrom installment tag, where U is the number of Brunnstrom installments, typically 6.

[0107] In one embodiment, the training loss may also take into account the physical prior constraint loss. This constraint, used to ensure the model output conforms to human kinematics and mechanical laws, can be expressed as: ; in, This indicates a loss of range of motion constraint in the joint. This represents the resistance to mechanical consistency constraint loss. This indicates that the act of compensation is a form of punishment for the loss.

[0108] in, This indicates the loss of joint range of motion constraint, which is calculated based on finger joint pose data, back of hand pose data, and arm pose data. It is used to constrain the joint range of motion of the patient when performing the g-th action to be within the preset reasonable range of the corresponding action. The resistance mechanical consistency constraint loss is calculated based on the servo motor displacement data, output torque, applied force value, and velocity-displacement-force value time sequence relationship collected during resistance training. It is used to constrain the motion response and mechanical response to remain consistent during the resistance action process. It represents the penalty loss for compensatory behavior, which is calculated based on postural deviation, abnormal movement of non-target joints, abnormal co-activation of electromyography, and consistency of mechanical response. It is used to penalize compensatory behaviors such as trunk movement, abnormal wrist deviation, and overactivation of non-target muscle groups.

[0109] In practice, all loss functions can be combined to construct a joint loss L: .

[0110] After the model training is completed, during the actual assessment process, the multimodal sensor data collected when the patient completes each FMA prescribed action is input into the trained hand function rehabilitation assessment system to directly predict the FMA continuous score results and Brunnstrom discrete staging results.

[0111] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. It should be noted that the terms "in one embodiment," "for example," and "again" are intended to illustrate the present invention and are not intended to limit the present invention.

[0112] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A hand function rehabilitation assessment system suitable for stroke patients, characterized in that, include: The data receiving module is used to receive multimodal temporal data of the upper limb when the patient performs the upper limb motor function assessment setting action, including arm electromyography data and kinematic data. The kinematic data includes finger joint posture data, back of hand posture data, arm posture data, and mechanical data resisting external forces. The feature fusion module is used to extract features from the upper limb multimodal temporal data and then fuse the features based on the cross-attention mechanism to obtain a spatiotemporal fusion feature matrix. The muscle coordination quantification module is used to synthesize the arm electromyography data. The electromyographic data of each muscle corresponding to the acquisition channel are constructed into an electromyographic signal matrix, and non-negative matrix decomposition is performed to obtain the muscle synergistic basis matrix. With activation coefficient matrix , Let K be the temporal step size of the electromyography (EMG) data, K be the number of coordinating patterns, and W be the different elements in the muscle coordinating basis matrix representing the activation weights of different muscles under different coordinating patterns. Coordinating pattern similarity is extracted. Muscle synergistic coupling degree And activation timing synchronization The similarity of the collaborative patterns Response muscle synergy basis matrix W and standard muscle synergy basis matrix of healthy individuals The similarity between them, the muscle synergistic coupling degree reflects the degree of abnormal activation coupling of different muscles in the muscle synergistic basis matrix W under each synergistic mode, the activation timing synchronization It reflects the synchronicity of activation time of different muscles in the same synergistic mode; The motion coordination quantization module is used to integrate different kinematic data and construct a kinematic feature matrix. , For feature dimension, The time step size of the kinematic data; the kinematic feature matrix is ​​determined based on principal component analysis. Calculate the kinematic feature matrix based on the first to rth principal components. The ratio of the variance of the first principal component feature to the sum of the variances of the r principal component features is used to construct the motion cooperative coupling degree. ; The input feature construction module is used to combine features under the same action. , , , The feature matrix is ​​concatenated with the spatiotemporal fusion feature matrix to obtain the input feature matrix of the corresponding action; The neural network prediction module is used to predict the functional assessment score of the patient's upper limb performing the corresponding action based on the input feature matrix of different actions, and / or predict the rehabilitation stage of the patient's upper limb motor function.

2. The hand function rehabilitation assessment system for stroke patients as described in claim 1, characterized in that, It also includes a data acquisition device, which includes: A flexible bend-sensing glove with a sensing module is used to collect data on the finger joint posture, back of hand posture, and mechanical data on resistance to external forces when a patient performs an action. An electromyography loop with a sensing module is deployed on the patient's forearm and upper arm to capture forearm electromyography data, forearm pose data, upper arm electromyography data, and upper arm pose data when the patient performs movements.

3. The hand function rehabilitation assessment system for stroke patients as described in claim 2, characterized in that, It also includes an external force application device, which comprises a support base, a servo motor, a linkage component, and a gripping carrier mounting end. The gripping carrier mounting end is used to mount a gripping carrier, which may be cylindrical, spherical, paper-shaped, hook-shaped, or pen-shaped. The servo motor is fixed to the support base and controls the movement of the gripping carrier by driving the linkage component to perform upper limb resistance training on the patient holding the gripping carrier. The external force application device is also used to collect mechanical data during the upper limb resistance training, including servo motor displacement data, output torque, and the force applied to the patient.

4. The hand function rehabilitation assessment system for stroke patients as described in claim 1, characterized in that, It also includes a data preprocessing module, which performs data preprocessing on the received upper limb multimodal time-series data and then sends it to the feature fusion module. The data preprocessing includes data time-series alignment processing and data denoising processing.

5. The hand function rehabilitation assessment system for stroke patients as described in claim 1, characterized in that, The muscle synergy quantification module determines the optimal number K of synergy patterns based on the variance contribution rate through dynamic iteration. The iteration process includes: Initialization: Initialize the number of cooperative modes K to K=1; Matrix decomposition: The electromyographic signal matrix E is decomposed into nonnegative matrix components. and and Closest to E; Iterative judgment: Calculate the electromyographic signal matrix E and the current decomposition result. The variance contribution rate (VAF) between the cooperative patterns is used to determine whether it reaches a preset threshold. If it does, the iteration ends and the final K value is output as the optimal number of cooperative patterns; otherwise, the iteration ends. The value is incremented by 1 and the process jumps to perform matrix decomposition until the variance contribution rate (VAF) reaches the preset threshold. Then the iteration ends and the final K value is output as the optimal number of collaborative modes.

6. The hand function rehabilitation assessment system for stroke patients as described in claim 1, characterized in that, Muscle Coordination The calculation formula is: ； In the formula, This refers to the number of muscle pieces collected. Let m be the total number of muscle pairs formed by combining two or more muscles. This represents the activation coefficient of the i-th muscle in the k-th coordination mode within the muscle coordination basis matrix W. This represents the activation weight of the j-th muscle in the k-th coordination mode within the muscle coordination basis matrix W. The preset abnormal co-activation weight threshold, This is an indicator function; its value is greater when the condition is true than when the condition is false.

7. The hand function rehabilitation assessment system for stroke patients as described in claim 1, characterized in that, Activate timing synchronization The calculation formula is: ； Where m is the number of muscle pieces collected. Let be the weighted average activation time of the i-th muscle in the k-th synergistic mode; is the mean of the weighted average activation times of all muscles in the k-th synergistic mode.

8. The hand function rehabilitation assessment system for stroke patients as described in claim 1, characterized in that, The neural network prediction module uses a spatiotemporal graph neural network for prediction. The spatiotemporal graph neural network includes a feature mapping layer and a prediction head. The neural network prediction module first maps the input feature matrix of the action to the action-level representation vector of the corresponding action based on the feature mapping layer, and then performs the prediction of the corresponding task based on the prediction head. Specifically, for the g-th action, the feature mapping layer obtains its feature matrix. After performing spatial graph convolution and temporal modeling operations, action-level deep representations are obtained. Then perform compensatory suppression and pooling operations to obtain the corresponding action-level representation vector. : ； In the formula, For the trainable compensatory inhibition weights for the g-th action, This represents element-wise multiplication. This indicates a time-series pooling operation. Let g be the action-level representation vector of the g-th action.

9. The hand function rehabilitation assessment system for stroke patients as described in claim 1, characterized in that, The neural network prediction module uses a spatiotemporal graph neural network for prediction. The spatiotemporal graph neural network includes a feature mapping layer and a prediction head. The neural network prediction module first maps the input feature matrix of the action to the action-level representation vector of the corresponding action based on the feature mapping layer, and then performs the prediction of the corresponding task based on the prediction head. The prediction head includes a motor function staging prediction head, which is used to predict the recovery stage of the patient's upper limb motor function. The motor function staging prediction head performs a weighted summation of the action-level representation vectors of all actions according to the attention weight of the actions to obtain the global representation vector z of the patient's overall upper limb motor function. Then, the global representation vector z is input into the classification prediction head to predict the recovery stage of the patient's upper limb motor function. Attention weight for the g-th action for: ； In the formula, For a trainable vector of adjustable parameters, Let N represent the action-level representation vector of the g-th action, where N is the number of actions.

10. A training method for a hand function rehabilitation assessment system suitable for stroke patients, characterized in that, include: The hand function rehabilitation assessment system for stroke patients as described in any one of claims 1 to 9 is trained using a training sample set; Each training sample in the training sample set includes an input sample and a sample label. The input sample is the upper limb multimodal temporal data of the patient when performing the upper limb motor function assessment setting action. The sample label is the actual functional assessment score of the patient's upper limb performing the corresponding action and / or the actual rehabilitation stage of the patient's upper limb motor function.

Images