Rehabilitation function and action standardization method and system combining multimode physiological signals and human body posture analysis

Abstract

The invention discloses a rehabilitation skill action standardization method combining multimode physiological signals and human body posture analysis, and belongs to the technical field of rehabilitation, and the method comprises the following steps: 1, constructing a preset standard action library; step 2, user data multi-modal acquisition and processing; step 3, comparing the user data with the standard library and scoring; and 4, performing closed-loop feedback correction. According to the rehabilitation skill action standardization method and system combining the multimode physiological signal and the human body posture analysis, through posture and physiological state monitoring and data processing, the action standardization of a practicer is ensured, and the rehabilitation effect is improved; the state of the practicer is analyzed through the physiological coordination and the mind concentration degree, it is ensured that the practicer can have both shapes and minds, and the rehabilitation effect is improved; according to the physiological and psychological states of the practicer, personalized feedback and correction suggestions are provided, and the pertinence of rehabilitation training is improved.

Description

Technical Field

[0001] The present invention belongs to the field of rehabilitation technology, and in particular to a method and system for standardizing rehabilitation exercises by combining multimodal physiological signals and human posture analysis. Background Art

[0002] Traditional rehabilitation exercises (such as Ba Duan Jin, Wu Qin Xi, Tai Chi, etc.) have an important position in the field of traditional Chinese medicine rehabilitation. They play an important role in promoting rehabilitation and strengthening physical fitness through limb movement, breathing regulation and psychological adjustment. However, there are some limitations in the practice of traditional exercises. First, the standardization of movements is difficult to guarantee, especially in the absence of professional guidance, practitioners are prone to movement deviations, which affects the rehabilitation effect. Secondly, existing technologies rely on manual guidance or single sensor monitoring (such as images, inertial sensors), lack real-time monitoring of the practitioner's physiological state, and cannot accurately judge whether the movement truly achieves the expected rehabilitation effect. In addition, the practice of traditional exercises often ignores the importance of "both form and spirit", that is, the combination of movement with physiological state and neurocognition.

[0003] While there are some sensor-based motion monitoring systems available, most focus solely on the outward manifestation of movement and lack comprehensive assessment of the practitioner's physiological state and neurocognition. For example, some systems capture movement posture through cameras or inertial sensors but fail to integrate physiological signal monitoring. Other systems, while capable of monitoring physiological signals, fail to synchronize analysis with movement posture. Furthermore, existing technologies lack mechanisms for simultaneous monitoring and feedback of posture, physiological state, and neurocognition, making it impossible to ensure that practitioners truly achieve a state of "both physical and mental well-being." Summary of the Invention

[0004] The present invention aims to solve the above-mentioned problems existing in the prior art, and proposes a method and system for standardizing rehabilitation exercises that combines multimodal physiological signals and human posture analysis to achieve synchronous monitoring and feedback of posture and physiological signals, improve rehabilitation effects, ensure the standardization and safety of movements, construct a "form and spirit" rehabilitation movement evaluation and feedback system, and realize dynamic optimization of the training process to solve the problems raised in the background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for standardizing rehabilitation exercises by combining multimodal physiological signals and human posture analysis, comprising the following steps:

[0006] Step 1: Build a preset standard action library

[0007] 1. Data collection stage:

[0008] Select professionally qualified rehabilitation therapists or athletes as action demonstrators to ensure the standardization and compliance of the movements;

[0009] An optical motion capture system is used to synchronously collect joint angles and coordinates of key points of the human body during expert demonstrations. Distributed surface electromyography acquisition equipment, EEG headsets, and ECG acquisition modules are used to synchronously record electromyography, EEG, and ECG.

[0010] 2. Feature modeling stage:

[0011] 2.1. Standard action feature extraction

[0012] 2.1.1. Kinematic feature extraction:

[0013] 1) Use the following formula to calculate the covariance matrix Cω of the joint angular velocity ω to characterize the stability of the movement:

[0014]

[0015] Where N represents the total number of sampling points in the time window;

[0016] 2) Obtain the coordinates of key points of the human body and form a stick figure human posture diagram for visual reference and comparison;

[0017] 3) Calculate the center of gravity based on the coordinates of the key points of the human body, obtain the projected area A of the center of gravity on the bottom surface, and quantify the balance. The calculation method is as follows:

[0018]

[0019] The time series coordinates of the centroid projection point are {(x1,y1),(x2,y2),…,(x n ,y n )}, and x n+1 =x1,y n+1 =y1;

[0020] 2.1.2. Physiological feature extraction

[0021] 1) Use EMG root mean square (RMS) to assess muscle activation:

[0022]

[0023] Where N represents the total number of sampling points in the time window, and xi is the voltage value of the electromyographic signal at the i-th sampling point in the time window;

[0024] 2) Using the EEG α / β wave power ratio R α / β , assess focus:

[0025]

[0026] Among them, P α 、P β are the power of EEG alpha and beta waves respectively;

[0027] 3) Real-time storage of heart rate HR and heart rate variability HRV parameters for reference and comparison;

[0028] 2.2. Standard Library Storage

[0029] The kinematic and physiological characteristic parameters of the expert's movements are stored in the database to form a standard template;

[0030] Step 2: Multimodal Collection and Processing of User Data

[0031] 1. Posture data collection:

[0032] The user's joint angles, motion trajectory, and center of gravity projection are acquired through the inertial measurement unit (IMU) and depth camera;

[0033] 2. Physiological signal collection:

[0034] Muscle signals are collected through surface electromyography (sEMG) sensors;

[0035] Collect EEG signals through EEG headset;

[0036] Collect ECG data through a heart rate monitor;

[0037] 3. Data processing and synchronization:

[0038] After preprocessing the acquired data, resampling is performed to ensure that the data of each modality are aligned on the same time axis;

[0039] After completing data synchronization and alignment, obtain the user's motion characteristics and physiological characteristics;

[0040] Step 3: Comparison and scoring of user data and standard library

[0041] 1. Action Standardization Scoring

[0042]

[0043] Where X represents the user action feature vector, which comes from the inertial sensor; μ is the mean vector of the corresponding action in the standard library, which comes from the standard action library; Σ is the covariance matrix of the standard action library; α is the kinematic weight, which adjusts the proportion of spatial deviation in the total score; β is the temporal weight factor, which adjusts the impact of temporal consistency on the total score; S user 、S expert are user action timing sequence and standard action timing sequence respectively; the specific expression of DTW(·) is as follows:

[0044] DTW(S1, S2)=||S1(i)-S2(j)|| 2 +λ·|t i -tj |

[0045] In DTW(·), S1(i) and S2(j) represent the signal values of the two input signals at the i-th and j-th sampling points respectively; i , t j The timestamp of the corresponding sampling point; λ is the time offset penalty factor;

[0046] Obtained D action The final score is normalized to 0-100;

[0047] 2. Physiological coordination score

[0048] The calculation process of the muscle force timing matching score SRMS is as follows:

[0049] First calculate the correlation coefficient r between the user curve and the expert curve RMS :

[0050]

[0051] Then convert the correlation coefficient rRMS into a 0-100 score, retaining only positive correlations:

[0052] S RMS =100·max(r,0)

[0053] Other characteristic deviation score calculations:

[0054]

[0055] Among them F user,k 、F ideal,k are the kth physiological indicators of the user and the ideal state, such as HR, HRV, R α / β β k is the weight of the kth category of physiological indicators;

[0056] Comprehensive score of physiological coordination:

[0057]

[0058] where γ k is the scoring weight of each feature;

[0059] 3. Balance Rating

[0060] Balance Rating: S balance The calculation process is as follows:

[0061]

[0062] Among them A user is the projection area of the center of gravity during the user's action, A expertis the mean projection area of the center of gravity of the experts for the corresponding action in the standard action library;

[0063] 4. Comprehensive score output

[0064] S total =μ1·D action +μ2·S RMS +μ3·S physio +μ4·S balance

[0065] Among them, μ1, μ2, μ3, and μ4 represent the weights of the corresponding scores;

[0066] Step 4: Closed-loop feedback correction

[0067] 1. Visual feedback:

[0068] Muscle force curve comparison chart: superimposed display of the user and standard library RMS curves, with the correlation coefficient r marked RMS , concentration, and heart rate variability are displayed in real time;

[0069] Abnormal point marking: locates the timing deviation of muscle activation being too early or too late, and displays the muscle force timing matching score in real time;

[0070] Action posture comparison chart: superimposes and displays the key point curves of the user and the standard library for comparison;

[0071] 2. Real-time posture correction based on tactile feedback system

[0072] A vibration motor is placed at the corresponding position of the user. The vibration intensity of the vibration motor is calculated based on the real-time joint angle deviation. PID control is used. The vibration intensity output formula is as follows:

[0073]

[0074] Among them, e(t) is the real-time joint angle deviation, K p , K i , K d are proportional, integral, and differential coefficients respectively, and the output u(t) is the tactile feedback intensity;

[0075] 3. Personalized adaptation mechanism:

[0076] Automatically switches evaluation weights based on action type.

[0077] As a preferred embodiment, the preprocessing includes denoising and filtering, the motion features include joint angles, human body key points, and center of gravity projection area, and the physiological features include RMS, HR, HRV, and Rα / β.

[0078] As a preferred embodiment, the standard action timing sequence is the trajectory of changes in joint angles and muscle force over time.

[0079] As a preferred implementation scheme, the ideal state is to refer to the standard library data or set it by oneself. Due to the large individual differences of some indicators, instead of directly using the expert data of the standard library, they are set according to one's own age, weight, etc.

[0080] A rehabilitation exercise movement standardization system combining multimodal physiological signals and human posture analysis, characterized by comprising:

[0081] Physiological signal monitoring module;

[0082] Posture monitoring module;

[0083] Data processing center;

[0084] Evaluation and feedback module.

[0085] As a preferred embodiment, the physiological signal monitoring module includes:

[0086] ECG monitoring unit: used to collect the practitioner's ECG signals in real time;

[0087] Myoelectric monitoring unit: used to collect the practitioner's electromyographic signals in real time;

[0088] EEG monitoring unit: used to collect the practitioner's EEG signals in real time.

[0089] As a preferred embodiment, the posture monitoring module includes:

[0090] Camera / depth sensor: used to capture the practitioner's posture and movement trajectory;

[0091] Inertial Measurement Unit (IMU): used to measure the practitioner's joint angles and limb movement data.

[0092] As a preferred embodiment, the data processing center is responsible for pre-processing the collected multi-mode signals, including data cleaning, denoising and synchronous alignment;

[0093] Fuse multimodal signals, run feature extraction algorithms, and extract key features.

[0094] As a preferred embodiment, the key features include joint angle, movement speed, heart rate variability, muscle co-contraction index, and α / β wave power spectral density.

[0095] As a preferred embodiment, the evaluation and feedback module evaluates the user's movements and physiological data based on a preset standard movement library and generates correction instructions;

[0096] Provide real-time feedback and correction suggestions to users through visual feedback devices (screen displays correction suggestions and action demonstration videos) and tactile feedback devices (providing real-time tactile prompts through vibration or electrical stimulation to help users adjust their actions);

[0097] Among them, the visual feedback device includes a screen display of correction suggestions and action demonstration videos, and the tactile feedback device provides real-time tactile prompts through vibration or electrical stimulation to help users adjust their movements.

[0098] Compared with the prior art, the technical effects and advantages of the present invention are:

[0099] This method and system for standardizing rehabilitation exercises that combines multimodal physiological signals and human posture analysis. Movement standardization: through posture and physiological state monitoring and data processing, it ensures the practitioner's movement standardization and improves rehabilitation effects.

[0100] This rehabilitation exercise standardization method and system combines multimodal physiological signals and human posture analysis to optimize the state of the practitioner: by analyzing the practitioner's physiological coordination and mental concentration, it ensures that they can achieve "both physical and mental well-being" and improve the rehabilitation effect;

[0101] This rehabilitation exercise standardization method and system combines multimodal physiological signals and human posture analysis, and provides personalized guidance: It provides personalized feedback and correction suggestions based on the practitioner's physiological and psychological state, thereby improving the pertinence of rehabilitation training;

[0102] The rehabilitation exercise movement standardization method and system combining multimodal physiological signals and human posture analysis can make the user's exercise movements more standardized. BRIEF DESCRIPTION OF THE DRAWINGS

[0103] Figure 1 It is a framework diagram of the present invention. DETAILED DESCRIPTION

[0104] In the following description, numerous specific details are provided to provide a more thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without one or more of these details. In other instances, certain technical features well known in the art are not described to avoid confusion with the present invention.

[0105] See also Figure 1 A method for standardizing rehabilitation exercises by combining multimodal physiological signals and human posture analysis, this embodiment includes the following steps:

[0106] Step 1: Build a preset standard action library

[0107] 1. Data collection stage:

[0108] Select professionally qualified rehabilitation therapists or athletes as action demonstrators to ensure the standardization and compliance of the movements;

[0109] An optical motion capture system is used to synchronously collect joint angles and coordinates of key points of the human body during expert demonstrations. Distributed surface electromyography acquisition equipment, EEG headsets, and ECG acquisition modules are used to synchronously record electromyography, EEG, and ECG.

[0110] 2. Feature modeling stage:

[0111] 2.1. Standard action feature extraction

[0112] 2.1.1. Kinematic feature extraction:

[0113] 1) Use the following formula to calculate the covariance matrix Cω of the joint angular velocity ω to characterize the stability of the movement:

[0114]

[0115] Where N represents the total number of sampling points in the time window;

[0116] 2) Obtain the coordinates of key points of the human body and form a stick figure human posture diagram for visual reference and comparison;

[0117] 3) Calculate the center of gravity based on the coordinates of the key points of the human body, obtain the projected area A of the center of gravity on the bottom surface, and quantify the balance. The calculation method is as follows:

[0118]

[0119] The time series coordinates of the centroid projection point are {(x1,y1),(x2,y2),…,(x n ,y n )}, and x n+1 =x1,y n+1 =y1;

[0120] 2.1.2. Physiological feature extraction

[0121] 1) Use EMG root mean square (RMS) to assess muscle activation:

[0122]

[0123] Where N represents the total number of sampling points in the time window, and xi is the voltage value of the electromyographic signal at the i-th sampling point in the time window;

[0124] 2) Using the EEG α / β wave power ratio R α / β , assess focus:

[0125]

[0126] Among them, P α 、P β are the power of EEG alpha and beta waves respectively;

[0127] 3) Real-time storage of heart rate HR and heart rate variability HRV parameters for reference and comparison;

[0128] 2.2. Standard Library Storage

[0129] The kinematic and physiological characteristic parameters of the expert's movements are stored in the database to form a standard template;

[0130] Step 2: Multimodal Collection and Processing of User Data

[0131] 1. Posture data collection:

[0132] The user's joint angles, motion trajectory, and center of gravity projection are acquired through the inertial measurement unit (IMU) and depth camera;

[0133] 2. Physiological signal collection:

[0134] Muscle signals are collected through surface electromyography (sEMG) sensors;

[0135] Collect EEG signals through EEG headset;

[0136] Collect ECG data through a heart rate monitor;

[0137] 3. Data processing and synchronization:

[0138] After preprocessing the acquired data, resampling is performed to ensure that the data of each modality are aligned on the same time axis;

[0139] After completing data synchronization and alignment, obtain the user's motion characteristics and physiological characteristics;

[0140] Step 3: Comparison and scoring of user data and standard library

[0141] 1. Action Standardization Scoring

[0142]

[0143] Where X represents the user action feature vector, which comes from the inertial sensor; μ is the mean vector of the corresponding action in the standard library, which comes from the standard action library; Σ is the covariance matrix of the standard action library; α is the kinematic weight, which adjusts the proportion of spatial deviation in the total score; β is the temporal weight factor, which adjusts the impact of temporal consistency on the total score; S user 、S expert are user action timing sequence and standard action timing sequence respectively; the specific expression of DTW(·) is as follows:

[0144] DTW(S1, S2)=||S1(i)-S2(j)|| 2 +λ·|t i -t j |

[0145] In DTW(·), S1(i) and S2(j) represent the signal values of the two input signals at the i-th and j-th sampling points respectively; i , t j The timestamp of the corresponding sampling point; λ is the time offset penalty factor;

[0146] Obtained D action The final score is normalized to 0-100;

[0147] 2. Physiological coordination score

[0148] The calculation process of the muscle force timing matching score SRMS is as follows:

[0149] First calculate the correlation coefficient r between the user curve and the expert curve RMS :

[0150]

[0151] Then convert the correlation coefficient rRMS into a 0-100 score, retaining only positive correlations:

[0152] S RMS =100·max(r,0)

[0153] Other characteristic deviation score calculations:

[0154]

[0155] Among them F user,k 、F ideal,k are the kth physiological indicators of the user and the ideal state, such as HR, HRV, R α / β β k is the weight of the kth category of physiological indicators;

[0156] Comprehensive score of physiological coordination:

[0157]

[0158] where γ k is the scoring weight of each feature;

[0159] 3. Balance Rating

[0160] Balance Rating: S balance The calculation process is as follows:

[0161]

[0162] Among them A user is the projection area of the center of gravity during the user's action, A expert is the mean projection area of the center of gravity of the experts for the corresponding action in the standard action library;

[0163] 4. Comprehensive score output

[0164] S total =μ1·D action +μ2·S RMS +μ3·S physio +μ4·S balance

[0165] Among them, μ1, μ2, μ3, and μ4 represent the weights of the corresponding scores;

[0166] Step 4: Closed-loop feedback correction

[0167] 1. Visual feedback:

[0168] Muscle force curve comparison chart: superimposed display of the user and standard library RMS curves, with the correlation coefficient r marked RMS , concentration, and heart rate variability are displayed in real time;

[0169] Abnormal point marking: locates the timing deviation of muscle activation being too early or too late, and displays the muscle force timing matching score in real time;

[0170] Action posture comparison chart: superimposes and displays the key point curves of the user and the standard library for comparison;

[0171] 2. Real-time posture correction based on tactile feedback system

[0172] A vibration motor is placed at the corresponding position of the user. The vibration intensity of the vibration motor is calculated based on the real-time joint angle deviation. PID control is used. The vibration intensity output formula is as follows:

[0173]

[0174] Among them, e(t) is the real-time joint angle deviation, K p , K i , K d are proportional, integral, and differential coefficients respectively, and the output u(t) is the tactile feedback intensity;

[0175] 3. Personalized adaptation mechanism:

[0176] Automatically switches evaluation weights based on action type.

[0177] As a preferred embodiment, the preprocessing includes denoising and filtering, the motion features include joint angles, human body key points, and center of gravity projection area, and the physiological features include RMS, HR, HRV, and Rα / β.

[0178] As a preferred embodiment, the standard action timing sequence is the trajectory of changes in joint angles and muscle force over time.

[0179] As a preferred embodiment, the ideal state is to refer to the standard library data or set it yourself. Some indicators have large individual differences (such as HR, HRV, R α / β etc.), instead of directly taking the expert data from the standard library, it is set according to its own age, weight, etc.

[0180] A rehabilitation exercise movement standardization system combining multimodal physiological signals and human posture analysis, characterized by comprising:

[0181] Physiological signal monitoring module;

[0182] Posture monitoring module;

[0183] Data processing center;

[0184] Evaluation and feedback module.

[0185] As a preferred embodiment, the physiological signal monitoring module includes:

[0186] ECG monitoring unit: used to collect the practitioner's ECG signals in real time;

[0187] Myoelectric monitoring unit: used to collect the practitioner's electromyographic signals in real time;

[0188] EEG monitoring unit: used to collect the practitioner's EEG signals in real time.

[0189] As a preferred embodiment, the posture monitoring module includes:

[0190] Camera / depth sensor: used to capture the practitioner's posture and movement trajectory;

[0191] Inertial Measurement Unit (IMU): used to measure the practitioner's joint angles and limb movement data.

[0192] As a preferred embodiment, the data processing center is responsible for pre-processing the collected multi-mode signals, including data cleaning, denoising and synchronous alignment;

[0193] Fuse multimodal signals, run feature extraction algorithms, and extract key features.

[0194] As a preferred embodiment, the key features include joint angle, movement speed, heart rate variability, muscle co-contraction index, and α / β wave power spectral density.

[0195] As a preferred embodiment, the evaluation and feedback module evaluates the user's movements and physiological data based on a preset standard movement library and generates correction instructions;

[0196] Provide real-time feedback and correction suggestions to users through visual feedback devices (screen displays correction suggestions and action demonstration videos) and tactile feedback devices (providing real-time tactile prompts through vibration or electrical stimulation to help users adjust their actions);

[0197] Among them, the visual feedback device includes a screen display of correction suggestions and action demonstration videos, and the tactile feedback device provides real-time tactile prompts through vibration or electrical stimulation to help users adjust their movements.

[0198] The hardware system of the present invention includes the following devices:

[0199] 1. Multi-mode physiological signal monitoring equipment: ECG sensor, surface electromyography sensor, EEG headset.

[0200] 2. Posture monitoring equipment: camera, IMU.

[0201] 3. Data processing terminal: computer or mobile device.

[0202] The software system of the present invention includes the following modules:

[0203] 1. Data acquisition module: used to synchronously collect physiological signals and posture data.

[0204] 2. Data preprocessing module: filter and reduce noise on the collected data.

[0205] 3. Feature extraction module: extracts key features of physiological signals and posture data.

[0206] 4. Evaluation module: Compare the extracted features with the preset standard movement model and physiological index thresholds to evaluate the practitioner's movement standardization and physiological state.

[0207] 5. Feedback and Guidance Module: Provides real-time feedback and corrective suggestions to practitioners based on the evaluation results.

[0208] The specific implementation steps of the present invention are:

[0209] 1. Data Collection

[0210] Physiological signal acquisition: Through electrocardiogram sensors, surface electromyography sensors, and EEG headsets, the practitioner's electrocardiogram, electromyography, and electroencephalogram signals are synchronously collected.

[0211] Posture data collection: Through cameras and IMUs, the practitioner's posture data is captured, including the positions of key limb points, joint angles, and motion trajectories.

[0212] 2. Data Processing

[0213] Data preprocessing: Filter and denoise the collected physiological signals and posture data to remove interference signals and ensure the accuracy and reliability of the data.

[0214] Feature extraction: Extract key features of physiological signals and posture data, such as heart rate, electromyography activation time series, EEG band distribution, joint angle changes, etc.

[0215] 3. Action Assessment

[0216] Posture comparison: Compare the practitioner's posture data with the preset standard movement model to evaluate the standardization of the movement.

[0217] Movement scoring: Based on the comparison results, the practitioner's movements are scored and deficiencies in the movements are pointed out, such as joint angle deviation, inaccurate limb movement trajectory, etc.

[0218] 4. Physiological status assessment

[0219] Physiological signal analysis: Analyze physiological signals to determine whether the practitioner's physiological state meets rehabilitation requirements. For example, whether the heart rate is within a safe range, whether the electromyographic activity indicates appropriate muscle strength, and the degree of concentration.

[0220] 5. Feedback and Correction

[0221] Real-time feedback: Based on the results of movement assessment and physiological status assessment, real-time feedback is provided to practitioners in the form of text, voice or video.

[0222] Corrective Suggestions: Based on the assessment results, corrective suggestions are provided to the practitioner, guiding them to adjust their movements and mental state. For example, they may be prompted to adjust joint angles, control muscle intensity, and relax their mental state.

[0223] Appendix: Terminology

[0224] Both form and spirit:

[0225] "Shape": Movement standardization (joint angles, trajectory accuracy, balance);

[0226] "Spirit": physiological coordination (heart rate / electromyography / electroencephalography concentration, thought-action coupling).

[0227] It should be noted that, in this document, relational terms such as one and two are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0228] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for standardizing rehabilitation exercises by combining multimodal physiological signals and human posture analysis, characterized by comprising the following steps: Step 1: Build a preset standard action library 1. Data collection stage: Select professionally qualified rehabilitation therapists or athletes as action demonstrators to ensure the standardization and compliance of the movements; An optical motion capture system is used to synchronously collect joint angles and coordinates of key points of the human body during expert demonstrations. Distributed surface electromyography acquisition equipment, EEG headsets, and ECG acquisition modules are used to synchronously record electromyography, EEG, and ECG.

2. Feature modeling stage: 2.

1. Standard action feature extraction 2.1.

1. Kinematic feature extraction: 1) Use the following formula to calculate the covariance matrix Cω of the joint angular velocity ω to characterize the stability of the movement: Where N represents the total number of sampling points in the time window; 2) Obtain the coordinates of key points of the human body and form a stick figure human posture diagram for visual reference and comparison; 3) Calculate the center of gravity based on the coordinates of the key points of the human body, obtain the projected area A of the center of gravity on the bottom surface, and quantify the balance. The calculation method is as follows: The time series coordinates of the centroid projection point are {(x1,y1),(x2,y2),…,(x n ,y n )}, and x n+1 =x1,y n+1 =y1; 2.1.

2. Physiological feature extraction 1) Use EMG root mean square (RMS) to assess muscle activation: Where N represents the total number of sampling points in the time window, and xi is the voltage value of the electromyographic signal at the i-th sampling point in the time window; 2) Using the EEG α / β wave power ratio R α / β , assess focus: Among them, P α 、P β are the power of EEG alpha and beta waves respectively; 3) Real-time storage of heart rate HR and heart rate variability HRV parameters for reference and comparison; 2.

2. Standard Library Storage The kinematic and physiological characteristic parameters of the expert's movements are stored in the database to form a standard template; Step 2: Multimodal Collection and Processing of User Data 1. Posture data collection: The user's joint angles, motion trajectory, and center of gravity projection are acquired through the inertial measurement unit (IMU) and depth camera; 2. Physiological signal collection: Muscle signals are collected through surface electromyography (sEMG) sensors; Collect EEG signals through EEG headset; Collect ECG data through a heart rate monitor; 3. Data processing and synchronization: After preprocessing the acquired data, resampling is performed to ensure that the data of each modality are aligned on the same time axis; After completing data synchronization and alignment, obtain the user's motion characteristics and physiological characteristics; Step 3: Comparison and scoring of user data and standard library 1. Action Standardization Scoring Where X represents the user action feature vector, which comes from the inertial sensor; μ is the mean vector of the corresponding action in the standard library, which comes from the standard action library; Σ is the covariance matrix of the standard action library; α is the kinematic weight, which adjusts the proportion of spatial deviation in the total score; β is the temporal weight factor, which adjusts the impact of temporal consistency on the total score; S user 、S expert are user action timing sequence and standard action timing sequence respectively; the specific expression of DTW(·) is as follows: DTW(S1,S2)=||S1(i)-S2(j)||2+λ·|t i -t j | In DTW(·), S1(i) and S2(j) represent the signal values of the two input signals at the i-th and j-th sampling points respectively; i , t j The timestamp of the corresponding sampling point; λ is the time offset penalty factor; Obtained D action The final score is normalized to 0-100; 2. Physiological coordination score The calculation process of the muscle force timing matching score SRMS is as follows: First calculate the correlation coefficient r between the user curve and the expert curve RMS : Then convert the correlation coefficient rRMS into a 0-100 score, retaining only positive correlations: S RMS =100·max(r,0) Other characteristic deviation score calculations: Among them F user,k 、F ideal,k are the kth physiological indicators of the user and the ideal state, such as HR, HRV, R α / β β k is the weight of the kth category of physiological indicators; Comprehensive score of physiological coordination: where γ k is the scoring weight of each feature; 3. Balance Rating Balance Rating: S balance The calculation process is as follows: Among them A user is the projection area of the center of gravity during the user's action, A expert is the mean projection area of the center of gravity of the experts for the corresponding action in the standard action library; 4. Comprehensive score output S total =μ1·D action +μ2·S RMS +μ3·S physio +μ4·S balance Among them, μ1, μ2, μ3, and μ4 represent the weights of the corresponding scores; Step 4: Closed-loop feedback correction 1. Visual feedback: Muscle force curve comparison chart: superimposed display of the user and standard library RMS curves, with the correlation coefficient r marked RMS , concentration, and heart rate variability are displayed in real time; Abnormal point marking: locates the timing deviation of muscle activation being too early or too late, and displays the muscle force timing matching score in real time; Action posture comparison chart: superimposes and displays the key point curves of the user and the standard library for comparison; 2. Real-time posture correction based on tactile feedback system A vibration motor is placed at the corresponding position of the user. The vibration intensity of the vibration motor is calculated based on the real-time joint angle deviation. PID control is used. The vibration intensity output formula is as follows: Among them, e(t) is the real-time joint angle deviation, K p , K i , K d are proportional, integral, and differential coefficients respectively, and the output u(t) is the tactile feedback intensity; 3. Personalized adaptation mechanism: Automatically switches evaluation weights based on action type.

2. The method for standardizing rehabilitation exercises by combining multimodal physiological signals and human posture analysis according to claim 1, characterized in that: The preprocessing includes denoising and filtering, the motion features include joint angles, key points of the human body, and center of gravity projection area, and the physiological features include RMS, HR, HRV, R α / β .

3. The method for standardizing rehabilitation exercises by combining multimodal physiological signals and human posture analysis according to claim 1, characterized in that: The standard action timing sequence is the trajectory of changes in joint angles and muscle force over time.

4. The method for standardizing rehabilitation exercises by combining multimodal physiological signals and human posture analysis according to claim 1, characterized in that: The ideal state is to refer to the standard library data or set it yourself. Due to the large individual differences of some indicators, we do not directly use the expert data from the standard library, but set it according to one's own age, weight, etc.

5. A rehabilitation exercise movement standardization system combining multimodal physiological signals and human posture analysis, based on the rehabilitation exercise movement standardization method combining multimodal physiological signals and human posture analysis according to any one of claims 1 to 4, characterized in that: include: Physiological signal monitoring module; Posture monitoring module; Data processing center; Evaluation and feedback module.

6. The rehabilitation exercise movement standardization system combining multimodal physiological signals and human posture analysis according to claim 5, characterized in that: The physiological signal monitoring module includes: ECG monitoring unit: used to collect the practitioner's ECG signals in real time; Myoelectric monitoring unit: used to collect the practitioner's electromyographic signals in real time; EEG monitoring unit: used to collect the practitioner's EEG signals in real time.

7. The rehabilitation exercise movement standardization system combining multimodal physiological signals and human posture analysis according to claim 5, characterized in that: The posture monitoring module includes: Camera / depth sensor: used to capture the practitioner's posture and movement trajectory; Inertial Measurement Unit (IMU): used to measure the practitioner's joint angles and limb movement data.

8. The rehabilitation exercise movement standardization system combining multimodal physiological signals and human posture analysis according to claim 5, characterized in that: The data processing center is responsible for preprocessing the collected multi-mode signals, including data cleaning, denoising and synchronization alignment; Fuse multimodal signals, run feature extraction algorithms, and extract key features.

9. The rehabilitation exercise movement standardization system combining multimodal physiological signals and human posture analysis according to claim 8, characterized in that: The key features include joint angle, movement speed, heart rate variability, muscle co-contraction index, and α / β wave power spectral density.

10. The rehabilitation exercise movement standardization system combining multimodal physiological signals and human posture analysis according to claim 5, characterized in that: The evaluation and feedback module evaluates the user's movements and physiological data based on a preset standard movement library and generates correction instructions; Provide real-time feedback and correction suggestions to users through visual feedback devices (screen displays correction suggestions and action demonstration videos) and tactile feedback devices (providing real-time tactile prompts through vibration or electrical stimulation to help users adjust their actions); Among them, the visual feedback device includes a screen display of correction suggestions and action demonstration videos, and the tactile feedback device provides real-time tactile prompts through vibration or electrical stimulation to help users adjust their movements.

Images