Intelligent orthopedic rehabilitation adaptive assessment method based on multi-modal physiological feedback

Abstract

The application discloses a kind of intelligent orthopedics rehabilitation adaptive evaluation method and device based on multi-modal physiological feedback, and wherein method includes: the multi-modal physiological data including the electromyogram of subject, knee joint three-dimensional motion trajectory, plantar pressure distribution and cognitive input degree are synchronously collected;According to the MMT muscle strength test of subject, knee joint ROM measurement and VAS pain score, construct three-dimensional baseline vector;Multi-modal physiological data and three-dimensional baseline vector are input into personalized orthopedics rehabilitation model, and the output knee joint function index prediction value and personalized rehabilitation adjustment parameter;According to the rehabilitation adjustment parameter that personalized orthopedics rehabilitation model outputs, dynamically adjust electric stimulation parameter;If cognitive load index exceeds early warning line, then reduce electric stimulation intensity;Otherwise, activate hippocampus wave stimulation mode.The application is adjusted according to the real-time state of subject, avoids blindness and subjectivity in rehabilitation process, improves rehabilitation efficiency and success rate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of knee function reconstruction rehabilitation, in particular to an intelligent orthopedic rehabilitation adaptive assessment method and device based on multi-modal physiological feedback, and a computing device. BACKGROUND

[0002] In the field of orthopedic rehabilitation, traditional rehabilitation assessment and treatment methods mainly rely on the experience of doctors, lack comprehensive analysis of multi-modal data, and the assessment results are not comprehensive and accurate. Since most of the assessment methods are static, they cannot monitor and adjust the dynamic changes in the rehabilitation process in real time, resulting in poor rehabilitation effect. In addition, existing methods usually ignore the cognitive load and attention distraction of patients, affecting the rehabilitation effect.

[0003] To solve the above problems, the present application provides an intelligent orthopedic rehabilitation adaptive assessment method based on multi-modal physiological feedback, which adjusts according to the real-time state of the subject to avoid blindness and subjectivity in the rehabilitation process, and improves the rehabilitation efficiency and success rate. SUMMARY

[0004] In view of the above problems, the present application provides an intelligent orthopedic rehabilitation adaptive assessment method and device based on multi-modal physiological feedback, and a computing device.

[0005] According to one aspect of the present application, an intelligent orthopedic rehabilitation adaptive assessment method based on multi-modal physiological feedback is provided, comprising:

[0006] Synchronously collecting multi-modal physiological data including the electromyographic signal of the subject, the three-dimensional motion trajectory of the knee joint, the plantar pressure distribution, and the cognitive input degree; constructing a three-dimensional baseline vector according to the MMT muscle strength test, the knee joint ROM measurement, and the VAS pain score of the subject;

[0007] Inputting the multi-modal physiological data and the three-dimensional baseline vector into a personalized orthopedic rehabilitation model, outputting a knee function index prediction value and personalized rehabilitation adjustment parameters; wherein the rehabilitation adjustment parameters include electrical stimulation adjustment parameters;

[0008] Taking the knee function index prediction value, the cognitive load index, and the VAS pain score as the state space, and taking the electrical stimulation parameters as the action space, dynamically adjusting the electrical stimulation parameters according to the rehabilitation adjustment parameters output by the personalized orthopedic rehabilitation model;

[0009] Real-time monitoring of the cognitive load index of the subject, wherein the cognitive load index is calculated according to the EEG frequency band energy ratio; comparing the cognitive load index with the set attention distraction warning line; if the cognitive load index exceeds the warning line, reducing the electrical stimulation intensity; otherwise, activating the hippocampal wave stimulation mode.

[0010] In an alternative way, the method for synchronous acquisition of the multi-modal physiological data further comprises:

[0011] The quadriceps muscle raw electromyography signals of the subject are acquired by a four-channel high-density surface electromyography electrode;

[0012] The three-dimensional motion trajectory of the knee joint is calculated in real time by two IMU nodes on the thigh and the lower leg, wherein the three-dimensional motion trajectory of the knee joint comprises the flexion / extension angle, the internal rotation / external rotation angle, and the adduction / abduction angle;

[0013] The standing phase / swinging phase pressure center offset trajectory is captured by a point flexible pressure sensing insole, and a plantar mechanics distribution heat map is constructed;

[0014] The θ wave and α power ratio are monitored by a frontal lobe bipolar lead to quantify the cognitive input degree.

[0015] In an alternative way, the method further comprises:

[0016] The electrical stimulation intensity and the cognitive state and the pain level are dynamically decoupled and controlled according to a three-dimensional state space transfer equation of the cognitive load-motion coupling coefficient; wherein the three-dimensional state space transfer equation is:

[0017]

[0018] wherein, H is the Hamiltonian of the knee joint kinematics; θ is the joint angle vector; CLIM is the cognitive load modulation index; ΔP stim is the adjustment amount of the electrical stimulation parameter; K P is a proportional coefficient for adjusting the amplitude of the electrical stimulation adjustment; ROM is the range of motion; θ i+1 is the i+1th joint angle; is the reference power spectrum integral in the frequency range [α, β]; is the actual power spectrum integral in the frequency range [α, β]; VAS is the pain score; n is the dimension of the joint angle vector; ω is the independent variable of the power spectral density function, representing the frequency.

[0019] In an alternative way, the personalized orthopedic rehabilitation model comprises a multi-modal data preprocessing module, a biomechanical feature extraction module, a neurophysiological feature extraction module, a multi-modal fusion module, and a prediction and optimization module;

[0020] The biomechanical feature extraction module comprises a GCN network layer, a GAT network layer, and a ChebNets network layer, which are used to convert the knee joint biomechanical model into a graph structure to extract deep biomechanical features;

[0021] The neurophysiological feature extraction module includes a TCN network layer and an SE Blocks network block, which are used to capture the time dependence of the EEG signal and the EMG signal.

[0022] The prediction and optimization module includes a high-level policy layer, a low-level policy layer, an Actor network layer, a Critic network layer, an A2C network layer, and a MAML independent meta-learning module, which are used to learn the optimal electrical stimulation strategy.

[0023] In an optional manner, the cost function of the personalized orthopedic rehabilitation model is: total =∫0 T [L biomechanical (t)+L neurophysiological (t)+L clinical (t)]dt+Φ coupling

[0024] Wherein,

[0025] L biomechanical (t) is a biomechanical integral term; L neurophysiological (t) is a neurophysiological integral term; L clinical (t) is a coupling term; θ pred (t) is a predicted knee joint motion trajectory; θ target (t) is a target knee joint motion trajectory; β is a biomechanical memory coefficient; τ is a time decay constant; H is a Hamiltonian; θ(s) is a knee joint motion trajectory at time s; θ α (t); θ α (t) is the power of the EEG signal in the alpha band; θ β (t) is the power of the EEG signal in the beta band; γ ref is the reference EEG band ratio; μ is the cumulative change weight of the electromyographic signal; EMG env (s) is the electromyographic signal envelope at time s; VAS(t) is the visual analog pain score at time t; ROM baseline is the baseline joint range of motion; CLI(s) is the cognitive load index at time s; CLI th is the threshold value of the cognitive load index; ΔP stim (t) is the electrical stimulation parameter change amount at time t; K P is the proportional gain coefficient; CLIM(t) is the cognitive load modulation index at time t.

[0026] In an alternative mode, the personalized orthopedic rehabilitation model further comprises an adaptive meta-learning module, which realizes rapid adaptation of patient-specific parameters through a MAML algorithm;

[0027] wherein, in the initial training stage, general dynamic characteristics of the rehabilitation process are extracted according to cross-patient meta-learning;

[0028] For new patients, a secondary rapid adaptation method is activated, and the prediction error of the personalized orthopedic rehabilitation model is reduced to less than 60% of the baseline level through 3-5 iterations.

[0029] In an alternative mode, the GCN network layer, GAT network layer and ChebNets network layer respectively extract and fuse node features and edge features in the knee biomechanical graph structure through spectral convolution, attention network layer and Chebyshev polynomial approximation network layer, to represent mechanical transmission and interaction during knee movement.

[0030] In an alternative mode, the high-level strategy layer is trained according to an A2C algorithm to learn a long-term electrical stimulation strategy;

[0031] The low-level strategy layer rapidly adapts to individual differences according to a MAML algorithm;

[0032] The Actor network layer and Critic network layer are used to estimate state value functions and policy functions.

[0033] According to another aspect of the present application, an intelligent orthopedic rehabilitation adaptive evaluation device based on multi-modal physiological feedback is provided, comprising:

[0034] A multi-modal data acquisition and baseline construction module is used to synchronously acquire multi-modal physiological data including electromyographic signals, three-dimensional motion trajectories of the knee joint, plantar pressure distribution and cognitive input of the subject, and construct a three-dimensional baseline vector according to MMT muscle strength test, knee joint ROM measurement and VAS pain score of the subject;

[0035] A personalized rehabilitation model prediction and adjustment module is used to input the multi-modal physiological data and the three-dimensional baseline vector into a personalized orthopedic rehabilitation model, and output a knee function index prediction value and personalized rehabilitation adjustment parameters; wherein the rehabilitation adjustment parameters include electrical stimulation adjustment parameters;

[0036] An adaptive electrical stimulation parameter adjustment module is used to take the knee function index prediction value, cognitive load index and VAS pain score as a state space, and electrical stimulation parameters as an action space, and dynamically adjust the electrical stimulation parameters according to the rehabilitation adjustment parameters output by the personalized orthopedic rehabilitation model;

[0037] The cognitive load monitoring and hippocampal wave stimulation module is used for monitoring a cognitive load index of a subject in real time, wherein the cognitive load index is calculated according to an EEG frequency band energy ratio; the cognitive load index is compared with a set distraction warning line; if the cognitive load index exceeds the warning line, the electric stimulation intensity is reduced; otherwise, a hippocampal wave stimulation mode is activated.

[0038] According to another aspect of the present application, a computing device is provided, comprising a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface are in communication with each other through the communication bus;

[0039] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operations corresponding to the above-mentioned intelligent orthopedic rehabilitation adaptive evaluation method based on multi-modal physiological feedback.

[0040] According to the scheme provided by the present application, the following steps are included: synchronously collecting multi-modal physiological data including the electromyographic signal of the subject, the three-dimensional motion trajectory of the knee joint, the plantar pressure distribution and the cognitive input degree; constructing a three-dimensional baseline vector according to the MMT muscle strength test, the knee joint ROM measurement and the VAS pain score of the subject; inputting the multi-modal physiological data and the three-dimensional baseline vector into a personalized orthopedic rehabilitation model to output a knee joint function index prediction value and a personalized rehabilitation adjustment parameter; wherein the rehabilitation adjustment parameter includes an electric stimulation adjustment parameter; taking the knee joint function index prediction value, the cognitive load index and the VAS pain score as a state space, and taking the electric stimulation parameter as an action space, dynamically adjusting the electric stimulation parameter according to the rehabilitation adjustment parameter output by the personalized orthopedic rehabilitation model; monitoring the cognitive load index of the subject in real time, wherein the cognitive load index is calculated according to the EEG frequency band energy ratio; comparing the cognitive load index with a set distraction warning line; if the cognitive load index exceeds the warning line, reducing the electric stimulation intensity; otherwise, activating a hippocampal wave stimulation mode. The present application adjusts according to the real-time state of the subject, avoids blindness and subjectivity in the rehabilitation process, and improves the rehabilitation efficiency and success rate.

[0041] The above description is only a summary of the technical scheme of the present application, in order to more clearly understand the technical means of the present application, the specific embodiments of the present application can be implemented according to the content of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0042] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a better understanding of the preferred embodiments, and are not intended to constrain the application. Moreover, like reference numerals denote same or similar components throughout the attached drawings. In the drawings:

[0043] Figure 1 A flowchart of a method for adaptive assessment of intelligent orthopedic rehabilitation based on multi-modal physiological feedback according to an embodiment of the present application is shown.

[0044] Figure 2 A schematic diagram of a knee rehabilitation device according to an embodiment of the present application is shown. Figure 1 ;

[0045] Figure 3 A schematic diagram of a knee rehabilitation device according to an embodiment of the present application is shown. Figure 2 ;

[0046] Figure 4 A schematic diagram of a knee rehabilitation effect according to an embodiment of the present application is shown. Figure 1 ;

[0047] Figure 2 A schematic diagram of a knee rehabilitation effect according to an embodiment of the present application is shown. Figure 6 ;

[0048] Figure 7 A schematic diagram of a framework of an adaptive assessment device for intelligent orthopedic rehabilitation based on multi-modal physiological feedback according to an embodiment of the present application is shown.

[0049] Figure 1 A schematic diagram of a structure of a computing device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0050] Exemplary embodiments of the present application will be described herein below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it is understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present application will be thoroughly and completely understood, and so that the scope of the present application will be completely conveyed to those skilled in the art.

[0051] Figure 1 A flowchart of a method for adaptive assessment of intelligent orthopedic rehabilitation based on multi-modal physiological feedback according to an embodiment of the present application is shown. Specifically, as shown in Figure 2 , the method comprises the following steps:

[0052] In step S101, multi-modal physiological data including electromyography signals of the subject, three-dimensional motion trajectory of the knee joint, plantar pressure distribution, and cognitive input degree are synchronously collected; and a three-dimensional baseline vector is constructed according to MMT muscle strength test, knee joint ROM measurement, and VAS pain score of the subject.

[0053] In this embodiment, by synchronously collecting multi-modal physiological data including electromyography signals, three-dimensional motion trajectory of the knee joint, plantar pressure distribution, and cognitive input degree, the limitation of single modal data is avoided. A three-dimensional baseline vector is constructed according to MMT muscle strength test, knee joint ROM measurement, and VAS pain score of the subject, which provides personalized rehabilitation evaluation basis for each subject and improves the pertinence of the rehabilitation scheme. Furthermore, by constructing the three-dimensional baseline vector, each parameter in the rehabilitation process is monitored and dynamically adjusted in real time, which improves the scientificity of the rehabilitation evaluation.

[0054] Specifically, as shown in Figure 3 、 Figure 4 , the quadriceps raw electromyography signals of the subject are collected by four-channel high-density surface electromyography electrodes to monitor muscle activity. The three-dimensional motion trajectory of the knee joint (including flexion / extension angle, internal / external rotation angle, and adduction / abduction angle) is calculated in real time by two IMU nodes on the thigh and the lower leg, thereby reflecting the motion state of the knee joint. The standing phase / swing phase pressure center offset trajectory is captured by the point flexible pressure sensing insole, and the plantar mechanics distribution heat map is constructed to evaluate the gait and mechanics distribution of the subject. The cognitive input degree of the subject is quantified by monitoring the θ wave and α power ratio of the frontal lobe bipolar lead to monitor the cognitive state in the rehabilitation process. Then, the muscle strength of the subject is evaluated by muscle strength test, the knee joint range of motion of the subject is evaluated by joint range of motion measurement, and the pain level of the subject is evaluated by pain score, and the above data (i.e., MMT muscle strength test, knee joint ROM measurement, and VAS pain score of the subject) are integrated to construct a three-dimensional baseline vector.

[0055] In an alternative way, the method for synchronously collecting multi-modal physiological data further comprises:

[0056] The quadriceps raw electromyography signals of the subject are collected by four-channel high-density surface electromyography electrodes;

[0057] The three-dimensional motion trajectory of the knee joint is calculated in real time by two IMU nodes on the thigh and the lower leg, wherein the three-dimensional motion trajectory of the knee joint includes flexion / extension angle, internal / external rotation angle, and adduction / abduction angle;

[0058] The standing phase / swing phase pressure center offset trajectory is captured by the point flexible pressure sensing insole, and the plantar mechanics distribution heat map is constructed;

[0059] The cognitive input degree is quantified by monitoring the θ wave and α power ratio of the frontal lobe bipolar lead.

[0060] In this embodiment, the electromyography signal reflects the degree of muscle excitation, the movement trajectory reflects the range and angle of joint movement, the plantar pressure distribution reflects the body balance and support condition, and the cognitive engagement reflects the degree of attention concentration of the patient. The data of each of the above modalities complement each other, and the rehabilitation state of the patient is more accurately evaluated. The four-channel high-density surface electromyography electrode more accurately captures the muscle activity pattern of the quadriceps femoris muscle and distinguishes the contribution of different muscles. The IMU (inertial measurement unit) solves the three-dimensional movement trajectory of the knee joint in real time, avoids the limitations of traditional optical motion capture systems, and can be measured in a natural environment. The point flexible pressure sensing insole accurately captures the plantar pressure distribution. The frontal lobe bipolar lead monitors the theta wave and alpha power ratio as an objective quantitative cognitive engagement method, which can avoid subjective bias. All sensors are non-invasive and will not cause pain and discomfort to the patient, improving the patient's acceptance and compliance. The above light IMU, pressure sensing insole and surface electromyography electrode and other equipment are convenient to carry and use, and can be measured in clinical, family and community environments.

[0061] Specifically, the four-channel high-density surface electromyography electrode (including electrode sheet, lead wire and electromyography signal acquisition instrument) is placed on the quadriceps femoris muscle, fixed according to the standard electrode placement position, and ensures good contact between the electrode and the skin. The IMU node is fixed on the thigh and lower leg respectively using a strap or tape, ensuring that the sensor does not slide or move and performing initialization calibration. The pressure sensing insole is placed inside the shoe and covers the entire plantar surface. The bipolar lead EEG electrode is placed in the frontal lobe position and fixed according to the standard electrode placement position. The synchronization trigger signal or software synchronization mechanism is used to make all sensors start data acquisition at the same time point, and the appropriate sampling frequency is set to capture sufficient signal details (for example, the sampling frequency of the electromyography signal is usually 1000 Hz, the sampling frequency of the IMU is usually 100 Hz, and the sampling frequency of the EEG signal is usually 250 Hz).

[0062] In step S102, the multi-modal physiological data and the three-dimensional baseline vector are input into the personalized orthopedic rehabilitation model, and a knee function index prediction value and personalized rehabilitation adjustment parameters are output; wherein the rehabilitation adjustment parameters include electrical stimulation adjustment parameters.

[0063] In this embodiment, according to the multi-modal physiological data (electromyographic signals, three-dimensional motion trajectory of the knee joint, plantar pressure distribution, cognitive engagement) and three-dimensional baseline vector (MMT muscle strength test, knee joint ROM measurement, VAS pain score) of the subject, the rehabilitation parameters can be dynamically adjusted according to the real-time physiological feedback and clinical evaluation data of the patient, adaptive rehabilitation is realized, the rehabilitation process is more accurately controlled, and the rehabilitation effect is improved. For example, the preprocessed multi-modal physiological data and three-dimensional baseline vector are input into the personalized orthopedic rehabilitation model to predict the knee joint function index (60 points) and the personalized electrical stimulation adjustment parameters (electrical stimulation intensity of 10 mA, frequency of 20 Hz, pulse width of 200 μs) of the patient. According to the electrical stimulation adjustment parameters output by the model, the parameters of the electrical stimulation device are set, the patient is guided to perform quadriceps femoris electrical stimulation training and the corresponding rehabilitation exercise is performed. The multi-modal physiological data of the patient is collected again and the clinical evaluation is performed, the model predicts that the knee joint function index of the patient is 65 points, indicating that the rehabilitation effect has improved, and according to the new electrical stimulation adjustment parameters output again (electrical stimulation intensity of 12 mA, frequency of 22 Hz, pulse width of 210 μs), the parameters of the electrical stimulation device are adjusted, and the rehabilitation training is continued.

[0064] In an alternative way, the personalized orthopedic rehabilitation model comprises a multi-modal data preprocessing module, a biomechanical feature extraction module, a neurophysiological feature extraction module, a multi-modal fusion module, and a prediction and optimization module;

[0065] The biomechanical feature extraction module comprises a GCN network layer, a GAT network layer, and a ChebNets network layer, which are used to convert the knee joint biomechanical model into a graph structure to extract deep biomechanical features;

[0066] The neurophysiological feature extraction module comprises a TCN network layer and an SE Blocks network block, which are used to capture the time dependence of EEG signals and EMG signals;

[0067] The prediction and optimization module comprises a high-level strategy layer, a low-level strategy layer, an Actor network layer, a Critic network layer, an A2C network layer, and a MAML independent meta-learning module, which are used to learn the optimal electrical stimulation strategy.

[0068] In this embodiment, the EEG signal reflects the neural activity of the patient, the EMG signal reflects the muscle activity, the joint angle is the flexion and extension angle range of the knee joint, the gait data includes step speed, step length, step frequency, etc., the pain score is the pain degree evaluated by the visual analogue scale (VAS), and the subjective feedback of the patient during rehabilitation includes fatigue degree, comfort and the like. Through the multi-modal data preprocessing module, the EEG signal, the EMG signal, the joint angle and the gait data are preprocessed, the biomechanical feature extraction module converts the knee joint biomechanical model into a graph structure, the GCN, GAT and ChebNets network layers are used to extract deep biomechanical features, the neurophysiological feature extraction module uses the TCN network layer to capture the time dependence of the EEG signal and the EMG signal, and the SE Blocks network block is used to enhance the representation ability of important features, and the multi-modal fusion module fuses the biomechanical features and the neurophysiological features to generate comprehensive feature representation. Through the high-level strategy layer, the low-level strategy layer, the Actor network layer, the Critic network layer, the A2C network layer and the MAML independent meta-learning module, the optimal electrical stimulation strategy shown in Table 1 is learned.

[0069] Table 1

[0070]

[0071] In an optional manner, the high-level strategy layer is trained according to the A2C algorithm to learn a long-term electrical stimulation strategy.

[0072] The low-level strategy layer quickly adapts to individual differences according to the MAML algorithm.

[0073] The Actor network layer and the Critic network layer are used to estimate the state value function and the policy function.

[0074] In this embodiment, combined with Actor (strategy) and Critic (value) network, A2C algorithm reduces the variance of policy update by using advantage function, so that the learning process is more stable and easier to learn long-term dependencies, which is crucial for electrical stimulation strategy, because the correct stimulation needs to be coordinated in time to achieve the expected effect. Fast adaptation to individual differences (MAML, Model-Agnostic Meta-Learning) is a meta-learning algorithm that can quickly adapt to new tasks during training. In the context of electrical stimulation strategy, it can quickly adapt to physiological differences, disease states, etc. of different patients without starting from scratch, significantly reducing the time required for personalized adjustment for each patient. Actor-Critic architecture learns the optimal policy (through Actor network) and state value function (through Critic network) simultaneously. Among them, the Critic network evaluates the pros and cons of the current policy and feeds back information to the Actor network, guiding the improvement of the policy, and the learning process is more efficient because the update of the policy is based on the estimation of the value function, rather than just relying on the reward signal.

[0075] In particular, the A2C high-level policy layer inputs the current state of the patient and outputs a series of low-level policy targets or instructions, for example, the high-level policy can decide to increase muscle force output by 20% in the next 10 seconds. Training is performed using the A2C algorithm, where the reward signal is based on long-term goals (improve motor ability, reduce pain, etc.). The actor network outputs a probability distribution over actions, and the critic network outputs a value estimate for the state. The MAML low-level policy layer inputs the target instructions from the high-level policy, the current state of the patient, and local observations (muscle activity, electrophysiological signals, etc.), and outputs specific electrical stimulation parameters (such as current intensity, pulse width, frequency). Training is performed using the MAML algorithm, which contains two loops: an inner loop that fine-tunes the low-level policy using a small amount of patient-specific data, and an outer loop that updates the initial parameters of the model to enable rapid adaptation to new patients in the inner loop. The actor-critic network layer learns a policy function that outputs actions (electrical stimulation parameters) based on the current state (input). The critic network evaluates a state value function that outputs an estimate of how good the state is based on the current state (input). For example, for a patient with slow walking speed after stroke, the high-level policy can instruct the low-level policy to increase the activity level of the thigh muscles during walking, especially during the swing phase. The low-level policy stimulates the current intensity and frequency of the thigh muscles based on the electromyography signals of the patient to help the patient lift the leg. For a patient with asymmetric gait after stroke, the high-level policy can instruct the low-level policy to adjust the stimulation strategy to balance the activity level of the muscles on both sides. The low-level policy improves gait symmetry based on gait data and electromyography signals of the patient.

[0076] In an alternative way, the cost function of the personalized orthopedic rehabilitation model is: total =∫0 T [L biomechanical (t)+L neurophysiological (t)+L clinical (t)]dt+Φ coupling

[0077] wherein,

[0078] L biomechanical (t) is the biomechanical integral term; L neurophysiological (t) is the neurophysiological integral term; L clinical (t) is the coupling term; θ pred (t) is the predicted knee joint motion trajectory; θ target(t) is the target knee motion trajectory; β is the biomechanical memory coefficient; τ is the time decay constant; H is the Hamiltonian; θ(s) is the knee motion trajectory at time s; θ α (t) is the target knee motion trajectory; β is the biomechanical memory coefficient; τ is the time decay constant; H is the Hamiltonian; θ(s) is the knee motion trajectory at time s; θ α (t) is the power of the EEG signal in the alpha band; θ β (t) is the power of the EEG signal in the beta band; γ ref is the reference EEG band ratio; μ is the cumulative change weight of the electromyographic signal; EMG env (s) is the electromyographic signal envelope at time s; VAS(t) is the visual analog pain score at time t; ROM baseline is the baseline range of joint motion; CLI(s) is the cognitive load index at time s; CLI th is the threshold value of the cognitive load index; ΔP stim (t) is the amount of change in the electrical stimulation parameter at time t; K P is the proportional gain coefficient; CLIM(t) is the cognitive load modulation index at time t.

[0079] In this embodiment, rehabilitation is a dynamic process, and the effect of past treatment will affect future results. The cumulative effect and time dependence of the rehabilitation process are considered by integral form, which can capture the dependence relationship in time. For example, The "memory effect" in biomechanics is considered, that is, the influence of past forces on the current joint motion trajectory will decay over time, but there is still a certain "memory", so it is more consistent with the real situation of human motion (rehabilitation effects are shown in Figure 5 、 Figure 6 Neurophysiological feedback uses the frequency band ratio of electroencephalogram signals to reflect the state of neural activity. Since neural plasticity is very important in the rehabilitation process, the cognitive state and neuromuscular control ability of the patient are evaluated by monitoring the electroencephalogram signal, so as to adjust the treatment plan.

[0080] In an alternative way, the GCN network layer, the GAT network layer and the ChebNets network layer respectively extract the node features and edge features in the knee biomechanics graph structure through spectral convolution, attention network layer and Chebyshev polynomial approximation network layer, and fuse them to represent the mechanical transmission and interaction in the knee motion process.

[0081] In this embodiment, the knee biomechanics is modeled as a graph structure, which can naturally express the connection relationship and mechanical transmission path between each component of the knee joint (e.g., bones, muscles, ligaments), wherein the nodes of the graph represent each component of the knee joint (femur, tibia, patella, quadriceps, hamstring, gastrocnemius, anterior cruciate ligament (ACL), posterior cruciate ligament (PCL), medial collateral ligament (MCL), lateral collateral ligament (LCL), articular cartilage), and the edges of the graph represent the connection relationship and mechanical transmission path between the nodes (joint connection between bones, attachment point between muscle and bone, ligament connected bone, mechanical interaction between nodes). The traditional machine learning method is difficult to effectively process the above complex structured data, and further uses Chebyshev polynomial approximation to approximate the spectral graph convolution, avoiding complex Fourier transform, thereby improving the calculation efficiency and faster training and reasoning.

[0082] In an optional mode, the personalized orthopedic rehabilitation model further comprises an adaptive meta-learning module, which realizes rapid adaptation of patient-specific parameters through a MAML algorithm;

[0083] In the initial training stage, the general dynamic characteristics of the rehabilitation process are extracted according to the cross-patient meta-learning;

[0084] For a new patient, a secondary rapid adaptation method is activated, and the prediction error of the personalized orthopedic rehabilitation model is reduced to 60% or less of the baseline level through 3-5 iterations.

[0085] In this embodiment, the MAML (Model-Agnostic Meta-Learning) algorithm can quickly adapt to new tasks, and through a few iterations, the general knowledge learned from a large amount of patient data can be transferred to a new patient, and the model does not need to be trained from scratch for each patient. In addition, the MAM algorithm itself is model-agnostic, so it can be applied to various different model architectures, and the most suitable basic model for orthopedic rehabilitation tasks can be flexibly selected.

[0086] Step S103, taking the knee function index prediction value, cognitive load index and VAS pain score as the state space, and taking the electrical stimulation parameter as the action space, dynamically adjusting the electrical stimulation parameter according to the rehabilitation adjustment parameter output by the personalized orthopedic rehabilitation model.

[0087] In this embodiment, based on the personalized orthopedic rehabilitation model, the electrical stimulation parameter is dynamically adjusted according to the state of the patient's knee function, cognitive load and pain degree, realizing truly personalized treatment, while the traditional electrical stimulation treatment usually uses fixed parameters, ignoring the differences between patients.

[0088] Specifically, a state space is defined, the knee function index is used to evaluate the functional state of the knee joint and convert it into a numerical value, such as the WOMAC (Western Ontario and McMaster Universities Osteoarthritis Index) or other similar scales. The cognitive load index refers to the mental effort required by the patient during rehabilitation, which can be evaluated using the NASA-TLX (Task Load Index) method, which is a subjective evaluation scale that measures cognitive load by evaluating mental demand, physical demand, temporal demand, performance, effort, and frustration levels. The VAS pain score uses the Visual Analog Scale (VAS) to allow patients to subjectively rate the level of pain (0-10). Then define the action space, combine the value range of each electrical stimulation parameter into an action vector (frequency, pulse width, current intensity, duty cycle, electrode position).

[0089] In an alternative way, the method further comprises:

[0090] According to the cognitive load-motion coupling coefficient three-dimensional state space transfer equation, the electrical stimulation intensity and the cognitive state and the pain level are dynamically decoupled and controlled; wherein the three-dimensional state space transfer equation is:

[0091]

[0092] wherein, H is the Hamiltonian of the knee joint kinematics; θ is the joint angle vector; CLIM is the cognitive load modulation index; ΔP stim is the adjustment amount of the electrical stimulation parameter; K P is a proportional coefficient for adjusting the amplitude of electrical stimulation adjustment; ROM is the range of motion; θ i+1 is the i+1 joint angle; is the reference power spectrum integral in the frequency range [α, β]; is the actual power spectrum integral in the frequency range [α, β]; VAS is the pain score; n is the dimension of the joint angle vector; ω is the independent variable of the power spectral density function, representing the frequency.

[0093] In this embodiment, the adjustment amount of electrical stimulation is closely related to the kinematics of the knee joint, cognitive load and pain level through the three-dimensional state space transition equation, realizing the fine regulation of the electrical stimulation parameters, avoiding the one-size-fits-all parameter setting in the traditional method, and being more in line with the individual differences of patients. Considering the influence of cognitive load on rehabilitation effect, the electrical stimulation intensity is dynamically decoupled from the cognitive state to avoid the increase of cognitive burden caused by excessive stimulation, thereby affecting the compliance and rehabilitation process of patients. The kinematics Hamiltonian of the knee joint is used to more comprehensively describe the motion state of the knee joint, so that the adjustment of the electrical stimulation parameters has more biomechanical basis.

[0094] In step S104, the cognitive load index of the subject is monitored in real time, wherein the cognitive load index is calculated according to the EEG frequency band energy ratio; the cognitive load index is compared with the set attention distraction warning line; if the cognitive load index exceeds the warning line, the electrical stimulation intensity is reduced; otherwise, the hippocampal wave stimulation mode is activated.

[0095] In this embodiment, the cognitive state of the subject is tracked in real time by monitoring the EEG data in real time and calculating the cognitive load index, so that the intervention measures are more accurate. The hippocampal wave stimulation mode is activated to promote memory consolidation and improve cognitive function. The present embodiment is easy to combine with other physiological indicators (such as eye tracking, heart rate variability, etc.) to further improve the accuracy of cognitive load assessment.

[0096] According to the scheme provided by the present application, the following steps are included: synchronously collecting multi-modal physiological data including the electromyographic signal of the subject, the three-dimensional motion trajectory of the knee joint, the plantar pressure distribution and the cognitive input degree; constructing a three-dimensional baseline vector according to the MMT muscle strength test, the knee joint ROM measurement and the VAS pain score of the subject; inputting the multi-modal physiological data and the three-dimensional baseline vector into a personalized orthopedic rehabilitation model to output a knee joint function index prediction value and personalized rehabilitation adjustment parameters; wherein the rehabilitation adjustment parameters include electrical stimulation adjustment parameters; taking the knee joint function index prediction value, the cognitive load index and the VAS pain score as a state space, and taking the electrical stimulation parameters as an action space, the electrical stimulation parameters are dynamically adjusted according to the rehabilitation adjustment parameters output by the personalized orthopedic rehabilitation model; monitoring the cognitive load index of the subject in real time, wherein the cognitive load index is calculated according to the EEG frequency band energy ratio; comparing the cognitive load index with the set attention distraction warning line; if the cognitive load index exceeds the warning line, the electrical stimulation intensity is reduced; otherwise, the hippocampal wave stimulation mode is activated. The present application adjusts according to the real-time state of the subject, avoids the blindness and subjectivity in the rehabilitation process, and improves the rehabilitation efficiency and success rate.

[0097] Figure 7A schematic diagram of the framework of an intelligent orthopedic rehabilitation adaptive assessment device based on multimodal physiological feedback according to an embodiment of the present invention is shown. The intelligent orthopedic rehabilitation adaptive assessment device based on multimodal physiological feedback includes:

[0098] The multimodal data acquisition and baseline construction module 610 is used to simultaneously acquire multimodal physiological data including the subject's electromyography signals, three-dimensional motion trajectory of the knee joint, plantar pressure distribution, and cognitive engagement; and to construct a three-dimensional baseline vector based on the subject's MMT muscle strength test, knee joint ROM measurement, and VAS pain score.

[0099] The personalized rehabilitation model prediction and adjustment module 620 is used to input the multimodal physiological data and the three-dimensional baseline vector into the personalized orthopedic rehabilitation model, and output the predicted value of the knee joint function index and personalized rehabilitation adjustment parameters; wherein, the rehabilitation adjustment parameters include electrical stimulation adjustment parameters;

[0100] The adaptive electrical stimulation parameter adjustment module 630 is used to dynamically adjust the electrical stimulation parameters according to the rehabilitation adjustment parameters output by the personalized orthopedic rehabilitation model, with the predicted value of the knee joint function index, the cognitive load index and the VAS pain score as the state space and the electrical stimulation parameters as the action space.

[0101] The cognitive load monitoring and hippocampal stimulation module 640 is used to monitor the cognitive load index of the subject in real time, wherein the cognitive load index is calculated based on the EEG band energy ratio; the cognitive load index is compared with a set attention distraction warning line; if the cognitive load index exceeds the warning line, the intensity of electrical stimulation is reduced; otherwise, the hippocampal stimulation mode is activated.

[0102] Figure 7 The diagram shows a structural schematic of an embodiment of the computing device of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the computing device.

[0103] like ​ As shown, the computing device may include: a processor 702, a communications interface 504, a memory 706, and a communications bus 708.

[0104] The processor 702, communication interface 704, and memory 706 communicate with each other via communication bus 708. Communication interface 704 is used to communicate with other network elements such as clients or other servers. Processor 702 executes program 710, specifically performing the relevant steps in the above embodiment of the intelligent orthopedic rehabilitation adaptive assessment method based on multimodal physiological feedback.

[0105] In particular, the program 710 can include program code comprising computer operation instructions.

[0106] The processor 702 can be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to perform the operations of embodiments of the application. The one or more processors included in the computing device can be of the same type or different types, such as one or more CPUs and one or more ASICs.

[0107] The memory 706 is used to store the program 710. The memory 706 can include a high-speed RAM memory and can also include a non-volatile memory such as at least one disk memory.

[0108] According to the scheme provided by the application, the scheme comprises: synchronously collecting multi-modal physiological data including an electromyography signal of a subject, a three-dimensional motion trajectory of a knee joint, a plantar pressure distribution, and a cognitive load; constructing a three-dimensional baseline vector according to a MMT muscle strength test, a knee joint ROM measurement, and a VAS pain score of the subject; inputting the multi-modal physiological data and the three-dimensional baseline vector into a personalized orthopedic rehabilitation model to output a knee joint function index prediction value and a personalized rehabilitation adjustment parameter; wherein the rehabilitation adjustment parameter comprises an electrical stimulation adjustment parameter; taking the knee joint function index prediction value, the cognitive load index, and the VAS pain score as a state space, and taking the electrical stimulation parameter as an action space, and dynamically adjusting the electrical stimulation parameter according to the rehabilitation adjustment parameter output by the personalized orthopedic rehabilitation model; monitoring the cognitive load index of the subject in real time, wherein the cognitive load index is calculated according to an EEG frequency band energy ratio; comparing the cognitive load index with a set attention distraction warning line; if the cognitive load index exceeds the warning line, reducing the electrical stimulation intensity; otherwise, activating a hippocampus wave stimulation mode. The application adjusts according to the real-time state of the subject, avoids blindness and subjectivity in the rehabilitation process, and improves the rehabilitation efficiency and success rate.

[0109] Those skilled in the art will appreciate that the modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and furthermore can be divided into multiple sub-modules or sub-units or sub-components. Any combination of all the features disclosed in the present specification (including the accompanying claims, abstract and drawings), and any method or apparatus so disclosed, can be taken in any combination, except that at least some of such features and / or processes or units are mutually exclusive, unless explicitly stated otherwise. Each feature disclosed in the present specification (including the accompanying claims, abstract and drawings) can be replaced by alternative features serving the same, equivalent or similar purpose, unless explicitly stated otherwise. Furthermore, the skilled person will appreciate that the combination of features of different embodiments implies that the features of the different embodiments are meant to be combined, unless explicitly stated otherwise. For example, in the claims below, any of the embodiments can be used in any combination. The application can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unitary claim, several of the devices mentioned in the embodiments can be implemented by means of one and the same hardware item. The steps of the above-described embodiments, unless explicitly stated otherwise, are not to be understood as having to be carried out in the order in which they are described.

Claims

1. An intelligent orthopedic rehabilitation adaptive assessment device based on multi-modal physiological feedback, characterized in that, The method comprises the following steps: A multi-modal data acquisition and baseline construction module is used to synchronously acquire multi-modal physiological data of a subject, including electromyography signals, three-dimensional motion trajectories of the knee joint, plantar pressure distribution, and cognitive load; a three-dimensional baseline vector is constructed according to the MMT muscle strength test, knee joint ROM measurement, and VAS pain score of the subject; wherein the cognitive load is quantified by monitoring the theta wave and alpha power ratio through the frontal lobe bipolar lead; An individualized rehabilitation model prediction and adjustment module is used to input the multi-modal physiological data and the three-dimensional baseline vector into an individualized orthopedic rehabilitation model, and output a knee function index prediction value and individualized rehabilitation adjustment parameters; wherein the rehabilitation adjustment parameters include electrical stimulation adjustment parameters; the individualized orthopedic rehabilitation model comprises a multi-modal data preprocessing module, a biomechanical feature extraction module, a neurophysiological feature extraction module, a multi-modal fusion module, and a prediction and optimization module; wherein the biomechanical feature extraction module comprises a GCN network layer, a GAT network layer, and a ChebNets network layer, which are used to convert the knee biomechanical model into a graph structure to extract deep biomechanical features; the neurophysiological feature extraction module comprises a TCN network layer and an SE Blocks network block, which are used to capture the time dependence of EEG signals and EMG signals; the prediction and optimization module comprises a high-level strategy layer, a low-level strategy layer, an Actor network layer, a Critic network layer, an A2C network layer, and a MAML independent meta-learning module, which are used to learn the optimal electrical stimulation strategy; An adaptive electrical stimulation parameter adjustment module is used to take the knee function index prediction value, cognitive load index, and VAS pain score as a state space, and take the electrical stimulation parameter as an action space, and dynamically adjust the electrical stimulation parameter according to the rehabilitation adjustment parameters output by the individualized orthopedic rehabilitation model; A cognitive load monitoring and hippocampal wave stimulation module is used to monitor the cognitive load index of the subject in real time, wherein the cognitive load index is calculated according to the EEG frequency band energy ratio; the cognitive load index is compared with a set attention distraction warning line; if the cognitive load index exceeds the warning line, the electrical stimulation intensity is reduced; otherwise, the hippocampal wave stimulation mode is activated.

2. The multi-modal physiological feedback based smart orthopedic rehabilitation adaptive assessment device as claimed in claim 1, wherein, The method for synchronously acquiring the multi-modal physiological data further comprises: The quadriceps muscle raw electromyography signals of the subject are acquired through a four-channel high-density surface electromyography electrode; The three-dimensional motion trajectories of the knee joint are calculated in real time through two IMU nodes on the thigh and the lower leg, wherein the three-dimensional motion trajectories of the knee joint include the flexion / extension angle, the internal / external rotation angle, and the adduction / abduction angle; The standing phase / swinging phase pressure center offset trajectory is captured through a point flexible pressure sensing insole, and a plantar mechanics distribution heat map is constructed.

3. The multi-modal physiological feedback based smart orthopedic rehabilitation adaptive assessment device as claimed in claim 1, wherein, Further comprising: The electrical stimulation intensity and the cognitive state and the pain level are dynamically decoupled and controlled according to a three-dimensional state space transfer equation of the cognitive load-motion coupling coefficient; wherein the three-dimensional state space transfer equation is: wherein, H is the knee kinematics Hamiltonian; θ is the joint angle vector; CLIM is the cognitive load modulation index; ΔP stim is the adjustment amount of the electrical stimulation parameter; K P is the proportionality coefficient for adjusting the amplitude of the electrical stimulation adjustment; ROM is the range of motion; θ i+1 is the (i+1)th joint angle; is the reference power spectrum integral in the frequency range [α,β]; is the actual power spectrum integral in the frequency range [α,β]; VAS is the pain score; n is the dimension of the joint angle vector; ω is the argument of the power spectral density function, representing the frequency.

4. The multi-modal physiological feedback based smart orthopedic rehabilitation adaptive assessment device according to claim 1, wherein, The cost function of the individualized orthopedic rehabilitation model is: wherein, L biomechanical (t) is a biomechanical integral term; L neurophysiological (t) is a neurophysiological integral term; L clinical (t) is a coupling term; θ pred (t) is a predicted knee joint motion trajectory; θ target (t) is a target knee joint motion trajectory; β is a biomechanical memory coefficient; τ is a time decay constant; H is a Hamiltonian; θ(s) is a knee joint motion trajectory at time s; θ α (t); θ α (t) is the power of the EEG signal in the alpha band; θ β (t) is the power of the EEG signal in the beta band; γ ref is the reference EEG band ratio; μ is the cumulative change weight of the EMG signal; EMG env (s) is the EMG signal envelope at time s; VAS(t) is the visual analog pain score at time t; ROM baseline is the baseline joint range of motion; CLI(s) is the cognitive load index at time s; CLI th is the threshold value of the cognitive load index; ΔP stim (t) is the amount of change in the electrical stimulation parameter at time t; K P is the proportional gain coefficient; CLIM(t) is the cognitive load modulation index at time t.

5. The multi-modal physiological feedback based smart orthopedic rehabilitation adaptive assessment device according to claim 1, wherein, The personalized orthopedic rehabilitation model further comprises an adaptive meta-learning module, which realizes rapid adaptation of patient-specific parameters through a MAML algorithm; In the initial training stage, general dynamic characteristics of the rehabilitation process are extracted according to cross-patient meta-learning; For a new patient, a secondary rapid adaptation method is activated, and the prediction error of the personalized orthopedic rehabilitation model is reduced to less than 60% of the baseline level through 3-5 iterations.

6. The multi-modal physiofeedback based smart orthopedic rehabilitation adaptive assessment device as claimed in claim 1, wherein, The GCN network layer, the GAT network layer and the ChebNets network layer extract and fuse node features and edge features in the knee biomechanical graph structure through spectral convolution, attention network layer and Chebyshev polynomial approximation network layer respectively, to represent the mechanical transmission and interaction in the knee movement process.

7. The multi-modal physiological feedback based smart orthopedic rehabilitation adaptive assessment device as claimed in claim 1 or 4, wherein, The high-level policy layer is trained according to the A2C algorithm to learn a long-term electrical stimulation strategy; The low-level policy layer rapidly adapts to individual differences according to the MAML algorithm; The Actor network layer and the Critic network layer are used to estimate the state value function and the policy function.

8. A computing device comprising: The processor, the memory, the communication interface and the communication bus complete communication among each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction makes the processor execute the operation corresponding to the intelligent orthopedic rehabilitation adaptive evaluation device based on multi-modal physiological feedback in any one of claims 1-7.

Images