Rehabilitation training method and system based on biofeedback

Abstract

The invention provides a rehabilitation training method and system based on biofeedback, and the method comprises the steps: carrying out the comprehensive physiological and motion function evaluation of a patient, so as to determine a rehabilitation target and initial state data; physiological signal data and motion data of a patient are collected in real time, wherein the physiological signal data comprise electromyogram, heart rate and electroencephalogram; and taking the DDPG model as a strategy network, dynamically adjusting training content, difficulty, frequency and duration through the DDPG model according to the physiological signal data, the exercise performance and the rehabilitation target of the patient, so as to maximize the rehabilitation progress of the patient, generate an optimal rehabilitation training strategy and carry out rehabilitation training. According to the method, the DDPG model serves as a strategy network, the complex relation among the physiological signal data, the exercise performance and the optimal rehabilitation training strategy of the patient is learned and mapped, and the strategy is dynamically adjusted according to the real-time state and demand of the patient, so that the effectiveness and adaptability of the rehabilitation training strategy are ensured, and the rehabilitation progress of the patient is maximized.

Description

Technical Field

[0001] The present invention relates to the technical field of rehabilitation training, and in particular to a rehabilitation training method and system based on biofeedback. Background Art

[0002] Rehabilitation training can cover a variety of aspects, from physical therapy, occupational therapy to speech therapy, aiming to promote the recovery or improvement of patients' impaired physical functions through a series of targeted training activities. Rehabilitation training is particularly important in scenarios such as nervous system injuries, musculoskeletal diseases, and postoperative recovery.

[0003] However, traditional rehabilitation training methods face many challenges. On the one hand, the experience of rehabilitation therapists and the self-perception of patients play a leading role in the training process, but this method often lacks scientific and objective data support. It is difficult for rehabilitation therapists to accurately judge the actual rehabilitation needs and progress of patients, resulting in greater subjectivity and blindness in the formulation and implementation of training strategies. On the other hand, there are significant differences in the physiological conditions and motor functions of patients. Rehabilitation training should be highly personalized to meet the specific needs of different patients. However, traditional methods often adopt a "one-size-fits-all" training model, which lacks sufficient consideration and response to individual differences among patients. Summary of the invention

[0004] Based on this, the purpose of the present invention is to propose a rehabilitation training method and system based on biofeedback to solve the above-mentioned problems.

[0005] A biofeedback-based rehabilitation training method according to the present invention comprises:

[0006] Conduct a comprehensive physiological and motor function assessment on the patient to determine rehabilitation goals and initial status data;

[0007] Through the biofeedback system and the motion capture system, the patient's physiological signal data and motion data are collected in real time, and the physiological signal data includes electromyography, heart rate and electroencephalogram;

[0008] Using the DDPG model as a strategy network, the DDPG model dynamically adjusts the training content, difficulty, frequency, and duration according to the patient's physiological signal data, exercise performance, and rehabilitation goals to maximize the patient's rehabilitation progress and generate the optimal rehabilitation training strategy;

[0009] Provide rehabilitation training to patients according to the optimal rehabilitation training strategy.

[0010] Furthermore, the DDPG model is used as a strategy network to dynamically adjust the training content, difficulty, frequency and duration according to the patient's physiological signal data, exercise performance and rehabilitation goals through the DDPG model to maximize the patient's rehabilitation progress and generate the optimal rehabilitation training strategy, including:

[0011] Collect the patient's physiological signal data, sports performance data, and rehabilitation goal information, and perform preprocessing to form a state vector for model training;

[0012] Construct a DDPG model that includes an Actor network and a Critic network. The Actor network is used to generate an action strategy for rehabilitation training based on the current state, and the Critic network is used to evaluate the value of the action strategy.

[0013] The DDPG model is trained based on the preprocessed data, and the model is continuously optimized through experience replay, target network update, and gradient descent.

[0014] After the training is completed, the optimal rehabilitation training strategy is generated based on the patient's current condition and through the Actor network, including adjustment suggestions for training content, difficulty, frequency and duration.

[0015] Furthermore, the construction of the DDPG model including the Actor network and the Critic network includes:

[0016] Encode the patient's physiological signals, motor performance, and rehabilitation goals into a state vector;

[0017] Define adjustments in training content, difficulty, frequency, and duration as a continuous action space;

[0018] Design a reward function to give positive or negative rewards based on the patient's recovery progress;

[0019] Construct the first deep neural network as an Actor, whose input is the state vector and output is the continuous action value;

[0020] Construct a second deep neural network as Critic, whose input is the state vector and action value, and output is the action-state value function Q(s, a).

[0021] Furthermore, after the training is completed, the optimal rehabilitation training strategy is generated according to the patient's current state through the Actor network, including suggestions for adjusting the training content, difficulty, frequency and duration, including:

[0022] Encode the patient's current state data into a state vector as input to the Actor network;

[0023] Load the trained Actor network model;

[0024] The encoded state vector is input into the Actor network, so that a continuous action value is output through the Actor network according to the input state vector, wherein the output action value represents the current optimal rehabilitation training strategy;

[0025] The action values ​​output by the Actor network are parsed into specific rehabilitation training strategies, including suggestions for adjusting training content, difficulty, frequency, and duration.

[0026] Furthermore, the DDPG model is trained based on the preprocessed data, and the model is continuously optimized through experience replay, target network update and gradient descent, including:

[0027] Initialize a target network for each Actor and Critic network to stabilize the training process;

[0028] Create an experience playback pool to store the experience gained by the patient during the rehabilitation training process, including physiological signals, motion data, training actions performed, and corresponding rehabilitation effects;

[0029] In each rehabilitation training time step, according to the patient's current physiological state and motor performance, as well as the training action suggestions generated by the Actor network, a training action is selected and applied to the training environment, and the next physiological state, motor performance and rehabilitation progress after the action is performed are obtained;

[0030] The current state, the selected training action, the next state, the rehabilitation progress and whether the rehabilitation goal has been completed are stored in the experience replay pool;

[0031] Randomly sample a batch of samples from the experience replay pool to train the Actor and Critic networks;

[0032] Take the weighted average of the old target network parameters and the new corresponding network parameters and assign them to the target network. For the Actor target network and the Critic target network, the update formulas are:

[0033] Actor target network: θμ′=τθμ+(1-τ)θμ′,

[0034] Critic target network: θQ′=τθQ+(1-τ)θQ′,

[0035] Among them, θμ′ is the parameter of the Actor target network, θμ is the parameter of the current Actor network, θQ′ is the parameter of the Critic target network, θQ is the parameter of the current Critic network, and τ is the step size of the soft update;

[0036] For the Critic network, the target Q value is calculated according to the patient's recovery progress and the maximum Q value of the next state. The square of the difference between the target Q value and the current Q value is calculated through the loss function of the Critic network, and the gradient descent algorithm is used to minimize the loss function to update the parameters of the Critic network.

[0037] For the Actor network, according to the Q-value gradient information provided by the Critic network, the policy gradient algorithm is used to update the parameters of the Actor network to generate better rehabilitation training action suggestions;

[0038] The above steps of experience replay, target network update and gradient descent optimization are repeated continuously until the DDPG model converges and a stable rehabilitation training strategy is generated.

[0039] Furthermore, in each rehabilitation training time step, according to the patient's current physiological state and motor performance, and the training action suggestions generated by the Actor network, a training action is selected and applied to the training environment, and the next physiological state, motor performance and rehabilitation progress after the action is performed are obtained, including:

[0040] In each rehabilitation training time step, based on the patient's current physiological state and motor performance, as well as the action candidate set generated by the Actor network, the upper confidence bound strategy is used to balance exploration and utilization, and the training action with the highest UCB value is selected as the execution action of the current time step;

[0041] The selected training actions are applied to the patient's rehabilitation training, and the execution effects are observed through the biofeedback system and motion capture system to update the experience replay pool and train the DDPG model.

[0042] Furthermore, in each rehabilitation training time step, according to the patient's current physiological state and motor performance, and the action candidate set generated by the Actor network, the upper confidence bound strategy is used to balance exploration and utilization, and the training action with the highest UCB value is selected as the execution action of the current time step, including:

[0043] Set an initial exploration rate ∈ to increase the exploratory nature of training movements in the early stages of rehabilitation training;

[0044] Set an initial confidence bound parameter c to control the proportion of exploration items in the UCB strategy;

[0045] In each rehabilitation training time step, the patient’s current physiological state and motor performance are obtained from the biofeedback system and motion capture system as state s;

[0046] According to the current state s, a training action suggestion μ(s) based on the current rehabilitation training strategy is generated through the Actor network;

[0047] For all actions a∈A, the UCB value of each training action is calculated by the Q value Q(s, a) and the confidence bound parameter c estimated by the Critic network, as well as the number of times the action is selected N(s, a). The calculation formula is:

[0048]

[0049] Where Q(s, a) is the Q value estimate of the Critic network for the state-action pair (s, a), t is the current time step, N(s, a) is the number of times the state-action pair (s, a) is visited, and c is the confidence bound parameter;

[0050] Randomly select a training action to explore with probability ∈;

[0051] Select the training action a with the highest UCB value with probability 1-τ * As the action to be executed at the current time step.

[0052] The present invention also provides a rehabilitation training system based on biofeedback, the system comprising:

[0053] Goal determination module: used to conduct a comprehensive physiological and motor function assessment of the patient to determine the rehabilitation goals and initial status data;

[0054] Acquisition module: used to collect the patient's physiological signal data and motion data in real time through the biofeedback system and motion capture system. The physiological signal data includes electromyography, heart rate and electroencephalogram;

[0055] Strategy generation module: It is used to use the DDPG model as a strategy network to dynamically adjust the training content, difficulty, frequency and duration through the DDPG model according to the patient's physiological signal data, exercise performance and rehabilitation goals, so as to maximize the patient's rehabilitation progress and generate the optimal rehabilitation training strategy;

[0056] Training module: used to provide rehabilitation training to patients based on the optimal rehabilitation training strategy.

[0057] In summary, the rehabilitation training method based on biofeedback of the present invention adopts the DDPG model as the strategy network to realize the intelligence and personalization of the rehabilitation training strategy. Through the combination of deep learning and strategy optimization, the DDPG model can learn and map the complex relationship between the patient's physiological signal data, motor performance and the optimal rehabilitation training strategy, and then dynamically adjust the training content, difficulty, frequency and duration according to the patient's real-time status and needs. This personalized and dynamically adjusted strategy capability ensures the effectiveness and adaptability of the rehabilitation training strategy, can maximize the patient's rehabilitation progress, and thus maximize the efficiency and effect of rehabilitation training. At the same time, the stability and robustness of the DDPG model also provide a strong guarantee for the continuity and reliability of rehabilitation training.

[0058] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description or will be learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0060] Figure 1 This is a flow chart of a rehabilitation training method based on biofeedback according to Embodiment 1 of the present invention;

[0061] Figure 2 This is a system block diagram of a rehabilitation training system based on biofeedback according to the second embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0063] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0065] Embodiment 1

[0066] See also Figure 1 The present invention proposes a rehabilitation training method based on biofeedback, which includes steps S101 to S104:

[0067] S101, conduct a comprehensive physiological and motor function assessment on the patient to determine the rehabilitation goals and initial status data.

[0068] It should be noted that before conducting rehabilitation training, first of all, a comprehensive assessment of the patient is required to determine the patient's rehabilitation goals and obtain the patient's initial status data. The assessment content may include the patient's physiological condition (such as muscle strength, joint range of motion, cardiopulmonary function, etc.) and motor function (such as balance, coordination, motor control, etc.). Through this assessment information, the patient's rehabilitation needs and potential can be objectively understood, so as to formulate feasible rehabilitation goals to ensure the pertinence and effectiveness of rehabilitation training, and avoid blindness and arbitrariness.

[0069] S102, collecting the patient's physiological signal data and motion data in real time through a biofeedback system and a motion capture system, wherein the physiological signal data includes electromyography, heart rate and electroencephalogram.

[0070] It should be noted that biofeedback systems (such as electromyography sensors, electrocardiogram sensors, electroencephalogram electrode monitoring equipment, etc.) can be used to collect physiological signal data of patients in real time, including electromyography (EMG), heart rate (HR) and electroencephalogram (EEG). Among them, electromyography sensors are usually placed on the surface of the patient's muscles to collect electrical signals generated when the muscles contract. These signals can reflect the activity and strength of the muscles. Electrocardiogram sensors can be used to monitor heart rate monitoring and can reflect the state of the patient's cardiovascular system in real time. Electroencephalogram monitoring can collect electrical activity signals of the brain through electrodes placed on the patient's scalp to evaluate the patient's nervous system state and cognitive function. These data provide an objective basis for evaluating the patient's physiological state and rehabilitation progress.

[0071] At the same time, the motion capture system (such as Xsens motion capture system, etc.) collects the patient's motion data in real time, such as joint angles, motion trajectories, motion speeds, etc. These data can be used to analyze the patient's sports performance, discover movement disorders and develop targeted training strategies.

[0072] S103, using the DDPG model as a strategy network, dynamically adjust the training content, difficulty, frequency and duration through the DDPG model according to the patient's physiological signal data, motor performance and rehabilitation goals, so as to maximize the patient's rehabilitation progress and generate the optimal rehabilitation training strategy.

[0073] It should be noted that the DDPG (Deep Deterministic Policy Gradient) model is a deep reinforcement learning algorithm that can automatically adjust the training strategy based on the patient's physiological signal data, exercise performance and rehabilitation goals. Through continuous learning and optimization, the DDPG model can find the training content, difficulty, frequency and duration that best suits the patient to maximize the patient's rehabilitation progress. This dynamically adjusted policy network not only improves the efficiency and effectiveness of rehabilitation training, but also enhances the personalization and pertinence of training.

[0074] S104, performing rehabilitation training on the patient according to the optimal rehabilitation training strategy.

[0075] It should be noted that after determining the optimal rehabilitation training strategy, the rehabilitation therapist or rehabilitation equipment will conduct rehabilitation training for the patient according to the strategy. During the training process, the patient's response and progress should be closely monitored, and the training plan should be adjusted in a timely manner to ensure the safety and effectiveness of the training. At the same time, the patient should be re-evaluated regularly so that the training strategy and goals can be adjusted according to the progress of rehabilitation.

[0076] Based on step S101 to step S104, the present invention adopts the DDPG model as a strategy network to realize the intelligence and personalization of rehabilitation training strategies. Through the combination of deep learning and strategy optimization, the DDPG model can learn and map the complex relationship between the patient's physiological signal data, motor performance and the optimal rehabilitation training strategy, and then dynamically adjust the training content, difficulty, frequency and duration according to the patient's real-time status and needs. This personalized and dynamically adjusted strategy capability ensures the effectiveness and adaptability of the rehabilitation training strategy, can maximize the patient's rehabilitation progress, and thus maximize the efficiency and effect of rehabilitation training. At the same time, the stability and robustness of the DDPG model also provide a strong guarantee for the continuity and reliability of rehabilitation training.

[0077] The following is a further detailed introduction to a rehabilitation training method based on biofeedback according to an embodiment of the present invention:

[0078] Further optionally, in step S103, the DDPG model is used as a strategy network to dynamically adjust the training content, difficulty, frequency and duration through the DDPG model according to the patient's physiological signal data, exercise performance and rehabilitation goals to maximize the patient's rehabilitation progress and generate the optimal rehabilitation training strategy, including:

[0079] Collect the patient's physiological signal data, sports performance data, and rehabilitation goal information, and perform preprocessing to form a state vector for model training;

[0080] Construct a DDPG model that includes an Actor network and a Critic network. The Actor network is used to generate an action strategy for rehabilitation training based on the current state, and the Critic network is used to evaluate the value of the action strategy.

[0081] The DDPG model is trained based on the preprocessed data, and the model is continuously optimized through experience replay, target network update, and gradient descent.

[0082] After the training is completed, the optimal rehabilitation training strategy is generated based on the patient's current condition and through the Actor network, including adjustment suggestions for training content, difficulty, frequency and duration.

[0083] Understandably, the rehabilitation training strategy generation method based on the DDPG model can dynamically adjust the training content, difficulty, frequency and duration according to the patient's physiological signal data, motor performance and rehabilitation goals to maximize rehabilitation progress.

[0084] By collecting the patient's physiological signals and sports performance data in real time and combining them with rehabilitation goals, a personalized rehabilitation strategy for each patient can be generated. This personalized strategy helps improve the rehabilitation effect because the physical conditions and rehabilitation needs of different patients are different. The DDPG model can dynamically adjust the training content, difficulty, frequency and duration according to the patient's real-time status. This dynamic adjustment can ensure that the training plan always matches the patient's current rehabilitation status and avoid overtraining or undertraining.

[0085] The Actor network is responsible for generating action strategies for rehabilitation training, while the Critic network evaluates the value of these strategies. This architecture helps to separate strategy optimization and value evaluation, thereby improving the efficiency and accuracy of the model.

[0086] Further optionally, the constructing of the DDPG model including the Actor network and the Critic network includes:

[0087] Encode the patient's physiological signals, motor performance, and rehabilitation goals into a state vector;

[0088] Define adjustments in training content, difficulty, frequency, and duration as a continuous action space;

[0089] Design a reward function to give positive or negative rewards based on the patient's recovery progress;

[0090] Construct the first deep neural network as an Actor, whose input is the state vector and output is the continuous action value;

[0091] Construct a second deep neural network as Critic, whose input is the state vector and action value, and output is the action-state value function Q(s, a).

[0092] It is understandable that in the process of building a DDPG model containing an Actor network and a Critic network, the patient's physiological signals, motor performance, and rehabilitation goals are encoded as a state vector to help the model better understand the patient's current state. The state vector needs to contain enough information to describe the patient's current state, but excessive dimensionality and complexity may increase the learning difficulty and computational cost of the model. Therefore, it is necessary to perform reasonable feature selection and dimensionality reduction on the state vector.

[0093] The adjustment of training content, difficulty, frequency and duration is defined as a continuous action space. By defining a continuous action space, the model can generate more delicate and accurate rehabilitation strategy adjustments, which is consistent with the actual needs of rehabilitation training, because the adjustments of rehabilitation training are often continuous and small. In a continuous action space, the model needs to effectively explore and utilize different action strategies, which requires the model to balance the relationship between exploration (trying new actions) and utilization (selecting known good actions) during training.

[0094] Giving positive or negative rewards based on the patient's recovery progress provides a clear optimization goal for the model, which directly affects the model's learning effect and the quality of the strategy. The reward function needs to accurately reflect the patient's recovery progress and give appropriate rewards or penalties. The setting of the reward function can be adjusted and optimized according to the specific rehabilitation scenario and patient.

[0095] The Actor network, as a policy network, is responsible for generating action strategies for rehabilitation training based on the current state. The Critic network, as a value network, is responsible for evaluating the value of the action strategy and providing improvement directions for the Actor network. This architecture combines the policy gradient and value function methods to improve the convergence speed and stability of the model.

[0096] Further optionally, after the training is completed, an optimal rehabilitation training strategy is generated according to the patient's current state through the Actor network, including suggestions for adjusting the training content, difficulty, frequency and duration, including:

[0097] Encode the patient's current state data into a state vector as input to the Actor network;

[0098] Load the trained Actor network model;

[0099] The encoded state vector is input into the Actor network, so that a continuous action value is output through the Actor network according to the input state vector, wherein the output action value represents the current optimal rehabilitation training strategy;

[0100] The action values ​​output by the Actor network are parsed into specific rehabilitation training strategies, including suggestions for adjusting training content, difficulty, frequency, and duration.

[0101] Understandably, the trained Actor network is used to generate the optimal rehabilitation training strategy according to the patient's current state. In the process, the patient's current state data is encoded into a state vector and used as the input of the Actor network. The Actor network outputs a continuous action value, which makes the adjustment of the rehabilitation training strategy more delicate and precise, and can meet the subtle changes in the patient's needs during the rehabilitation process. The encoded current state vector is input into the Actor network, and the Actor network outputs a continuous action value based on the input state vector. The action value represents the current optimal rehabilitation training strategy.

[0102] The action values ​​output by the Actor network are parsed into specific rehabilitation training strategies, including suggestions for adjusting the training content, difficulty, frequency, and duration. For example, difficulty can be achieved by changing the complexity, resistance, or duration of the training action, and frequency can be achieved by adjusting the number of repetitions or training intervals of the training action. By changing parameters such as training content, complexity, resistance, duration, number of repetitions, or training intervals of training actions, rehabilitation training strategies can be flexibly adjusted to suit the rehabilitation needs and progress of different patients.

[0103] Further optionally, the DDPG model is trained based on the preprocessed data, and the model is continuously optimized through experience replay, target network update and gradient descent, including:

[0104] Initialize a target network for each Actor and Critic network to stabilize the training process;

[0105] Create an experience playback pool to store the experience gained by the patient during the rehabilitation training process, including physiological signals, motion data, training actions performed, and corresponding rehabilitation effects;

[0106] In each rehabilitation training time step, according to the patient's current physiological state and motor performance, as well as the training action suggestions generated by the Actor network, a training action is selected and applied to the training environment, and the next physiological state, motor performance and rehabilitation progress after the action is performed are obtained;

[0107] Store information such as the current state, selected training action, next state, rehabilitation progress, and whether the rehabilitation goal has been achieved in the experience replay pool;

[0108] Randomly sample a batch of samples from the experience replay pool to train the Actor and Critic networks;

[0109] Take the weighted average of the old target network parameters and the new corresponding network parameters and assign them to the target network. For the Actor target network and the Critic target network, the update formulas are:

[0110] Actor target network: θμ′=τθμ+(1-τ)θμ′,

[0111] Critic target network: θQ′=τθQ+(1-τ)θQ′,

[0112] Among them, θμ′ is the parameter of the Actor target network, θμ is the parameter of the current Actor network, θQ′ is the parameter of the Critic target network, θQ is the parameter of the current Critic network, and τ is the step size of the soft update, which usually takes a value between 0 and 1. Smaller values ​​(such as 0.001 or 0.005) help stabilize training;

[0113] For the Critic network, the target Q value is calculated according to the patient's recovery progress and the maximum Q value of the next state. The square of the difference between the target Q value and the current Q value is calculated through the loss function of the Critic network, and the gradient descent algorithm is used to minimize the loss function to update the parameters of the Critic network.

[0114] For the Actor network, according to the Q-value gradient information provided by the Critic network, the policy gradient algorithm is used to update the parameters of the Actor network to generate better rehabilitation training action suggestions;

[0115] The above steps of experience replay, target network update and gradient descent optimization are repeated continuously until the DDPG model converges and a stable rehabilitation training strategy is generated.

[0116] Understandably, in the process of using the DDPG model to train and optimize rehabilitation training strategies, a target network is first initialized for each Actor and Critic network, and a soft update strategy (i.e., weighted average update) is used to stabilize the training process and reduce fluctuations and instability during training. Then, an experience replay pool is created to store the patient's interactive experience during rehabilitation training, so that the model can learn from past experience, improve sample utilization, and break the temporal correlation between samples, which is conducive to model convergence. By calculating the target Q value and minimizing the loss function using the gradient descent algorithm, the Critic network can learn how to evaluate the value of different rehabilitation training actions, thereby guiding the Actor network to generate better action suggestions. The policy gradient algorithm is used to update the parameters of the Actor network, so that the Actor network can directly learn how to generate training actions that can maximize long-term rehabilitation effects. Compared with traditional value-based methods, this method is more direct and efficient.

[0117] Further optionally, in each rehabilitation training time step, according to the patient's current physiological state and motor performance, and the training action suggestion generated by the Actor network, a training action is selected, applied to the training environment, and the next physiological state, motor performance and rehabilitation progress after the action is performed are obtained, including:

[0118] In each rehabilitation training time step, based on the patient's current physiological state and motor performance, as well as the action candidate set generated by the Actor network, the upper confidence bound strategy is used to balance exploration and utilization, and the training action with the highest UCB value is selected as the execution action of the current time step;

[0119] The selected training actions are applied to the patient's rehabilitation training, and the execution effects are observed through the biofeedback system and motion capture system to update the experience replay pool and train the DDPG model.

[0120] Understandably, based on the original DDPG model training, the UCB (upper confidence bound) strategy is introduced to find a balance between exploring new and potentially better training actions and using known, well-performing training actions, thereby avoiding the model from falling into a local optimal solution and improving the diversity and potential effects of rehabilitation training strategies. The UCB strategy is used to select training actions with the highest UCB value from the action candidate set generated by the Actor network, which means that when selecting an action, not only the expected value of the action itself is considered, but also the uncertainty of the action, thereby improving the accuracy and robustness of action selection.

[0121] The selected training movements are applied to the patient's rehabilitation training, and the execution effects are observed through the biofeedback system and motion capture system. This real-time feedback mechanism helps to update the experience replay pool and train the DDPG model in a timely manner, so that the model can adapt to the patient's rehabilitation progress and status changes more quickly.

[0122] When selecting training movements, the program comprehensively considers the patient's current physiological state, motor performance, and the action candidate set generated by the Actor network. This multi-dimensional information fusion helps to improve the targetedness and personalization of rehabilitation training strategies.

[0123] Further optionally, in each rehabilitation training time step, according to the patient's current physiological state and motor performance, and the action candidate set generated by the Actor network, an upper confidence bound strategy is used to balance exploration and utilization, and the training action with the highest UCB value is screened out as the execution action of the current time step, including:

[0124] Set an initial exploration rate ∈ to increase the exploratory nature of training movements in the early stages of rehabilitation training;

[0125] Set an initial confidence bound parameter c to control the proportion of exploration items in the UCB strategy;

[0126] In each rehabilitation training time step, the patient’s current physiological state and motor performance are obtained from the biofeedback system and motion capture system as state s;

[0127] According to the current state s, a training action suggestion μ(s) based on the current rehabilitation training strategy is generated through the Actor network;

[0128] For all optional actions a∈A, the UCB value of each training action is calculated by the Q value Q(s, a) and the confidence bound parameter c estimated by the Critic network, as well as the number of times the action is selected N(s, a). The calculation formula is:

[0129]

[0130] Where Q(s, a) is the Q value estimate of the Critic network for the state-action pair (s, a), t is the current time step, N(s, a) is the number of times the state-action pair (s, a) is visited, and c is the confidence bound parameter;

[0131] A training action is randomly selected with probability ∈ for exploration to increase the diversity of rehabilitation training strategies;

[0132] Select the training action a with the highest UCB value with probability 1-∈ * as the execution action of the current time step to utilize the existing rehabilitation training experience.

[0133] Understandably, the upper confidence bound strategy and the Actor-Critic architecture in the DDPG model are combined to select the optimal training action in each rehabilitation training time step. In the process, the initial exploration rate ∈ is set to increase the exploratory nature of the training action in the early stage of rehabilitation training. The UCB strategy is used to calculate the UCB value of each training action based on the Q value estimate, the confidence bound parameter c, and the number of times the action is selected N(s, a). This takes into account both the value and uncertainty of the action, achieving a balance between exploration and utilization. Training action recommendations are then generated based on the patient's current physiological state and motor performance (obtained through the biofeedback system and motion capture system). By calculating the UCB value of each training action and selecting the action with the highest UCB value as the execution action, the accuracy and robustness of action selection are improved.

[0134] Embodiment 2

[0135] See also Figure 2 The present invention proposes a rehabilitation training system based on biofeedback, the system comprising:

[0136] Goal determination module: used to conduct a comprehensive physiological and motor function assessment of the patient to determine the rehabilitation goals and initial status data;

[0137] Acquisition module: used to collect the patient's physiological signal data and motion data in real time through the biofeedback system and Xsens motion capture system. The physiological signal data includes electromyography, heart rate and electroencephalogram;

[0138] Strategy generation module: It is used to use the DDPG model as a strategy network to dynamically adjust the training content, difficulty, frequency and duration through the DDPG model according to the patient's physiological signal data, exercise performance and rehabilitation goals, so as to maximize the patient's rehabilitation progress and generate the optimal rehabilitation training strategy;

[0139] Training module: used to provide rehabilitation training to patients based on the optimal rehabilitation training strategy.

[0140] Further optionally, the strategy generation module is also used for:

[0141] Collect the patient's physiological signal data, sports performance data, and rehabilitation goal information, and perform preprocessing to form a state vector for model training;

[0142] Construct a DDPG model that includes an Actor network and a Critic network. The Actor network is used to generate an action strategy for rehabilitation training based on the current state, and the Critic network is used to evaluate the value of the action strategy.

[0143] The DDPG model is trained based on the preprocessed data, and the model is continuously optimized through experience replay, target network update, and gradient descent.

[0144] After the training is completed, the optimal rehabilitation training strategy is generated based on the patient's current condition and through the Actor network, including adjustment suggestions for training content, difficulty, frequency and duration.

[0145] Further optionally, the strategy generation module is also used for:

[0146] Encode the patient's physiological signals, motor performance, and rehabilitation goals into a state vector;

[0147] Define adjustments in training content, difficulty, frequency, and duration as a continuous action space;

[0148] Design a reward function to give positive or negative rewards based on the patient's recovery progress;

[0149] Construct the first deep neural network as an Actor, whose input is the state vector and output is the continuous action value;

[0150] Construct a second deep neural network as Critic, whose input is the state vector and action value, and output is the action-state value function Q(s, a).

[0151] Further optionally, the strategy generation module is also used for:

[0152] Encode the patient's current state data into a state vector as input to the Actor network;

[0153] Load the trained Actor network model;

[0154] The encoded state vector is input into the Actor network, so that a continuous action value is output through the Actor network according to the input state vector, wherein the output action value represents the current optimal rehabilitation training strategy;

[0155] The action values ​​output by the Actor network are parsed into specific rehabilitation training strategies, including suggestions for adjusting training content, difficulty, frequency, and duration.

[0156] Further optionally, the strategy generation module is also used for:

[0157] Initialize a target network for each Actor and Critic network to stabilize the training process;

[0158] Create an experience playback pool to store the experience gained by the patient during the rehabilitation training process, including physiological signals, motion data, training actions performed, and corresponding rehabilitation effects;

[0159] In each rehabilitation training time step, according to the patient's current physiological state and motor performance, as well as the training action suggestions generated by the Actor network, a training action is selected and applied to the training environment, and the next physiological state, motor performance and rehabilitation progress after the action is performed are obtained;

[0160] The current state, the selected training action, the next state, the rehabilitation progress and whether the rehabilitation goal has been completed are stored in the experience replay pool;

[0161] Randomly sample a batch of samples from the experience replay pool to train the Actor and Critic networks;

[0162] Take the weighted average of the old target network parameters and the new corresponding network parameters and assign them to the target network. For the Actor target network and the Critic target network, the update formulas are:

[0163] Actor target network: θμ′=τθμ+(1-τ)θμ′,

[0164] Critic target network: θQ′=τθQ+(1-τ)θQ′,

[0165] Among them, θμ′ is the parameter of the Actor target network, θμ is the parameter of the current Actor network, θQ′ is the parameter of the Critic target network, θQ is the parameter of the current Critic network, and τ is the step size of the soft update;

[0166] For the Critic network, the target Q value is calculated according to the patient's recovery progress and the maximum Q value of the next state. The square of the difference between the target Q value and the current Q value is calculated through the loss function of the Critic network, and the gradient descent algorithm is used to minimize the loss function to update the parameters of the Critic network.

[0167] For the Actor network, according to the Q-value gradient information provided by the Critic network, the policy gradient algorithm is used to update the parameters of the Actor network to generate better rehabilitation training action suggestions;

[0168] The above steps of experience replay, target network update and gradient descent optimization are repeated continuously until the DDPG model converges and a stable rehabilitation training strategy is generated.

[0169] Further optionally, the strategy generation module is also used for:

[0170] In each rehabilitation training time step, based on the patient's current physiological state and motor performance, as well as the action candidate set generated by the Actor network, the upper confidence bound strategy is used to balance exploration and utilization, and the training action with the highest UCB value is selected as the execution action of the current time step;

[0171] The selected training actions are applied to the patient's rehabilitation training, and the execution effects are observed through the biofeedback system and motion capture system to update the experience replay pool and train the DDPG model.

[0172] Further optionally, the strategy generation module is also used for:

[0173] Set an initial exploration rate ∈ to increase the exploratory nature of training movements in the early stages of rehabilitation training;

[0174] Set an initial confidence bound parameter c to control the proportion of exploration items in the UCB strategy;

[0175] In each rehabilitation training time step, the patient’s current physiological state and motor performance are obtained from the biofeedback system and motion capture system as state s;

[0176] According to the current state s, a training action suggestion μ(s) based on the current rehabilitation training strategy is generated through the Actor network;

[0177] For all actions a∈A, the UCB value of each training action is calculated by the Q value Q(s, a) and the confidence bound parameter c estimated by the Critic network, as well as the number of times the action is selected N(s, a). The calculation formula is:

[0178]

[0179] Where Q(s, a) is the Q value estimate of the Critic network for the state-action pair (s, a), t is the current time step, N(s, a) is the number of times the state-action pair (s, a) is visited, and c is the confidence bound parameter;

[0180] Randomly select a training action to explore with probability ∈;

[0181] Select the training action a with the highest UCB value with probability 1-∈ * As the action to be executed at the current time step.

[0182] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A rehabilitation training method based on biofeedback, characterized in that: The method comprises: Conduct a comprehensive physiological and motor function assessment on the patient to determine rehabilitation goals and initial status data; Through the biofeedback system and the motion capture system, the patient's physiological signal data and motion data are collected in real time, and the physiological signal data includes electromyography, heart rate and electroencephalogram; Using the DDPG model as a strategy network, the DDPG model dynamically adjusts the training content, difficulty, frequency, and duration according to the patient's physiological signal data, exercise performance, and rehabilitation goals to maximize the patient's rehabilitation progress and generate the optimal rehabilitation training strategy; Provide rehabilitation training to patients according to the optimal rehabilitation training strategy.

2. The rehabilitation training method based on biofeedback according to claim 1, characterized in that: The DDPG model is used as a strategy network to dynamically adjust the training content, difficulty, frequency and duration according to the patient's physiological signal data, exercise performance and rehabilitation goals through the DDPG model to maximize the patient's rehabilitation progress and generate the optimal rehabilitation training strategy, including: Collect the patient's physiological signal data, sports performance data, and rehabilitation goal information, and perform preprocessing to form a state vector for model training; Construct a DDPG model that includes an Actor network and a Critic network. The Actor network is used to generate an action strategy for rehabilitation training based on the current state, and the Critic network is used to evaluate the value of the action strategy. The DDPG model is trained based on the preprocessed data, and the model is continuously optimized through experience replay, target network update, and gradient descent. After the training is completed, the optimal rehabilitation training strategy is generated based on the patient's current condition and through the Actor network, including adjustment suggestions for training content, difficulty, frequency and duration.

3. The biofeedback-based rehabilitation training method according to claim 2, characterized in that: The construction of the DDPG model containing the Actor network and the Critic network includes: Encode the patient's physiological signals, motor performance, and rehabilitation goals into a state vector; Define adjustments in training content, difficulty, frequency, and duration as a continuous action space; Design a reward function that gives positive or negative rewards based on the patient’s recovery progress; Construct the first deep neural network as an Actor, whose input is the state vector and output is the continuous action value; Construct a second deep neural network as Critic, whose input is the state vector and action value, and output is the action-state value function Q(s, a).

4. The biofeedback-based rehabilitation training method according to claim 2, characterized in that: After the training is completed, the optimal rehabilitation training strategy is generated according to the patient's current state through the Actor network, including suggestions for adjusting the training content, difficulty, frequency and duration, including: Encode the patient's current state data into a state vector as input to the Actor network; Load the trained Actor network model; The encoded state vector is input into the Actor network, so that a continuous action value is output through the Actor network according to the input state vector, wherein the output action value represents the current optimal rehabilitation training strategy; The action values ​​output by the Actor network are parsed into specific rehabilitation training strategies, including suggestions for adjusting training content, difficulty, frequency, and duration.

5. The biofeedback-based rehabilitation training method according to claim 2, characterized in that: The DDPG model is trained based on the preprocessed data, and the model is continuously optimized through experience replay, target network update and gradient descent, including: Initialize a target network for each Actor and Critic network to stabilize the training process; Create an experience playback pool to store the experience gained by the patient during the rehabilitation training process, including physiological signals, motion data, training actions performed, and corresponding rehabilitation effects; In each rehabilitation training time step, according to the patient's current physiological state and motor performance, as well as the training action suggestions generated by the Actor network, a training action is selected and applied to the training environment, and the next physiological state, motor performance and rehabilitation progress after the action is performed are obtained; The current state, the selected training action, the next state, the rehabilitation progress and whether the rehabilitation goal has been completed are stored in the experience replay pool; Randomly sample a batch of samples from the experience replay pool to train the Actor and Critic networks; Take the weighted average of the old target network parameters and the new corresponding network parameters and assign them to the target network. For the Actor target network and the Critic target network, the update formulas are: Actor target network: θμ′=τθμ+(1-τ)θμ′, Critic target network: θQ′=τθQ+(1-τ)θQ′, Among them, θμ′ is the parameter of the Actor target network, θμ is the parameter of the current Actor network, θQ′ is the parameter of the Critic target network, θQ is the parameter of the current Critic network, and τ is the step size of the soft update; For the Critic network, the target Q value is calculated according to the patient's recovery progress and the maximum Q value of the next state. The square of the difference between the target Q value and the current Q value is calculated through the loss function of the Critic network, and the gradient descent algorithm is used to minimize the loss function to update the parameters of the Critic network. For the Actor network, according to the Q-value gradient information provided by the Critic network, the policy gradient algorithm is used to update the parameters of the Actor network to generate better rehabilitation training action suggestions; The above steps of experience replay, target network update and gradient descent optimization are repeated continuously until the DDPG model converges and a stable rehabilitation training strategy is generated.

6. The rehabilitation training method based on biofeedback according to claim 5, characterized in that: In each rehabilitation training time step, according to the patient's current physiological state and motor performance, and the training action suggestions generated by the Actor network, a training action is selected and applied to the training environment, and the next physiological state, motor performance and rehabilitation progress after the action is performed are obtained, including: In each rehabilitation training time step, based on the patient's current physiological state and motor performance, as well as the action candidate set generated by the Actor network, the upper confidence bound strategy is used to balance exploration and utilization, and the training action with the highest UCB value is selected as the execution action of the current time step; The selected training actions are applied to the patient's rehabilitation training, and the execution effects are observed through the biofeedback system and motion capture system to update the experience replay pool and train the DDPG model.

7. The biofeedback-based rehabilitation training method according to claim 6, characterized in that: In each rehabilitation training time step, according to the patient's current physiological state and motor performance, and the action candidate set generated by the Actor network, the upper confidence bound strategy is used to balance exploration and utilization, and the training action with the highest UCB value is selected as the execution action of the current time step, including: Set an initial exploration rate ∈ to increase the exploratory nature of training movements in the early stages of rehabilitation training; Set an initial confidence bound parameter c to control the proportion of exploration items in the UCB strategy; In each rehabilitation training time step, the patient’s current physiological state and motor performance are obtained from the biofeedback system and motion capture system as state s; According to the current state s, a training action suggestion μ(s) based on the current rehabilitation training strategy is generated through the Actor network; For all actions a∈A, the UCB value of each training action is calculated by the Q value Q(s, a) and the confidence bound parameter c estimated by the Critic network, as well as the number of times the action is selected N(s, a). The calculation formula is: Where Q(s, a) is the Q value estimate of the Critic network for the state-action pair (s, a), t is the current time step, N(s, a) is the number of times the state-action pair (s, a) is visited, and c is the confidence bound parameter; Randomly select a training action to explore with probability ∈; Select the training action a with the highest UCB value with probability 1-∈ * As the action to be executed at the current time step.

8. A rehabilitation training system based on biofeedback, characterized in that: The system comprises: Goal determination module: used to conduct a comprehensive physiological and motor function assessment of the patient to determine the rehabilitation goals and initial status data; Acquisition module: used to collect the patient's physiological signal data and motion data in real time through the biofeedback system and motion capture system. The physiological signal data includes electromyography, heart rate and electroencephalogram; Strategy generation module: It is used to use the DDPG model as a strategy network to dynamically adjust the training content, difficulty, frequency and duration through the DDPG model according to the patient's physiological signal data, exercise performance and rehabilitation goals, so as to maximize the patient's rehabilitation progress and generate the optimal rehabilitation training strategy; Training module: used to provide rehabilitation training to patients based on the optimal rehabilitation training strategy.

Images