A human-computer interactive control method for flexible lower limb exoskeleton

Abstract

The invention provides a human-machine interactive control method for a flexible lower limb exoskeleton, comprising: an impedance control algorithm, a reinforcement learning algorithm, an adaptive control algorithm and a nonlinear disturbance observer; a mass, stiffness and damping model is established by the impedance control algorithm and a joint reference trajectory of a user is solved, thereby revealing a mathematical relationship between the joint interaction torque and the motion trajectory of a human-machine coupling system; an impedance model parameter is optimized by the reinforcement learning algorithm under the condition that the parameters of a robot and a human body dynamics model are limited, with the goal of minimizing muscle activation and joint stiffness; an adaptive control algorithm is used to minimize the tracking error between the actual trajectory of the exoskeleton system and the output trajectory of the impedance model, and a nonlinear disturbance observer is introduced to compensate for disturbances caused by external factors such as material friction, estimation error, and tight coupling of a model, thereby achieving the best assisted walking effect.

Description

Technical Field

[0001] The present invention relates to the technical field of exoskeleton suits, and in particular to a human-computer interaction control method for a flexible lower limb exoskeleton. Background Art

[0002] In recent years, with the rapid economic development and the aging of the population, the number of people with limb disabilities caused by cardiovascular and cerebrovascular diseases and accidental injuries has increased year by year. Lower limb motor dysfunction such as hemiplegia and paraplegia caused by stroke, spinal cord injury and other causes is the main cause of limb disability in adults. This type of disabled people not only brings pain to the patients themselves due to limb motor dysfunction, but also seriously affects their daily life and work.

[0003] Studies have shown that for patients with lower limb motor dysfunction such as hemiplegia and paraplegia, early intervention in rehabilitation treatment will greatly improve the subsequent rehabilitation effect. After rehabilitation training, patients can often complete walking, climbing and other movements independently, but often need auxiliary support from power-assisted walking devices. Lower limb exoskeleton robots are a type of new auxiliary equipment that can quantify, prolong and standardize power-assisted walking. It can provide effective lower limb motor function assistance, promote the remodeling of damaged nerve function in patients, quantitatively evaluate the rehabilitation process and feedback the rehabilitation progress. Compared with traditional power-assisted walking devices, lower limb exoskeleton robots have the characteristics of long-term assistance, flexible and controllable force, and high control accuracy. They can promote the active control of exoskeletons for different groups of people with lower limb motor dysfunction, improve safety, patient participation and compliance. However, due to the large uncertainty and strong nonlinearity of the flexible materials used in flexible lower limb exoskeletons, as well as the changes in human performance during the power-assisted process, it is difficult to model the human-machine coupling dynamics, and the parameter information of the impedance model is unclear, which cannot guarantee accurate control and effective assistance. Therefore, the control of flexible exoskeleton robots is a great challenge.

[0004] At present, there is a lower limb exoskeleton resistance robot control method (publication number: CN117565025A) that can assist the lower limb knee and hip joints in daily walking. The control method outputs a corresponding power assistance curve according to the user's gait type, but lacks the wearer's intention perception, and the control method does not consider the controller compensation problem when there is external interference; for example, a lower limb exoskeleton robot control method for different rehabilitation stages (publication number: CN117180061A) can judge the wearer's rehabilitation stage according to the electromyographic signal measurement data, and design different rehabilitation training models, but the control method is difficult to solve the problem of individual differences among patients, and the database bloat caused by frequent data collection; a lower limb rehabilitation robot control method based on the TD3 algorithm (publication number: CN117688276A) mainly solves the problem of individual difference adaptation between the robot and the patient and the robot's adaptability challenges in different environments, but the controller compensation problem caused by external interference has not been solved, and unknown interference and sudden movement of the patient are prone to secondary injuries. For the specific content of the flexible exoskeleton robot system, please refer to the content of the invention patent application with the patent name "A variable stiffness twist rope drive device for flexible lower limb exoskeleton", publication number: ZL202220690601.X, and the paper "A Variable Stiffness Exosuitfor Walking Assistance", doi: 10.1088 / 1742-6596 / 2402 / 1 / 012033. Summary of the invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a human-machine interactive control method for a flexible lower limb exoskeleton in view of the deficiencies of the prior art, so as to achieve accurate trajectory tracking and ensure the accuracy and effectiveness of rehabilitation training. The present invention comprises the following steps:

[0006] Step 1, establishing an impedance model through an impedance control algorithm, wherein the impedance control algorithm takes into account the impedance characteristics of human joints, establishes an impedance model and solves the user's joint reference trajectory, thereby describing the mathematical relationship between the joint interaction torque and the motion trajectory of the human-machine coupling system;

[0007] Step 2: Optimize stiffness and damping parameters under the condition of limited parameters of robot and human dynamics model through reinforcement learning algorithm (reinforcement learning algorithm is a machine learning algorithm that aims to learn how to make optimal decisions in different situations through the interaction between the agent and the environment. It includes the following concepts: agent, environment, state, action, reward function, value function and strategy, which is the existing technology. In different cases, the definitions of the above concepts are different. Reinforcement learning algorithm is used for specific problems, and different reward functions, states and actions are designed to achieve the best decision. The goal is to minimize muscle activation and joint stiffness.

[0008] Step 3, through the adaptive control algorithm, to minimize the tracking error between the actual trajectory of the exoskeleton system and the output trajectory of the impedance model, and introduce a nonlinear disturbance observer to compensate for the disturbance caused by external factors to achieve the best assisted walking effect.

[0009] Step 1 includes: establishing a human-machine coupling dynamics model, the formula is:

[0010] ,

[0011] Where q is the user's actual joint angle, is the user's actual joint angular velocity, is the user's joint angular acceleration, is the inertia matrix, are the centrifugal and Coriolis force equations, is the gravity torque, D1 is the interference caused by external factors, is the assisting torque of the exoskeleton system, is the output torque of the joint of the user's damaged limb; the external factors include material friction, estimation error and model tight coupling;

[0012] Considering the impedance characteristics of human joints, the impedance model including joint torque and angle is expressed as:

[0013] ,

[0014] in, is the expected trajectory angle, i.e., the position of the motion trajectory of the healthy limb; is the reference trajectory angle, i.e. the output trajectory position of the impedance model, and represent the corresponding desired speed and reference speed respectively, and represent the desired damping and desired stiffness respectively.

[0015] In step 2, the reinforcement learning algorithm includes the current state s of the robot and the person, the action of the robot , reward function r;

[0016] The current state s of the robot and the person is expressed as:

[0017] ,

[0018] Among them, K represents the joint stiffness of the damaged limb of the human body;

[0019] Actions of the robot It is expressed as:

[0020] ,

[0021] The reward function r is expressed as:

[0022] ,

[0023] in, , , is the weight coefficient, T represents the matrix transpose, Represents the expected joint stiffness. The reward function consists of three parts. The first part is to make the user's actual joint angle q and the expected joint angle The second part is to make the user's actual joint speed and the expected joint velocity The third part is to make the difference between the joint stiffness of the damaged limb of the hemiplegic patient and the expected joint stiffness as small as possible.

[0024] The learning algorithm is applicable to problems in continuous action space. Through experience replay, target network and noise exploration, the value of the reward function is maximized to obtain the optimal impedance parameters.

[0025] Among them, experience replay means that the agent stores the obtained experience data into the experience pool (in reinforcement learning, the agent is a core concept. The agent is an entity that learns the optimal strategy by interacting with the environment. It can be understood that the agent is a learning and decision-making system. It interacts with the environment, continuously obtains information, and takes actions based on the information. In the present invention, the environment is a person, and the information obtained is the person's electromyographic signal and knee joint angle signal). When updating the network, it is adopted in batches; the target network refers to a set of target action networks and target critic networks used to estimate the target in addition to the action network and the critic network. When updating the target network, in order to avoid too fast parameter updates, a soft update method is adopted; noise exploration means adding noise to the output action during the training phase, so that the agent has exploration capabilities.

[0026] In step 2, the following mathematical model for estimating muscle activation and joint stiffness is established:

[0027] ,

[0028] ,

[0029] in, is the joint stiffness of the knee joint at time t, agonist refers to the stimulating muscle during knee joint movement, and antagonist refers to the antagonistic muscle during knee joint movement. is the preprocessed electromyographic signal of the i-th muscle at time t, is the muscle activation of the i-th muscle at time t, n is the total number of muscles, e is a natural constant; A is the nonlinear shape factor, and is the weight coefficient, is the joint torque of the knee joint at time t, expressed as:

[0030] ,

[0031] in, For muscle strength, For the muscle arm.

[0032] In step 2, an adaptive parameter is introduced Adjust the value of the Q function in reinforcement learning:

[0033] ,

[0034] in, Represents the total value of the current state, Indicates that an action is performed in state s The immediate value of Indicates the next state Perform a new action Total value of

[0035] In order to adapt to rewards of different magnitudes, the regularized adaptive parameters are calculated using the following formula: :

[0036] ,

[0037] in, is the instantaneous value at time t, Represents the maximum value of instant value in all time steps;

[0038] The Q function in the reinforcement learning algorithm is calculated using the following formula :

[0039] ,

[0040] Where Q1 and Q2 are two independent Q networks; Indicates that the Q network Q1 performs an action in state s The action-value function of It means that the Q network Q2 performs action in state s The action-value function of

[0041] Use exploration noise to explore new actions. The action with exploration noise added is expressed as:

[0042] ,

[0043] in, To add the action after exploring the noise, is a normal distribution, is a random parameter, and x is the exploration noise;

[0044] The reinforcement learning algorithm uses the Q function to minimize the parameters corresponding to the Q network Q1 And the parameters corresponding to the Q network Q2 The Bellman error, that is, the loss function:

[0045] ,

[0046] ,

[0047] in, For random parameters The Bellman error, For random parameters The Bellman error of is in state , action s, random parameters The function of the Q network Q1 under ; is in state , action s, random parameters The function of the Q network Q2 under ; D is the experience replay pool, which is the collected state transition s, 、r、 A collection of; It represents the expected operation on the state s sampled from the experience replay pool D. The expectation refers to the weighted average of all possible values ​​of the random variable, where the weight is the probability of each value. The intermediate parameter z is defined as:

[0048] ,

[0049] in, is the discount factor;

[0050] The reinforcement learning algorithm described here updates the policy using the following function to maximize:

[0051] ,

[0052] The reinforcement learning algorithm uses soft update to update network parameters to reduce the pre-training error:

[0053] ,

[0054] ,

[0055] in, represents the parameters of the next Q network, are the parameters of the next policy network, is the soft update rate.

[0056] In step 3, the adaptive control algorithm introduces a virtual control of the trajectory error , in order to reduce the tracking error between the actual trajectory of the exoskeleton system and the output trajectory of the impedance model, expressed as:

[0057] ,

[0058] ,

[0059] in, and is the tracking error, The derivative of and The derivative of Satisfies the following formula:

[0060] ,

[0061] ,

[0062] in Indicates virtual control The rate of change.

[0063] In step 3, the external factors satisfy the limited energy:

[0064] ,

[0065] in, is an unknown positive number, the value of which is determined by the magnitude of the interference caused by gravity, material friction, and tight coupling of the model, and is obtained through theoretical calculation and parameter estimation. is the rate of change of the external disturbance;

[0066] Observation error after the introduction of nonlinear observer for:

[0067] ,

[0068] in, is a nonlinear function, the observation error The rate of change is:

[0069] ,

[0070] Among them, the intermediate parameters ;

[0071] The nonlinear disturbance observer is used to estimate the nonlinear disturbance in the system. The estimated value is:

[0072] ,

[0073] in, It refers to the estimate of the error rate of change by the nonlinear disturbance observer. Refers to the estimated value of the error by the nonlinear disturbance observer, and then the estimated value of the disturbance D1 caused by external factors is obtained and error :

[0074] ,

[0075] .

[0076] Step 3 also includes: obtaining the derivative of the disturbance error estimate :

[0077] .

[0078] Step 3 also includes: the adaptive control algorithm based on the nonlinear disturbance observer is expressed as:

[0079] ,

[0080] in, is a gain constant.

[0081] The present invention also provides a storage medium storing a computer program or instruction. When the computer program or instruction is executed, the human-computer interaction control method for a flexible lower limb exoskeleton is implemented.

[0082] The method of the present invention is applied to a flexible lower limb exoskeleton robot, and the control method includes a human-machine interactive control method, and the human-machine interactive control method includes: an impedance control algorithm, a reinforcement learning algorithm, an adaptive control algorithm and a nonlinear disturbance observer; the impedance control algorithm takes into account the impedance characteristics of human joints, establishes a mass-stiffness-damping model and solves the user's joint reference trajectory, and reveals the mathematical relationship between the joint interaction torque and the motion trajectory of the human-machine coupling system; the reinforcement learning algorithm optimizes the impedance model parameters under the condition that the robot and human body dynamics model parameters are limited, and its goal is to minimize muscle activation and joint stiffness; the adaptive control algorithm minimizes the tracking error between the actual trajectory of the exoskeleton system and the output trajectory of the impedance model, and introduces a nonlinear disturbance observer to compensate for the disturbance caused by external factors such as material friction, estimation error, model tight coupling, etc., so as to achieve the best assisted walking effect.

[0083] Compared with the prior art, the present invention has the following beneficial effects:

[0084] 1. This control method comprehensively considers the physiological parameters of the user's damaged limbs and healthy limbs, and adjusts the output torque of the exoskeleton system according to the user's specific gait. Compared with traditional control methods, this method optimizes impedance parameters based on reinforcement learning to prevent users from falling and getting injured due to inappropriate auxiliary output.

[0085] 2. It is difficult to establish a human-machine coupling model for a flexible exoskeleton system. The mathematical model established by the impedance control can be optimized according to the user status and achieve accurate trajectory tracking without knowing the human-machine model information.

[0086] 3. The control method is applicable to any desired trajectory. Users with different degrees of limb impairments or walking assistance tasks of different difficulty can use this control method, and can achieve good trajectory tracking and reduce the burden on users.

[0087] 4. The adaptive control algorithm introduces a nonlinear disturbance observer, which can compensate for the impact of external disturbances with finite energy. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0089] Figure 1 It is a schematic block diagram of the method of the present invention.

[0090] Figure 2 This is a schematic diagram of a flexible exoskeleton robot system for knee joint assistance.

[0091] Figure 3It is a schematic diagram of the parameter updating process. DETAILED DESCRIPTION

[0092] like Figure 1 As shown, this embodiment discloses a control method for a flexible lower limb exoskeleton, and the specific implementation method is as follows: an embodiment is developed for a flexible exoskeleton robot system assisted by a knee joint, such as Figure 2 As shown in the figure, the system consists of nylon straps, flexible exoskeleton, twist rope, servo motor, Bowden cable and other components. The flexible exoskeleton system weighs 5.2kg. The system uses posture sensors and electromyography sensors ( Figure 2 The user's joint angles and electromyographic signals (not shown) are collected, and the pre-processed electromyographic signals and joint angles are fed back to the controller. The controller calculates the corresponding auxiliary torque and compensates for the interference error through the adaptive control method based on reinforcement learning of the flexible lower limb exoskeleton. Then, the servo motor generates the corresponding auxiliary torque, and the torque is applied to the user's knee joint through the torsion rope and Bowden cable to assist the user in completing gait training. Considering the close physical interaction between the flexible exoskeleton robot and the user's limbs, a human-machine coupling dynamics model is established, and the formula is:

[0093] ,

[0094] Where q is the user's actual joint angle, is the user's actual joint angular velocity, is the user's joint angular acceleration, is the inertia matrix, are the centrifugal and Coriolis force equations, is the gravitational torque, The torque generated by the nylon strap and friction, is the assisting torque of the exoskeleton system, is the output torque of the joint of the user's impaired limb;

[0095] Due to the tight coupling between the flexible structure of the exoskeleton and the human-machine system, it is difficult to determine the human-machine dynamic coupling model, and many parameters in the dynamic model cannot be determined. Therefore, considering the mass, spring, and damping characteristics of the human joints, the mathematical model of the joint torque and angle is expressed as:

[0096] ,

[0097] in, is the expected trajectory angle, i.e., the position of the motion trajectory of the healthy limb; is the reference trajectory angle, i.e. the output trajectory position of the impedance model, and represent the corresponding desired speed and reference speed respectively, and represent the desired damping and desired stiffness respectively.

[0098] The flexible exoskeleton is worn on the user's damaged limbs, and the physiological signals (millivolt signals, about 0-5mV) of the user's damaged and healthy limbs are collected through electromyographic sensors. After filtering, denoising, rectification and calculation, the corresponding joint torque is obtained. The joint torque of a normal person is about 20-50Nm. The peak torque will change based on different stages. The corresponding calculation formula is as follows:

[0099] ,

[0100] ,

[0101] in, is the joint stiffness of the knee joint at time t, agonist refers to the stimulating muscle during knee joint movement, and antagonist refers to the antagonistic muscle during knee joint movement. is the preprocessed electromyographic signal of the i-th muscle at time t, is the muscle activation of the i-th muscle at time t, n is the total number of muscles, e is a natural constant; A is the nonlinear shape factor, and is the weight coefficient, is the joint torque of the knee joint at time t. The stiffness of the knee joint of a normal person during walking is 50~150N·mm / deg. This value will change with factors such as gait cycle, walking speed, terrain, individual weight and muscle strength, and is expressed as:

[0102] ,

[0103] in, For muscle strength, For the muscle arm.

[0104] For hemiplegic patients, the electromyographic state of the healthy limb can be used as a reference to stimulate the impaired limb for mirror training. The motion trajectory of the healthy limb and the joint stiffness are used as the desired motion trajectory of the controller. The reinforcement learning algorithm is used to minimize human effort and optimize the parameters of the impedance model. Therefore, the state s of the reinforcement learning is expressed as:

[0105] ,

[0106] Among them, K represents the joint stiffness of the damaged limb of the human body;

[0107] Robot's movements It is expressed as:

[0108] ,

[0109] The reward function r is expressed as:

[0110] ,

[0111] in, , , is the weight coefficient, T represents the matrix transpose, Represents the expected joint stiffness. The reward function consists of three parts. The first part is to make the user's actual joint angle q and the expected joint angle The second part is to make the user's actual joint speed and the expected joint velocity The third part is to make the difference between the joint stiffness of the damaged limb of the hemiplegic patient and the expected joint stiffness as small as possible.

[0112] The value of the reward function is affected by the weight coefficients of each reward item. After about 100 rounds of algorithm iteration, the reward function reaches its maximum value and finally converges.

[0113] The reinforcement learning algorithm should be applicable to problems in continuous action space, characterized in that the value of the reward function is maximized through experience replay, target network and noise exploration technology to obtain the optimal impedance parameters.

[0114] Among them, experience replay means that the agent stores the obtained experience data in the experience pool and adopts it in batches when updating the network. The target network refers to the use of a set of target action networks and target critic networks for estimating targets in addition to the action network and the critic network. When updating the target network, in order to avoid too fast parameter updates, a soft update method is used. Noise exploration means adding noise to the output actions during the training phase, so that the agent has a certain exploration ability.

[0115] Based on the double-delayed deep deterministic policy gradient (TD3) algorithm, the specific implementation method of reinforcement learning is introduced in detail. First, random parameters are used , , Initialize the critic network and the target network, and then introduce an adaptive parameter to adjust the value of the Q function in reinforcement learning to avoid overestimation and underestimation:

[0116] ,

[0117] in, Represents the total value of the current state, Indicates that an action is performed in state s The immediate value of Indicates the next state Perform a new action Total value of

[0118] In order to adapt to rewards of different magnitudes, the regularized adaptive parameters are calculated using the following formula: :

[0119] ,

[0120] in, is the instantaneous value at time t, Represents the maximum value of instant value in all time steps;

[0121] The Q function in the reinforcement learning algorithm is calculated using the following formula :

[0122] ,

[0123] Q1 and Q2 are two independent Q networks; in the reinforcement learning algorithm, each network has its own set of parameters. Figure 3 In and ,in and Represent the parameters of Q1 and Q2 respectively. In the TD3 algorithm, Q1 and Q2 are used to estimate the Q value obtained for a given state and action. Specifically, given the current state and actions After that, the output values ​​of the two Q networks are and Then, according to the loss function, the error between the Q value calculated for the current state and the target Q value is calculated. Indicates that the Q network Q1 performs an action in state s The action-value function of It means that the Q network Q2 performs action in state s The action-value function of

[0124] In order to solve the impact of noise on the results during the learning process, the learning algorithm uses exploration noise to explore new actions and thus better explore new strategies. The action with exploration noise added is expressed as:

[0125] ,

[0126] in, To add the action after exploring the noise, is a normal distribution, is a random parameter, and x is the exploration noise;

[0127] The reinforcement learning algorithm uses the Q function to minimize the parameters corresponding to the Q network Q1 And the parameters corresponding to the Q network Q2 The Bellman error, that is, the loss function:

[0128] ,

[0129] ,

[0130] in, For random parameters The Bellman error, For random parameters The Bellman error of is in state , action s, random parameters The function of the Q network Q1 under ; is in state , action s, random parameters The function of the Q network Q2 under ; D is the experience replay pool, which is the collected state transition s, 、r、 A collection of; It represents the expected operation on the state s sampled from the experience replay pool D. The expectation refers to the weighted average of all possible values ​​of the random variable, where the weight is the probability of each value. The intermediate parameter z is defined as:

[0131] ,

[0132] in, is the discount factor;

[0133] The reinforcement learning algorithm described here updates the policy using the following function to maximize:

[0134] ,

[0135] The reinforcement learning algorithm uses soft update to update network parameters to reduce the pre-training error:

[0136] ,

[0137] ,

[0138] in, represents the parameters of the next Q network, are the parameters of the next policy network, is the soft update rate. The update diagram of the above parameters is shown in Figure 3 shown.

[0139] In order to make the output trajectory of the flexible exoskeleton track the reference trajectory of the impedance model, an adaptive controller is proposed and a virtual control of the trajectory error is introduced. , in order to minimize the tracking error between the actual trajectory of the exoskeleton system and the output trajectory of the impedance model, expressed as:

[0140] ,

[0141] ,

[0142] in, and is the tracking error, The derivative of and The derivative of Satisfies the following formula:

[0143] ,

[0144] ,

[0145] in Indicates virtual control The rate of change.

[0146] In order to improve the trajectory tracking capability of the flexible exoskeleton system, eliminate the interference caused by gravity, material friction, and tight coupling of the model, introduce a nonlinear disturbance observer through an adaptive control algorithm to compensate for external interference, minimize the motion trajectory tracking error of the flexible exoskeleton system, and reduce the active knee joint torque of the user. The external factors meet the limited energy:

[0147] ,

[0148] in, is an unknown positive number, the value of which is determined by the magnitude of the interference caused by gravity, material friction, and tight coupling of the model, and is obtained through theoretical calculation and parameter estimation. is the rate of change of the external disturbance;

[0149] Observation error after the introduction of nonlinear observer for:

[0150] ,

[0151] in, is a nonlinear function, the observation error The rate of change is:

[0152] ,

[0153] Among them, the intermediate parameters ;

[0154] The nonlinear disturbance observer is used to estimate the nonlinear disturbance in the system. The estimated value is:

[0155] ,

[0156] in, It refers to the estimate of the error rate of change by the nonlinear disturbance observer. Refers to the estimated value of the error by the nonlinear disturbance observer, and then the estimated value of the disturbance D1 caused by external factors is obtained and error :

[0157] ,

[0158] .

[0159] Get the derivative of the disturbance error estimate :

[0160] .

[0161] So far, in the entire control framework, without involving robot dynamics and more model parameters, the user's active torque is minimized and the tracking error between the actual trajectory and the desired trajectory is reduced. Finally, the adaptive control algorithm based on the nonlinear disturbance observer is expressed as:

[0162] ,

[0163] in, is a gain constant.

[0164] According to statistical analysis, through the above reinforcement learning algorithm, when the learning rate of the critic network is 0.0001 and the learning rate of the target network is 0.001, the experience pool is set to 10 6 , discount factor When the algorithm iterates 100 times, the reward function reaches its maximum value, the muscle activation of the impaired limb of the hemiplegic patient is 21.34%, and the tracking performance of the controller is the best.

[0165] In a specific implementation, this embodiment provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium can store a computer program, and when the computer program is executed by the data processing unit, the invention content of a human-computer interaction control method for a flexible lower limb exoskeleton provided by the present invention and some or all of the steps in each embodiment can be executed. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0166] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention are essentially or the part that contributes to the prior art can be embodied in the form of a computer program, i.e., a software product, which can be stored in a storage medium and includes several instructions for enabling a device including a data processing unit (which can be a personal computer, a server, a single-chip microcomputer, a MUU or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0167] The present invention provides a human-machine interactive control method for a flexible lower limb exoskeleton. There are many methods and ways to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A human-computer interaction control method for a flexible lower limb exoskeleton, characterized in that: The steps include: Step 1, establishing an impedance model through an impedance control algorithm, wherein the impedance control algorithm takes into account the impedance characteristics of human joints, establishes an impedance model and solves the user's joint reference trajectory, thereby describing the mathematical relationship between the joint interaction torque and the motion trajectory of the human-machine coupling system; Step 2: Through the reinforcement learning algorithm, the stiffness and damping parameters are optimized under the condition of limited parameters of the robot and human dynamics model, with the goal of minimizing muscle activation and joint stiffness; In step 2, the following mathematical model for estimating muscle activation and joint stiffness is established: Among them, K e (t) is the joint stiffness of the knee joint at time t, agonist refers to the stimulating muscle during knee joint movement, antagonist refers to the antagonistic muscle during knee joint movement, u i (t) is the preprocessed electromyographic signal of the ith muscle at time t, a i (t) is the muscle activation of the ith muscle at time t, n is the total number of muscles, e is a natural constant; A is the nonlinear shape factor, α j and β j is the weight coefficient, τ i (t) is the joint torque of the knee joint at time t, expressed as: in, For muscle strength, is the muscle moment arm; Step 3: Reduce the tracking error between the actual trajectory of the exoskeleton system and the output trajectory of the impedance model through an adaptive control algorithm, and introduce a nonlinear disturbance observer to compensate for the disturbance caused by external factors to achieve the best assisted walking effect.

2. The method according to claim 1, characterized in that Step 1 includes: establishing a human-machine coupling dynamics model, the formula is: Where q is the user's actual joint angle, is the user's actual joint angular velocity, is the user's joint angular acceleration, M(q) is the inertia matrix, is the centrifugal force and Coriolis force equation, G(q) is the gravitational torque, D1 is the interference caused by external factors, τ a is the assisting torque of the exoskeleton system, τ h is the output torque of the joint of the user's damaged limb; the external factors include material friction, estimation error and model tight coupling; Considering the impedance characteristics of human joints, the impedance model including joint torque and angle is expressed as: Among them, q d is the expected trajectory angle, i.e., the trajectory position of the healthy limb; q r is the reference trajectory angle, i.e. the output trajectory position of the impedance model, and represent the corresponding desired speed and reference speed respectively, and represent the desired damping and the desired stiffness respectively.

3. The method according to claim 2, characterized in that In step 2, the reinforcement learning algorithm includes the current state s of the robot and the person, the action a of the robot, and the reward function r; The current state s of the robot and the person is expressed as: Among them, K represents the joint stiffness of the damaged limb of the human body; The action a of the robot is expressed as: The reward function r is expressed as: Among them, Λ1, Λ2, Λ3 are weight coefficients, T represents matrix transpose, K d represents the desired joint stiffness.

4. The method according to claim 3, characterized in that: In step 2, an adaptive parameter ζ is introduced to adjust the value of the Q function in reinforcement learning: ζ=Q(s,a)-[r(s,a)+Q(s′,a′)] Among them, Q(s,a) represents the total value of the current state, r(s,a) represents the immediate value of executing action a in state s, and Q(s′,a′) represents the total value of executing a new action a′ in the next state s′; The regularized adaptive parameters are calculated using the following formula Among them, r t is the instantaneous value at time t, max(r1,r2,…,r t ) represents the maximum value of the instantaneous value in all time steps; The Q function Q in the reinforcement learning algorithm is calculated using the following formula ζ (s,a): Where Q1 and Q2 are two independent Q networks; Q1(s,a) represents the action value function of Q network Q1 performing action a in state s; Q2(s,a) represents the action value function of Q network Q2 performing action a in state s; Use exploration noise to explore new actions. The action with exploration noise added is expressed as: Among them, a1 is the action after adding exploration noise, is a normal distribution, is a random parameter, and x is the exploration noise; The reinforcement learning algorithm uses the Q function to minimize the Bellman error of the parameter θ1 corresponding to the Q network Q1 and the parameter θ2 corresponding to the Q network Q2, that is, the loss function: Where, L(θ1) is the Bellman error about the random parameter θ1, and L(θ2) is the Bellman error about the random parameter θ2; is a function of the Q network Q1 in state a1, action s, and random parameters θ1; is the function of the Q network Q2 under state a2, action s, and random parameter θ2; D is the experience replay pool, which is the collection of collected state transitions s, a1, r, s′; the intermediate parameter z is defined as: z=r+γQ ζ (s,a1) Among them, γ is the discount factor; The reinforcement learning algorithm described here updates the policy using the following function to maximize: in, represents the expected operation on the state s sampled from the experience replay pool D. The expectation refers to the weighted average of all possible values ​​of the random variable, where the weight is the probability of each value; The reinforcement learning algorithm uses soft update to update network parameters to reduce the pre-training error: θ′ y ←tth y +(1-τ)θ′ y || (y＝1,2) Among them, θ′ y represents the parameters of the next Q network, is the parameter of the next policy network, and τ is the rate of soft update.

5. The method according to claim 4, characterized in that In step 3, the adaptive control algorithm introduces a virtual control α1 of the trajectory error to reduce the tracking error between the actual trajectory of the exoskeleton system and the output trajectory of the impedance model, which is expressed as: e1=qq r Among them, e1 and e2 are tracking errors, and the derivative of e1 is and the derivative of e2 Satisfies the following formula: in Indicates the rate of change of the virtual control α1.

6. The method according to claim 5, characterized in that In step 3, the external factors satisfy the limited energy: Among them, d m is an unknown positive number, is the rate of change of the external disturbance; The observation error e3 after the introduction of the nonlinear observer is: e3=D1-φ(e2) Among them, φ(e2) is a nonlinear function, and the rate of change of the observation error e3 is for: Among them, the intermediate parameters The nonlinear disturbance observer is used to estimate the nonlinear disturbance in the system. The estimated value is: in, It refers to the estimate of the error rate of change by the nonlinear disturbance observer. Refers to the estimated value of the error by the nonlinear disturbance observer, and then the estimated value of the disturbance D1 caused by external factors is obtained and error 7. The method according to claim 6, characterized in that Step 3 also includes: obtaining the derivative of the disturbance error estimate 8. The method according to claim 7, characterized in that Step 3 also includes: the adaptive control algorithm based on the nonlinear disturbance observer is expressed as: Where ε is a gain constant.

9. A storage medium, characterized in that: A computer program or instruction is stored, and when the computer program or instruction is executed, the method according to any one of claims 1 to 8 is implemented.

Images