A comfortable motion control method for a transfer and carrying nursing robot

Abstract

The present invention is a comfortable motion control method for a transfer back-carrying nursing robot. First, the motion trajectory of the robot is obtained through a back-carrying experiment, the axillary pressure and plantar pressure at each stage of the back-carrying motion are collected, the ratio of the vertical component of the axillary pressure to the plantar pressure at each stage is calculated, and the ratios of all stages are averaged to obtain the desired comfortable mechanical characteristics; then, the human-computer interaction process is modeled based on Markov decision making, and the state space, action space and reward function of the robot are constructed; finally, the Actor current policy network is trained using the DDPG algorithm, and the trained Actor current policy network is the optimal control strategy model, which maximizes the cumulative reward obtained by the robot, and the optimal control strategy model is used to guide the robot's back-carrying motion. This method takes the comfort of the person being cared for as the control target, and the robot can maintain the comfort of the person being cared for in real time while completing the back-carrying task.

Description

Technical Field

[0001] The invention belongs to the technical field of robot intelligent control, and in particular relates to a comfortable motion control method for a transfer and back-carrying nursing robot. Background Art

[0002] As the aging of the population in my country intensifies, the number of unhealthy, disabled and semi-disabled elderly people has increased significantly, and the problems of elderly care caused by this have become increasingly prominent. In order to reduce the burden on caregivers, transfer and back-carrying nursing robots have emerged. Transfer and back-carrying nursing robots can transfer the caregivers between different locations by imitating the human back-carrying movements.

[0003] During the carrying process, there is a large amount of human-machine interaction between the robot and the human body. At the same time, due to the weak mobility of disabled and semi-disabled people, rigid human-machine interaction will not only lead to a poor comfort experience, but also easily cause secondary injuries to the person being cared for. Therefore, comfortable motion control of the transfer carrying nursing robot is particularly important.

[0004] In the field of human-machine interaction, human comfort is mainly ensured by compliant control, which is mainly achieved by adjusting the position, speed, acceleration and other information of the robot's end actuator by feeding back the human-machine interaction force to the control system. However, the transfer back-carrying nursing robot cannot meet the needs of the back-carrying task by simply following the motion trajectory. On the one hand, the motion trajectories of different care recipients are different, and the same motion trajectory cannot guarantee the safety of different care recipients. Otherwise, the motion trajectory needs to be obtained for different care recipients, which is a huge workload. On the other hand, during the back-carrying movement, there is a large area of ​​contact between the care recipient and the robot. Excessive contact pressure will cause discomfort or even pain in the contact area of ​​the care recipient, which is less comfortable.

[0005] Reinforcement learning generates a large amount of data through interaction with the environment and uses it to train control strategies. After several learning iterations, the optimal strategy is learned, which can solve the robot's sequential decision-making problem in an unknown environment. This application takes the comfort of the person being cared for as the control goal, and based on reinforcement learning, proposes a comfortable motion control method for a transfer and carrying nursing robot, which can ensure the comfort of the person being cared for during the carrying motion. Summary of the invention

[0006] In view of the deficiencies in the prior art, the technical problem that the present invention intends to solve is to propose a comfortable motion control method for a transfer and back-carrying nursing robot.

[0007] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0008] A method for controlling the comfortable motion of a transfer and carrying nursing robot, characterized in that the method comprises the following steps:

[0009] Step 1: Obtain the robot's motion trajectory through the back-holding experiment, collect the axillary pressure and plantar pressure at each stage of the back-holding movement, calculate the ratio of the vertical component of the axillary pressure to the plantar pressure at each stage, and average the ratios of all stages to obtain the desired comfort mechanical characteristics K0;

[0010] Step 2: Model the human-machine interaction process based on Markov decision making, and construct the state space, action space, and reward function of the transfer and carrying nursing robot;

[0011] The state space of the transfer and carrying nursing robot is recorded as S = {X, Y, θ c ,F1,F2,K,V1,V2,V3}, X is the horizontal coordinate of the chest support, Y is the vertical coordinate of the chest support, θ c is the chest leaning angle, {X,Y,θ c} indicates chest leaning position; F1=F 腋 sinθ c is the vertical component of the axillary pressure, F 腋 is the armpit pressure, F2 is the plantar pressure, K = F1 / F2 is the comfort mechanical characteristic, {F1, F2, K} represents the human-machine interaction force; the robot action space is recorded as A = {V1, V2, V3}, V1, V2 and V3 are the speeds of the No. 1 electric cylinder, No. 2 electric cylinder and No. 3 electric cylinder respectively; the reward function is recorded as λ1, λ2 and λ3 are reward weights, X(t), Y(t) and K(t) are the horizontal and vertical coordinate values ​​and comfort mechanical characteristics of the chest support at time t. are the expected values ​​of the horizontal and vertical coordinates of the chest at time t respectively;

[0012] Step 3: Use the DDPG algorithm to train the Actor's current policy network. The trained Actor's current policy network is the optimal control policy model, which maximizes the cumulative reward obtained by the robot;

[0013] Step 4: Transfer the trained optimal control strategy model to the control system, place the robot's chest-resting position at the initial point of the motion trajectory, obtain the robot's current state, and input it into the optimal control strategy model. The optimal control strategy model outputs the action at the current moment, controls the speed of the three electric cylinders according to the action, and the robot transfers to the state at the next moment; input the state at the next moment into the optimal control strategy model, and repeat this step until the back-hugging movement is completed.

[0014] Furthermore, in step 3, the trained network also includes the Actor target policy network, the Critic current evaluation network, and the Critic target evaluation network;

[0015] The parameters of the Actor's current policy network are denoted as θμ , the input is the current state s t , the output is the current strategic action μ(s t |θ μ ); the parameter of the Actor target strategy network is denoted as θ μ' , the input is the state s at the next moment t+1 , the output is the optimal strategy action μ'(s t+1 |θ μ' );The parameter of the critic’s current evaluation network is recorded as θ Q , the input is the current state s t and perform action a t , the output is the action value Q(s) of the Critic's current evaluation network t ,a t |θ Q );The parameter of the Critic target evaluation network is recorded as θ Q' , the input is the state s at the next moment t+1 and the optimal strategy action μ'(s t+1 |θ μ' ), the output is the action value Q'(s) of the Critic target evaluation network t+1 ,μ'(s t+1 |θ μ' )|θ Q' ); the steps of network training are as follows:

[0016] 3.1) Initialize the parameters θ of the Critic's current evaluation network Q and the parameters θ of the Actor's current policy network μ , the initialized parameters θ Q and θ μ They are used as the initial values ​​of the Critic target evaluation network parameters and the Critic target strategy network parameters respectively;

[0017] 3.2) Initialize the UO random process for action exploration and obtain the initial state s1 of the transfer and carrying nursing robot;

[0018] 3.3) Select an execution action a according to the behavior strategy t =μ(s t |θμ)+N t , N t represents random UO noise; the robot performs action a at the current moment t , get the state s at the next moment t+1 and the current reward value r t , and repeat this process continuously to obtain a large amount of interaction data;

[0019] 3.4) Randomly sample N batches of interaction data (s i ,a i ,r i ,s i+1 ), four networks are trained; among them, s i 、a i 、r i Respectively represent the current state, execution action and reward value of the i-th sample data, s i+1 is the state of the i-th sampling data at the next moment;

[0020] Calculate the target value y of the Critic's current evaluation network i , update the parameters θ of the current evaluation network by minimizing the loss function Q , update the parameters θ of the current policy network by sampling gradients μ , update the parameters θ of the target policy network by sliding average μ' and the parameters θ of the target evaluation network Q' ; The loss function is:

[0021]

[0022] 3.5) Repeat steps 3.3) to 3.4) until the maximum number of steps is reached;

[0023] 3.6) Repeat steps 3.2) to 3.5) until the maximum number of training rounds is reached.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. The present invention obtains the motion trajectory of the transfer back-carrying nursing robot by designing a back-carrying experiment, and simultaneously collects the axillary pressure and plantar pressure at each stage of the back-carrying movement, processes the axillary pressure and plantar pressure data, and obtains comfortable mechanical characteristics that are unrelated to individual differences, providing a basis for judging the comfort of the person being cared for.

[0026] 2. The comfortable mechanical characteristics are used as the design basis of the reinforcement learning reward function, and the Markov decision modeling of the human-computer interaction process is performed; the DDPG algorithm is used to train the network to obtain the optimal control strategy model. Under the optimal control strategy model, the transfer back-carrying nursing robot can output the optimal action at the current moment according to the current state. This control method takes the comfort of the person being cared for as the control target. The robot can maintain the comfort of the person being cared for in real time while completing the back-carrying task. The comfortable mechanical characteristics are independent of individual differences. While eliminating the tedious work of planning motion trajectories for different care recipients, the comfort is better, meeting the comfort requirements of different individuals, and having stronger adaptability and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a schematic diagram of the mechanism of the transfer and carrying nursing robot;

[0028] Figure 2 It is the overall flow chart of the method of the present invention;

[0029] Figure 3 This is the training flow chart of the DDPG algorithm;

[0030] Explanation of the reference numerals in the accompanying drawings: 1. Electric cylinder No. 1; 2. Electric cylinder No. 2; 3. Electric cylinder No. 3; 4. Boom No. 1; 5. Boom No. 2; 6. Chest rest; 7. Auxiliary armrest; 8. Armpit support structure; 9. Driving wheel; 10. Driven wheel; 11. Frame; 12. Foot pedal; 13. Knee baffle; 14. Armpit force sensor; 15. Plantar pressure sensor; 16. Person being cared for. DETAILED DESCRIPTION

[0031] The technical solution of the present invention is described in detail below in conjunction with specific embodiments and drawings, but the protection scope of the present application is not limited thereto.

[0032] The transfer and carrying nursing robot is mainly aimed at people with lower limb disability. It can realize the safe and comfortable transfer of the person being cared for between bed, wheelchair, toilet, sofa, etc. by imitating the carrying action of people. Figure 1 The transfer and carrying nursing robot consists of two parts: a carrying mechanism and a transfer mechanism. The carrying mechanism is a series-parallel hybrid mechanism consisting of an electric cylinder No. 1, an electric cylinder No. 2, an electric cylinder No. 3, an arm No. 1, an arm No. 2, and a chest support 6. Auxiliary armrests 7 and an underarm support structure 8 are symmetrically arranged on both sides of the chest support 6; the electric cylinder No. 1 and the electric cylinder No. 2 respectively drive the arm No. 1 and the arm No. 2, 5 to realize the movement of the chest support 6 in the horizontal and vertical directions, and the electric cylinder No. 3 drives the chest support 6 to realize rotation, and the chest of the person being cared 16 is close to the chest support 6. Under the drive of the electric cylinder No. 3, the chest support 6 cooperates with the underarm support structure 8 to lift the person being cared 16, thereby completing the carrying movement; the transfer mechanism is a body consisting of two driving wheels 9 and a driven wheel 10, a frame 11, a foot pedal 12, and a knee baffle 13. The knee baffle 13 is installed on the frame 11, and the knee of the person being cared 16 is naturally in contact with the knee baffle 13.

[0033] The present invention provides a comfortable motion control method for a transfer and carrying nursing robot (hereinafter referred to as the method), which specifically comprises the following steps:

[0034] Step 1: Design a back-carrying test to obtain the motion trajectory of the transfer back-carrying nursing robot, collect the axillary pressure and plantar pressure at each stage of the back-carrying movement; calculate the ratio of the vertical component of the axillary pressure to the plantar pressure at each stage, and average the ratios of all stages to obtain the desired comfort mechanical characteristics K0;

[0035] Recruit a number of healthy adults with a height of 1.65 to 1.85 meters and a weight of 50 to 85 kg as subjects, and perform a back-carrying test to simulate a care recipient with intact upper limbs and disabled lower limbs; during the back-carrying exercise, the subject needs to maintain the standard test posture, that is, the trunk is tilted forward, so that the front chest fits tightly against the chest support 6, the armpits on both sides of the subject are placed on the armpit support structures 8 on both sides of the robot chest support, the hands hold the auxiliary handrails 7, the feet are placed at the center of the plantar pressure sensor 15 and remain in the same position, the knees are naturally in contact with the knee baffles 13, the whole body is kept relaxed, and no force is applied to the lower limbs;

[0036] The whole back-carrying movement process is divided into five stages including lifting, rising, placing, descending and placing. The robot mainly completes the back-carrying movement by changing the chest support posture. Therefore, as a human-machine interaction medium, the chest support posture will seriously affect the comfort of the human body. Based on the PVT motion mode of the PowerPMAC motion controller, the transfer back-carrying nursing robot performs the back-carrying movement, and the subjective comfort feeling of the subjects is obtained through the subjective comfort evaluation method, and the chest support posture of each stage is optimized and adjusted according to the subjective comfort feeling of the subjects. The subjective comfort evaluation obtains the subjective comfort feeling of the subjects in the form of questionnaire scoring. The subjective comfort scoring table is shown in Table 1.

[0037] Table 1 Subjective comfort rating table

[0038] Discomfort Mild discomfort Acceptable More comfortable Comfort -2 -1 0 1 2

[0039] In each stage, the chest leaning posture that the human body feels comfortable is selected as the typical feature point, and a total of five typical feature points are obtained, which are recorded as P1~P5 respectively; the chest leaning posture is recorded as (X, Y, θ c ), X is the horizontal coordinate of the chest support, Y is the vertical coordinate of the chest support, θ c is the chest leaning angle, that is, the angle between the chest leaning and the horizontal direction; according to the five typical feature points, the motion trajectory of the transfer and back-carrying nursing robot is obtained by the piecewise cubic Hermite interpolation algorithm;

[0040] The chest-rest posture was adjusted to the initial point P1 of the motion trajectory, and the subjects were told to maintain the standard experimental posture, keep the whole body relaxed, and breathe steadily. The axillary pressure and plantar pressure in the lifting stage were collected through the force testing system; to reduce the experimental error, the collection was repeated 3 times and the average value was taken, with an interval of 5 minutes each time to ensure that the subjects had sufficient rest; the chest-rest posture was adjusted to the other typical feature points of the motion trajectory, and the axillary pressure and plantar pressure in the remaining stages were collected in turn; the ratio of the vertical component of the axillary pressure to the plantar pressure in each stage was calculated, and the ratios of the five stages were averaged to obtain the desired comfort mechanical characteristics K0, which were used to guide the design of reinforcement learning reward functions; the data processing results showed that the comfort mechanical characteristics were not related to the individual differences of the subjects.

[0041] The force testing system includes an underarm force sensor 14 and a plantar pressure sensor 15; the underarm force sensor 14 is a one-dimensional force sensor, and both underarm support structures 8 are equipped with an underarm force sensor 14 for collecting underarm pressure information of the human body; the plantar pressure sensor 15 is a three-dimensional force sensor, which is installed on the foot pedal 12 for collecting plantar pressure information of the human body.

[0042] Step 2: Use reinforcement learning to solve the sequential decision-making problem of the transfer-carrying nursing robot during the carrying movement, and conduct Markov decision modeling for the human-computer interaction process, that is, construct the state space of the transfer-carrying nursing robot according to the information required during the carrying movement, construct the action space according to the control instructions of the transfer-carrying nursing robot, and construct the reward function according to the carrying movement requirements;

[0043] The Markov decision model consists of a five-tuple (S, A, P, R, γ), where S is the state space of the mobile back-carrying nursing robot, which is mainly composed of the information required by the mobile back-carrying nursing robot to complete the back-carrying movement, and is used to describe the state of the mobile back-carrying nursing robot; A is the action space of the mobile back-carrying nursing robot, which is mainly composed of the control instructions of the mobile back-carrying nursing robot and defines the actions that the robot can perform; P is the state transition probability, R is the reward function, which is mainly determined by the back-carrying movement requirements of the mobile back-carrying nursing robot and is used to evaluate the quality of the action; γ is the discount factor;

[0044] The state space of the transfer nursing robot is S = {X, Y, θ c ,F1,F2,K,V1,V2,V3}, X is the horizontal coordinate of the chest support, Y is the vertical coordinate of the chest support, θ c is the chest leaning angle, {X,Y,θ c} indicates chest leaning position; F1=F 腋 sinθ c is the vertical component of the axillary pressure, F 腋 is the armpit pressure, F2 is the plantar pressure, K = F1 / F2 is the comfort mechanical characteristic, {F1, F2, K} represents the human-machine interaction force; V1, V2 and V3 are the speeds of No. 1 electric cylinder 1, No. 2 electric cylinder 2 and No. 3 electric cylinder 3 respectively, {V1, V2, V3} represents the speed of the robot; the chest posture and speed of the robot are obtained by the host computer, and the human-machine interaction force is obtained by the force test system; the robot action space is recorded as A = {V1, V2, V3}, and the speed command is executed by No. 1 electric cylinder 1, No. 2 electric cylinder 2 and No. 3 electric cylinder 3; the reward function is recorded as Among them, λ1, λ2 and λ3 are reward weights, X(t), Y(t) and K(t) are the horizontal and vertical coordinate values ​​and comfort mechanical characteristics of the chest support at time t. are the expected values ​​of the horizontal and vertical coordinates of the chest support at time t, which can be read from the motion trajectory of the transfer and back-carrying nursing robot;

[0045] The general process of reinforcement learning is: the state of the back-carrying nursing robot at time t is s t , that is, the vector of variables in the state space at time t. The robot performs an action a according to the strategy π(a|s) at time t. t , which is the vector of variables in the action space at time t, and then the back-carrying nursing robot is transferred to the state s at time t+1 t+1 , and get a reward value r t ; Strategy refers to the mapping from state to action, which defines the probability of the transfer back-carrying nursing robot performing each available action in the action space under a certain state; this cycle is repeated, and a large amount of data including actions, states and immediate reward values ​​are generated through continuous interaction. The transfer back-carrying nursing robot uses these data to correct its own strategy for learning. After several iterative learning, the optimal strategy is obtained, and the cumulative reward value under the optimal strategy is the largest; during the back-carrying movement, the transfer back-carrying nursing robot selects actions in different states according to the optimal strategy, so that the entire sequential decision is optimal.

[0046] Step 3: Use the Deep Deterministic Policy Gradient (DDPG) algorithm to train the network. The trained Actor’s current policy network is the optimal control policy model.

[0047] The trained network includes the Actor policy network and the Critic evaluation network. The Actor policy network is divided into the Actor current policy network and the Actor target policy network. The Critic evaluation network is divided into the Critic current evaluation network and the Critic target evaluation network.

[0048] Actor's current strategy network μ: network parameters are θ μ , the input is the current state s of the transfer and carrying nursing robot t , the output is the current strategic action μ(s t |θμ);

[0049] Actor target policy network μ': network parameters are θ μ' , the input is the state s of the next moment of the transfer carrying nursing robot t+1 , the output is the optimal strategy action μ'(s t+1 |θ μ' ), that is, the optimal action at the next moment is obtained through the Actor target strategy network;

[0050] Critic currently evaluates the network Q: the network parameters are θ Q , the input is the current state s of the transfer and carrying nursing robot t and perform action a t , the output is the action value of the Critic's current evaluation network, that is, Q(s t ,a t |θ Q );

[0051] Critic target evaluation network Q': the network parameter is θ Q' , the input is the state s of the next moment of the transfer carrying nursing robot t+1 and the optimal strategy action μ'(s t+1 |θ μ' ), the output is the action value of the Critic target evaluation network, i.e. Q'(s t+1 ,μ'(s t+1 |θ μ' )|θ Q' ).

[0052] Network parameter update method:

[0053] θ μ : By sampling gradient To update, the formula is: Where N is the number of sampled data, The gradient of the action value of the Critic current evaluation network to the action a, s i is the state of the i-th sampling data at the current moment, a i is the execution action of the i-th sampling data at the current moment, The policy action pair parameter θ output by the Actor's current policy network μ The gradient of

[0054] θ μ ': Update using sliding average, the formula is θ μ '=τθ μ +(1-τ)θ μ ', τ is the sliding average coefficient;

[0055] θ Q :Update by minimizing the loss function L, the formula is: Among them, y i y is the target value of the Critic's current evaluation network, expressed as: i =r i +γQ'(s i+1 ,μ'(s i+1 |θ μ ')|θ Q'), r i is the reward value, γ is the discount factor;

[0056] θ Q ': Update using sliding average, the formula is: θ Q '=τθ Q +(1-τ)θ Q ';

[0057] The training process is as follows:

[0058] 3.1) Initialize the parameters θ of the Critic's current evaluation network Q and the parameters θ of the Actor's current policy network μ , the initialized parameters θ Q and θ μ As Critic targets, we evaluate the network parameters θ Q ' and Critic target strategy network parameters θ μ '; Initialize the replay memory buffer; Set the maximum number of training rounds M and the maximum number of steps per round T;

[0059] 3.2) Initialize the UO random process for action exploration and obtain the initial state s1 of the transfer and carrying nursing robot;

[0060] 3.3) Select an execution action a according to the behavior strategy t =μ(s t |θμ)+N t , the behavior strategy is a random process generated according to the current strategy network and random UO noise, N t represents random UO noise; the transfer carrying nursing robot performs the action a at the current moment t , get the state s at the next moment t+1 and the current reward value r t ; The current state s t , perform action a t , reward value r t and the state s at the next moment t+1 A set of interaction data (s t ,a t ,r t ,s t+1 ), store the interaction data in the replay memory buffer, and repeat this process by moving the nursing robot to the back to generate a large amount of interaction data, which constitutes the data set for training the network; when the size of the interaction data exceeds the capacity of the replay memory buffer, replace the old interaction data with the new interaction data;

[0061] 3.4) Randomly sample N batches of interaction data (s) from the replay memory buffer i ,a i ,r i ,s i+1 ), four networks are trained; among them, s i 、a i 、r i Respectively represent the current state, execution action and reward value of the i-th sample data, s i+1 is the state of the i-th sampling data at the next moment;

[0062] Calculate the target value y of the Critic's current evaluation network i , update the parameters θ of the current evaluation network by minimizing the loss function Q , update the parameters θ of the current policy network by sampling gradients μ , update the parameters θ of the target policy network by sliding average μ ' and the parameters θ of the target evaluation network Q ';

[0063] 3.5) Repeat steps 3.3) to 3.4) until the maximum number of steps T is reached;

[0064] 3.6) Repeat steps 3.2) to 3.5) until the maximum number of training rounds M is reached;

[0065] After the above steps, the training of the four networks is completed, and the trained Actor current strategy network is the optimal control strategy model; the input of the optimal control strategy model is the current state of the transfer back-carrying nursing robot, and the output is the current action, so that the cumulative reward obtained by the transfer back-carrying nursing robot is maximized.

[0066] Step 4: Transfer the trained optimal control strategy model to the control system, place the chest leaning posture of the transfer back-holding nursing robot at the initial point of the motion trajectory, obtain the current state of the robot, that is, the chest leaning posture, human-computer interaction force information and speed information in the state space, and input them into the optimal control strategy model. The optimal control strategy model outputs the action at the current moment, converts the action into a speed instruction and orders the robot to execute it, and the robot transfers to the state at the next moment; inputs the state at the next moment into the optimal control strategy model, and repeats this step until the back-holding movement is completed; under the optimal control strategy model, the transfer back-holding nursing robot can not only complete the back-holding movement, but also maintain the comfort of the person being cared for during the back-holding movement.

[0067] The above network training is carried out on the human-computer interaction simulation platform. First, the physical model of the transfer and carrying nursing robot is designed based on the Mujoco multi-joint dynamic contact simulator, and the human-computer interaction simulation platform of the transfer and carrying nursing robot is built; the Mujoco model file includes the following tags: the Mujoco tag represents the model file, and the map information and the transfer and carrying nursing robot model information are all under this tag; the STL model files of the robot components are imported at the mesh tag in the asset; the mass and inertia characteristics of the robot are defined in the body tag, and the geometric characteristics are filled in with the geom tag; the joint motion range and speed are defined in the joint tag; the drivers of the No. 1 electric cylinder 1, No. 2 electric cylinder 2 and No. 3 electric cylinder 3 are defined in the acutator tag; the armpit and plantar pressure sensors are defined in the sensor tag. Call the humanoid of mujoco as the model of the person being cared for, and define the constraints and robot contact under the contrast tag. Select Gym as the reinforcement learning algorithm development toolkit, select Pytorch as the development framework, use Mujoco.py as the interface library between the Mujoco physics engine and Python, and call Mujoco functions through its library functions.

[0068] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A method for controlling the comfortable motion of a transfer and back-carrying nursing robot, wherein the transfer and back-carrying nursing robot comprises a No. 1 electric cylinder, a No. 2 electric cylinder, a No. 3 electric cylinder and a chest support, and the chest support posture is controlled by the three electric cylinders to complete the back-carrying motion; characterized in that: The method comprises the following steps: Step 1: Obtain the robot's motion trajectory through the back-holding experiment, collect the axillary pressure and plantar pressure at each stage of the back-holding movement, calculate the ratio of the vertical component of the axillary pressure to the plantar pressure at each stage, and average the ratios of all stages to obtain the desired comfort mechanical characteristics K0; Step 2: Model the human-machine interaction process based on Markov decision making, and construct the state space, action space, and reward function of the transfer and carrying nursing robot; The state space of the transfer and carrying nursing robot is recorded as S = {X, Y, θ c ,F1,F2,K,V1,V2,V3}, X is the horizontal coordinate of the chest support, Y is the vertical coordinate of the chest support, θ c is the chest leaning angle, {X,Y,θ c } indicates chest leaning position; F1=F 腋 sinθ c is the vertical component of the axillary pressure, F 腋 is the armpit pressure, F2 is the plantar pressure, K = F1 / F2 is the comfort mechanical characteristic, {F1, F2, K} represents the human-machine interaction force; the robot action space is recorded as A = {V1, V2, V3}, V1, V2 and V3 are the speeds of the No. 1 electric cylinder, No. 2 electric cylinder and No. 3 electric cylinder respectively; the reward function is recorded as λ1, λ2 and λ3 are reward weights, X(t), Y(t) and K(t) are the horizontal and vertical coordinate values ​​and comfort mechanical characteristics of the chest support at time t. are the expected values ​​of the horizontal and vertical coordinates of the chest at time t respectively; Step 3: Use the DDPG algorithm to train the Actor's current policy network. The trained Actor's current policy network is the optimal control policy model, which maximizes the cumulative reward obtained by the robot; Step 4: Transfer the trained optimal control strategy model to the control system, place the robot's chest-resting position at the initial point of the motion trajectory, obtain the robot's current state, and input it into the optimal control strategy model. The optimal control strategy model outputs the action at the current moment, controls the speed of the three electric cylinders according to the action, and the robot transfers to the state at the next moment; input the state at the next moment into the optimal control strategy model, and repeat this step until the back-hugging movement is completed.

2. The comfortable motion control method of the transfer and carrying nursing robot according to claim 1 is characterized in that: In step 3, the trained network also includes the Actor target policy network, the Critic current evaluation network, and the Critic target evaluation network; The parameters of the Actor's current policy network are denoted as θ μ , the input is the current state s t , the output is the current strategic action μ(s t |θ μ ); the parameter of the Actor target strategy network is denoted as θ μ ', the input is the state s at the next moment t+1 , the output is the optimal strategy action μ'(s t+1 |θ μ' );The parameter of the current evaluation network of Critic is recorded as θ Q , the input is the current state s t and perform action a t , the output is the action value Q(s) of the Critic's current evaluation network t ,a t |θ Q );The parameter of the Critic target evaluation network is recorded as θ Q ', the input is the state s at the next moment t+1 and the optimal strategy action μ'(s t+1 |θ μ' ), the output is the action value Q'(s) of the Critic target evaluation network t+1 ,μ'(s t+1 |θ μ' )|θ Q' ); the steps of network training are as follows: 3.1) Initialize the parameters θ of the Critic's current evaluation network Q and the parameters θ of the Actor's current policy network μ , the initialized parameters θ Q and θ μ They are used as the initial values ​​of the Critic target evaluation network parameters and the Critic target strategy network parameters respectively; 3.2) Initialize the UO random process for action exploration and obtain the initial state s1 of the transfer and carrying nursing robot; 3.3) Select an execution action a according to the behavior strategy t =μ(s t |θ μ )+N t , N t represents random UO noise; the robot performs action a at the current moment t , get the state s at the next moment t+1 and the current reward value r t , and repeat this process continuously to obtain a large amount of interaction data; 3.4) Randomly sample N batches of interaction data (s i ,a i ,r i ,s i+1 ), four networks are trained; among them, s i 、a i 、r i Respectively represent the current state, execution action and reward value of the i-th sample data, s i+1 is the state of the i-th sampling data at the next moment; Calculate the target value y of the Critic's current evaluation network i , update the parameters θ of the current evaluation network by minimizing the loss function Q , update the parameters θ of the current policy network by sampling gradients μ , update the parameters θ of the target policy network by sliding average μ ' and the parameters θ of the target evaluation network Q ';The loss function is: 3.5) Repeat steps 3.3) to 3.4) until the maximum number of steps is reached; 3.6) Repeat steps 3.2) to 3.5) until the maximum number of training rounds is reached.

3. The comfortable motion control method of the transfer and carrying nursing robot according to claim 1 or 2 is characterized in that: The back-holding test includes the following: The whole back-holding movement is divided into five stages: holding, rising, placing, descending and placing. The subject is asked to stand on the robot in the standard experimental posture. The robot performs the back-holding task and adjusts the chest-resting posture. In each stage, the chest-resting posture that the subject feels comfortable subjectively is selected as the typical feature point, and a total of five typical feature points are obtained. Based on the five typical feature points, the robot's motion trajectory is obtained by interpolation. The chest-resting posture was adjusted to the initial point of the motion trajectory, and the subjects maintained the standard experimental posture, and the axillary pressure and plantar pressure were collected during the lifting stage; the chest-resting posture was adjusted to the typical characteristic points of the remaining stages, and the axillary pressure and plantar pressure were collected in turn in the remaining stages.

4. The comfortable motion control method of the transfer and carrying nursing robot according to claim 3 is characterized in that: The standard test posture is that the subject's torso leans forward, the front chest is tightly fitted against the chest support, the subject's two armpits are respectively placed on the armpit support structures on both sides of the robot's chest support, the subject's hands hold the robot's auxiliary armrests on both sides of the chest support, both feet are placed on the plantar pressure sensors on the foot pedals, both knees are naturally in contact with the robot's knee baffles, the whole body remains relaxed, and the lower limbs do not actively exert force.

Images