Fall self-recovery learning method, device, electronic device and storage medium
Through the fall recovery learning method, combining posture information and preset actions to calculate the target movement, the problem of long training time and poor results of four-legged robot fall self-recovery training for a long time and poor effect is solved, and efficient robot fall self-recovery learning is achieved.
Patent Information
- Application Number
- CN202210692345.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-17
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-06-17
AI Technical Summary
In the prior art, the four-legged robot fall self-recovery method has a long training time and poor learning effect. The reinforcement learning method based on deep neural networks has problems of inefficiency.
The fall recovery learning method is adopted. By obtaining the robot's posture information, inputting it to the fall recovery learning model, calculating the target action with preset actions, generating control instructions, causing the robot to execute the action, and updating the model parameters until the training stop condition is reached.
The training time of the fall recovery learning model is shortened, the learning efficiency of the robot's fall recovery self-recovery is improved, and efficient training and good learning results are achieved in a shorter time.
Smart Images

Figure CN114986510B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of robotics technology, and in particular to a fall self-recovery learning method, device, electronic equipment and storage medium. Background Art
[0002] With the development of robot technology, people's demands on robots in all aspects are constantly increasing. As a type of leg-footed robot, the study of quadrupedal robot's self-recovery after falling is a research hotspot.
[0003] In the related art, the self-recovery method of a quadruped robot after falling mainly relies on the reinforcement learning method of a deep neural network. The posture information and joint angle information of the quadruped robot are input into the deep neural network to obtain the action output by the deep neural network, and the action is directly sent to the quadruped robot so that the quadruped robot performs the action. The above content is repeated to train the deep neural network. However, the reinforcement learning method based on the deep neural network directly trains the quadruped robot, which has the disadvantages of long training time and poor learning effect. Summary of the invention
[0004] The present invention provides a fall self-recovery learning method, device, electronic device and storage medium, which are used to solve the defects of the prior art that the training time is relatively long and the learning effect is relatively poor, and the training can be completed in a shorter time with better learning effect.
[0005] The present invention provides a fall self-recovery learning method, the method comprising:
[0006] Acquire first posture information of the robot; the first posture information includes first rotation angle information and first joint angle information; the first rotation angle information is used to indicate the rotation angle corresponding to the current posture of the robot; the first joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot in the current posture;
[0007] The first rotation angle information and the first joint angle information are input into a fall recovery learning model to obtain a first action output by the fall recovery learning model; the fall recovery learning model is trained based on sample posture information and is used to determine a first action of the robot; the first action is used to represent the torque of at least one joint of each leg of the robot;
[0008] Based on the first action and a preset action corresponding to the first action, a target action of the robot is determined; the preset action is used to represent the action corresponding to the first action in a preset strategy; the preset strategy includes a sequence of actions performed by the robot to recover from a falling state to a standing state; the target action is used to represent the torque of at least one joint of each leg when the robot actually performs the action.
[0009] According to a fall self-recovery learning method provided by the present invention, determining the target action of the robot based on the first action and a preset action corresponding to the first action includes:
[0010] The first action and the preset action are calculated using formula (1) to determine the target action of the robot;
[0011]
[0012] in, represents the target action of the robot, represents a first action output by the fall recovery learning model; represents the preset action; t represents the tth step; t∈[1,K], K is a positive integer; Represents the Hadamard product.
[0013] According to a fall self-recovery learning method provided by the present invention, the method further includes:
[0014] Acquire second posture information of the robot; the second posture information includes second rotation angle information and second joint angle information; the second rotation angle information is used to indicate the rotation angle corresponding to the falling posture of the robot; the second joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot in the falling posture;
[0015] Based on the second rotation angle, determining a first foot-end force of each leg of the robot; the first foot-end force is used to represent a force corresponding to at least one joint of each leg of the robot in a falling posture;
[0016] Based on the first foot-end force and the second joint angle information, determining a second action of the robot; the second action is used to represent the torque of at least one joint of each leg of the robot when adjusting the center of gravity of the robot;
[0017] Acquire third posture information of the robot; the third posture information includes third rotation angle information and third joint angle information; the third rotation angle information is used to indicate the rotation angle corresponding to the robot after adjusting the center of gravity of the robot; the third joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot after adjusting the center of gravity of the robot;
[0018] Based on the third rotation angle information, determining a second foot-end force of each leg of the robot; the second foot-end force is used to represent the force corresponding to at least one joint of each leg after adjusting the center of gravity of the robot;
[0019] Based on the second foot-end force and the third joint angle information, determining a third action of the robot; the third action is used to represent the torque of at least one joint of each leg of the robot when restoring the posture of the robot from a falling posture to a semi-inclined posture;
[0020] Based on the third action, an action sequence in the preset strategy is determined.
[0021] According to a fall self-recovery learning method provided by the present invention, the determining of the action sequence in the preset strategy based on the third action includes:
[0022] Based on the third action, fourth posture information of the robot is obtained; the fourth posture information includes fourth rotation angle information and fourth joint angle information; the fourth rotation angle information is used to indicate the rotation angle corresponding to the semi-tilted posture of the robot; the fourth joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot when the robot is in the semi-tilted posture;
[0023] Based on the fourth rotation angle information, determining a third foot-end force of each leg of the robot; the third foot-end force is used to represent a force corresponding to at least one joint of each leg when the robot is in a semi-tilted posture;
[0024] Based on the third foot-end force and the fourth joint angle information, determining a fourth action of the robot; the fourth action is used to represent the torque of at least one joint of each leg of the robot when restoring the posture of the robot from a semi-tilted state to a standing state;
[0025] Based on the third action, the fourth action and the second action, an action sequence in the preset strategy is determined.
[0026] According to a fall self-recovery learning method provided by the present invention, after determining the target action of the robot, the method further includes:
[0027] Based on the target action, generating a control instruction corresponding to the target action; the control instruction is used to control the robot to perform the target action;
[0028] sending the control instruction to the robot;
[0029] Acquire fifth posture information of the robot; the fifth posture information includes fifth rotation angle information and fifth joint angle information; the fifth rotation angle information is used to indicate the corresponding rotation angle after the robot performs the target action; the fifth joint angle information is used to indicate the corresponding joint angle of at least one joint of each leg of the robot after the robot performs the target action;
[0030] Calculating a reward function of the fall recovery learning model;
[0031] Based on the reward function, updating parameters of the fall recovery learning model;
[0032] Determine whether the fall recovery learning model meets the training stop condition; the training stop condition is that the reward function converges.
[0033] The present invention also provides a fall self-recovery learning device, the device comprising:
[0034] A first acquisition module is used to acquire first posture information of the robot; the first posture information includes first rotation angle information and first joint angle information; the first rotation angle information is used to indicate the rotation angle corresponding to the current posture of the robot; the first joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot in the current posture;
[0035] a fall recovery learning module, configured to input the first rotation angle information and the first joint angle information into a fall recovery learning model, and obtain a first action output by the fall recovery learning model; the fall recovery learning model is trained based on sample posture information, and is used to determine a first action of the robot; the first action is used to represent a torque of at least one joint of each leg of the robot;
[0036] A first determination module is used to determine the target action of the robot based on the first action and a preset action corresponding to the first action; the preset action is used to represent the action corresponding to the first action in a preset strategy; the preset strategy includes a sequence of actions performed by the robot to recover from a falling state to a standing state; the target action is used to represent the torque of at least one joint of each leg when the robot actually performs the action.
[0037] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the fall self-recovery learning method as described in any one of the above-mentioned methods is implemented.
[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described fall self-recovery learning methods.
[0039] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-mentioned fall self-recovery learning methods.
[0040] The fall self-recovery learning method, device, electronic device and storage medium provided by the present invention obtain the first posture information of the robot; the first posture information includes the first rotation angle information and the first joint angle information; the first rotation angle information and the first joint angle information are input into the fall recovery learning model to obtain the first action output by the fall recovery learning model; according to the first action and the preset action corresponding to the first action, the target action of the robot is determined. The method provided by the present invention determines the target action of the robot by combining the first action and the preset action corresponding to the first action, realizes the reinforcement learning process of the robot from falling to self-recovery to standing state, shortens the training time of the fall recovery learning model, and improves the learning efficiency of the robot's fall self-recovery. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0042] Figure 1 This is one of the flow charts of the fall self-recovery learning method provided by the present invention;
[0043] Figure 2 It is a structural schematic diagram of the robot provided by the present invention;
[0044] Figure 3 This is the second flow chart of the fall self-recovery learning method provided by the present invention;
[0045] Figure 4 This is a schematic diagram of the change of rotation angle information of the robot falling and self-recovering provided by the present invention;
[0046] Figure 5It is a structural schematic diagram of the fall self-recovery learning device provided by the present invention;
[0047] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0048] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0049] The fall self-recovery learning method provided by the present invention is described in detail below through some embodiments and application scenarios in conjunction with the accompanying drawings.
[0050] The present invention provides a fall self-recovery learning method, which is applicable to the scenario of robot self-recovery after falling, and obtains the first posture information of the robot; the first posture information includes first rotation angle information and first joint angle information; the first rotation angle information is used to represent the rotation angle corresponding to the current posture of the robot; the first joint angle information is used to represent the joint angle corresponding to at least one joint of each leg of the robot's current posture; the first rotation angle information and the first joint angle information are input into a fall recovery learning model to obtain a first action output by the fall recovery learning model; the fall recovery learning model is trained based on sample posture information and is used to determine the first action of the robot; the first action is used to represent the torque of at least one joint of each leg of the robot; based on the first action and the preset action corresponding to the first action, the target action of the robot is determined; the preset action is used to represent the action corresponding to the first action in the preset strategy; the preset strategy includes the action sequence performed by the robot to recover from a falling state to a standing state; the target action is used to represent the torque of at least one joint of each leg when the robot actually performs the action. The method provided by the present invention determines the target action of the robot through a first action and a preset action corresponding to the first action, thereby realizing a reinforcement learning process of the robot's self-recovery from falling to a standing state, shortening the training time of the robot's self-recovery model after falling, and improving the learning efficiency of the robot's self-recovery after falling.
[0051] Combine the following Figure 1-Figure 4 The fall self-recovery learning method of the present invention is described.
[0052] Figure 1FIG. 1 is one of the flow charts of the fall self-recovery learning method provided by the present invention, such as Figure 1 As shown, the method includes steps 101 to 103, wherein:
[0053] Step 101, obtain the first posture information of the robot; the first posture information includes first rotation angle information and first joint angle information; the first rotation angle information is used to represent the rotation angle corresponding to the current posture of the robot; the first joint angle information is used to represent the joint angle corresponding to at least one joint of each leg of the current posture of the robot.
[0054] It should be noted that the fall self-recovery learning method provided by the present invention can be applicable to the scenario of robot fall self-recovery. The execution subject of the method can be a fall self-recovery learning device, such as an electronic device, or a control module in the fall self-recovery learning device for executing the fall self-recovery learning method.
[0055] Specifically, the robot can be a quadruped robot or other types of robots, and this application does not impose any restrictions on this. It should be noted that an inertial measurement unit (IMU) is installed on the body of the robot, and each joint of each leg of the robot is equipped with a motor and a sensor. The first rotation angle information of the robot in any posture can be obtained through the IMU, wherein the first rotation angle information is used to indicate the rotation angle corresponding to the current posture of the robot, which can be expressed as S IMU =(θ x ,θ y ,θ z ), where θ x ,θ y and θ z Respectively represent the rotation angles of the robot around the x, y, and z axes in the world coordinate system; the first joint angle information of the robot can be obtained through the sensor, wherein the first joint angle information is used to represent the joint angle corresponding to at least one joint of each leg of the robot's current posture, and the joint may include at least one of the hip joint, knee joint, and ankle joint. Therefore, the first posture information of the robot can be obtained, and the first posture information may include the first rotation angle information and the first joint angle information.
[0056] Step 102: input the first rotation angle information and the first joint angle information into a fall recovery learning model to obtain a first action output by the fall recovery learning model; the fall recovery learning model is trained based on sample posture information and is used to determine the first action of the robot; the first action is used to represent the torque of at least one joint of each leg of the robot.
[0057] Specifically, during the training process of the fall recovery learning model, the acquired first rotation angle information and first joint angle information of the robot are input into the fall recovery learning model to obtain the first action output by the fall recovery learning model; wherein, the fall recovery learning model can be a deep deterministic policy gradient (DDPG) or a structure of other neural network models; the network structure of the fall recovery learning model is a four-layer neural network, including an input layer, two hidden layers and an output layer; wherein, the number of nodes in the two hidden layers can be 256, and the activation function of the output layer is a hyperbolic tangent function (tanh function), which limits the output result of the fall recovery learning model to the range of [-1, 1].
[0058] In practice, before the first rotation angle information and the first joint angle information are input into the fall recovery learning model, the first rotation angle information and the first joint angle information are normalized to the range of [-1, 1], and then the processed first rotation angle information and the first joint angle information are input into the input layer of the fall recovery learning model, and then pass through the two hidden layers in the middle in sequence, and finally the output layer outputs the result to obtain the first action output by the fall recovery learning model; wherein, the first action is used to represent the torque of at least one joint of each leg of the robot, and the fall recovery learning model is trained based on sample posture information, and is used to determine the first action of the robot.
[0059] Step 103, based on the first action and the preset action corresponding to the first action, determine the target action of the robot; the preset action is used to represent the action corresponding to the first action in the preset strategy; the preset strategy includes the action sequence performed by the robot to recover from a falling state to a standing state; the target action is used to represent the torque of at least one joint of each leg when the robot actually performs the action.
[0060] Specifically, after the fall recovery learning model outputs the first action, a preset action corresponding to the first action is determined from the preset strategy, and then the target action of the robot is determined based on the first action and the preset action; wherein the preset strategy is a pre-set action sequence for the robot to recover from a falling state to a standing state, that is, a set of actions performed by the robot to recover from a falling state to a standing state, and each action in the action set is an action performed by the robot corresponding to each posture of the robot in the process of fall recovery; each action represents the torque of at least one joint of each leg when the robot performs the action, that is, a torque is given to at least one joint of the robot, and the joints of the robot perform corresponding actions based on the torque; the target action is used to represent the torque of at least one joint of each leg when the robot actually performs the action.
[0061] The fall self-recovery learning method provided by the present invention obtains the first posture information of the robot; the first posture information includes the first rotation angle information and the first joint angle information; the first rotation angle information and the first joint angle information are input into the fall recovery learning model to obtain the first action output by the fall recovery learning model; and the target action of the robot is determined according to the first action and the preset action corresponding to the first action. The method provided by the present invention determines the target action of the robot by combining the first action and the preset action corresponding to the first action, realizes the reinforcement learning process of the robot from falling to self-recovery to standing state, shortens the training time of the fall recovery learning model, and improves the learning efficiency of the robot from falling self-recovery.
[0062] Optionally, determining a target action of the robot based on the first action and a preset action corresponding to the first action includes:
[0063] The first action and the preset action are calculated using formula (1) to determine the target action of the robot;
[0064]
[0065] in, represents the target action of the robot, represents a first action output by the fall recovery learning model; represents the preset action; t represents the tth step, t∈[1,K], K is a positive integer; Represents the Hadamard product.
[0066] Specifically, according to the first action output by the fall recovery learning model, the first action and the preset action corresponding to the first action are combined, and the above formula (1) is used for calculation to determine the target action actually to be performed by the robot at each step in the process of training the fall recovery learning model; wherein the first action output by the fall recovery learning model can be used as a control ratio of the preset action corresponding to the first action in the preset strategy, that is, it can be used as a control coefficient of the torque of at least one joint of each leg of the robot, so that the robot executes the target action in accordance with the standard action of the fall recovery standing state, and achieves a better effect.
[0067] In practice, during the training of the fall recovery learning model, the preset action corresponding to the first action is determined according to the first action output by the fall recovery learning model; wherein the preset action is determined according to the number of training steps in a single round recorded during the training of the fall recovery learning model, and the preset action corresponds to the number of training steps one by one, so that the preset action corresponding to the first action can be determined. For example, if the number of training steps in a single round of the fall recovery learning model is 5, then the fall recovery learning model is trained to the 5th step, then the preset action corresponding to the 5th step in the prediction strategy is searched, and then the target action of the robot in the 5th step is determined according to the above formula (1).
[0068] The fall self-recovery learning method provided by the present invention determines the target action of the robot by calculating the first action and the preset action, so that the target action executed by the robot is more in line with the standard action in the process of the robot's fall self-recovery, thereby realizing the reinforcement learning process of the robot's fall self-recovery to a standing state, shortening the training time of the fall recovery learning model, and improving the learning efficiency of the robot's fall self-recovery.
[0069] According to the description of the above embodiment, since the preset action performed by the robot to recover from a falling state to a standing state is used in the training process of the fall recovery learning model, and the preset action is the action corresponding to the first action in the preset strategy, the specific implementation method of the robot's preset strategy is described in detail below.
[0070] Figure 2 : is a schematic diagram of the structure of the robot provided by the present invention, such as Figure 2 As shown, the robot includes a first supporting leg, a second supporting leg, a third supporting leg and a fourth supporting leg, and each leg includes a hip joint, a knee joint and an ankle joint.
[0071] In order to calculate the joint torque required for each joint of each leg of the robot to recover from a fall, the three-dimensional force F at the foot end of the i-th leg is converted into i Converted into the torque τ of the three joints of the i-th leg i ,in:
[0072]
[0073] Where i represents the i-th leg, J 3×3 represents the Jacobian matrix, F i represents the three-dimensional force at the foot end of the i-th leg, θ i1 ,θ i2 ,θ i3 denote the joint angles of the hip joint, knee joint, and ankle joint of the robot’s i-th leg, respectively, and d 01 represents half the distance between the hip joint of the first supporting leg and the hip joint of the second supporting leg, a01 represents the distance between the hip joint and the knee joint of the ith leg, a 02 represents the distance between the knee and ankle joints of the ith leg, a 03 Represents the distance between the ankle joint and the foot of the i-th leg.
[0074] Optionally, a specific implementation method of the robot's force distribution preset strategy includes the following steps:
[0075] Step 1) Obtain the second posture information of the robot; the second posture information includes second rotation angle information and second joint angle information; the second rotation angle information is used to indicate the rotation angle corresponding to the falling posture of the robot; the second joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot's falling posture.
[0076] Specifically, the second posture information of the robot is obtained; wherein the second posture information includes second rotation angle information and second joint angle information, and the second rotation angle information is obtained by the inertial measurement unit IMU of the robot body, which can be expressed as S IMU =(θ x ,θ y ,θ z ), where θ x ,θ y ,θ z They respectively represent the rotation angles of the robot around the x, y, and z axes in the world coordinate system; the second joint angle information is obtained through the motor of at least one joint of each leg of the robot.
[0077] It should be noted that the second posture information of the robot is obtained when the robot is in a falling state, wherein the falling state may be a side-falling state.
[0078] Step 2) Based on the second rotation angle, determine the first foot-end force of each leg of the robot; the first foot-end force is used to represent the force corresponding to at least one joint of each leg of the robot in the falling posture.
[0079] Specifically, according to the second rotation angle of the robot, the following formula (3) can be used to determine the first foot end force F of the i-th leg of the robot: i ,in:
[0080]
[0081] Among them, θ x It represents the rotation angle of the robot around the x-axis when it falls, M represents the robot's own weight, and g represents the acceleration due to gravity.
[0082] In practice, if the robot falls sideways, the first supporting leg and the third supporting leg of the robot touch the ground, or the second supporting leg and the fourth supporting leg touch the ground. In either case, the first foot end force of each leg of the robot can be determined by the above formula (3).
[0083] Step 3) Based on the first foot end force and the second joint angle information, determine the second action of the robot; the second action is used to represent the torque of at least one joint of each leg of the robot when adjusting the center of gravity of the robot.
[0084] Specifically, according to the first foot end force and the second joint angle information, the torque τ of at least one joint of the i-th leg of the robot can be calculated using the above formula (2): i , thus determining the second action of the robot, which can be expressed as a1 init =[τ 11 τ 12 τ 13 τ 14 ], where τ 11 , τ 12 , τ 13 and τ 14 They respectively represent the torque of at least one joint of the first supporting leg, the second supporting leg, the third supporting leg and the fourth supporting leg, that is, the torque of 12 joints.
[0085] In practice, at least one joint of each leg of the robot is given a corresponding torque, so that the robot can perform the corresponding second action. In other words, the θ in the second rotation angle information obtained by the robot in the falling state is x , if θ x >0, the first and third supporting legs of the robot extend, and the second and fourth supporting legs contract, so that the center of gravity of the robot moves to θ x The direction of the decrease is offset; if θ x <0, the second and fourth supporting legs of the robot extend, and the first and third supporting legs contract, so that the center of gravity of the robot moves to θ x Increased direction deviation.
[0086] Step 4) Obtain the third posture information of the robot; the third posture information includes third rotation angle information and third joint angle information; the third rotation angle information is used to indicate the corresponding rotation angle of the robot after adjusting the center of gravity of the robot; the third joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot after adjusting the center of gravity of the robot.
[0087] Specifically, after adjusting the center of the robot, the third posture information of the robot is obtained, and the third posture information includes third rotation angle information and third joint angle information; wherein the third rotation angle information is used to indicate the rotation angle corresponding to the robot after adjusting the center of gravity of the robot; the third joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot after adjusting the center of gravity of the robot.
[0088] It should be noted that the third posture information of the robot is obtained after the robot adjusts the center of the robot. At this time, the robot is in a state where the first supporting leg and the third supporting leg are extended, and the second supporting leg and the fourth supporting leg are contracted, or the second supporting leg and the fourth supporting leg are extended, and the first supporting leg and the third supporting leg are contracted.
[0089] Step 5) Based on the third rotation angle information, determine the second foot-end force of each leg of the robot; the second foot-end force is used to represent the force corresponding to at least one joint of each leg after adjusting the center of gravity of the robot.
[0090] Specifically, according to the third rotation angle of the robot, the second foot end force F of the i-th leg of the robot can be determined by the following formula (4): i ,in:
[0091]
[0092] Among them, θ x It represents the rotation angle of the robot around the x-axis after adjusting the center of gravity, M represents the robot's own weight, and g represents the acceleration due to gravity.
[0093] In practice, if the robot falls sideways, after adjusting the center of gravity of the robot, the robot is still in a state where the first supporting leg and the third supporting leg are on the ground, or the second supporting leg and the fourth supporting leg are on the ground; in either case, the second foot end force of each leg of the robot can be determined by the above formula (4).
[0094] Step 6) Based on the second foot end force and the third joint angle information, determine the third action of the robot; the third action is used to represent the torque of at least one joint of each leg of the robot when restoring the posture of the robot from a falling posture to a semi-leaning posture.
[0095] Specifically, according to the second foot end force and the third joint angle information, the torque τ of at least one joint of the i-th leg of the robot can be calculated using the above formula (2): i , thus determining the third action of the robot, which can be expressed as Among them, τ 21 , τ 22 , τ 23 and τ24 Respectively represent the torque of at least one joint of the first supporting leg, the second supporting leg, the third supporting leg and the fourth supporting leg.
[0096] In practice, at least one joint of each leg of the robot is given a corresponding torque, and the robot performs the corresponding third action. In other words, the robot's posture is restored from the fallen state to the semi-inclined state by using the supporting leg in contact with the ground.
[0097] Step 7) Based on the third action, determine the action sequence in the preset strategy.
[0098] Next, the specific implementation method of the above step 7) is further explained.
[0099] Optionally, the determining, based on the third action, an implementation manner of the action sequence in the preset strategy comprises the following steps:
[0100] Step 7-1) Based on the third action, obtain the fourth posture information of the robot; the fourth posture information includes fourth rotation angle information and fourth joint angle information; the fourth rotation angle information is used to indicate the rotation angle corresponding to the semi-tilted posture of the robot; the fourth joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot when the robot is in a semi-tilted posture.
[0101] Specifically, when the robot's posture is in a semi-tilted state, the fourth posture information of the robot is obtained, and the fourth posture information includes fourth rotation angle information and fourth joint angle information; wherein the fourth rotation angle information is used to indicate the rotation angle corresponding to the robot's semi-tilted posture; the fourth joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot when the robot is in a semi-tilted posture.
[0102] It should be noted that the fourth posture information of the robot is obtained when the robot posture is in a semi-inclined state. At this time, the first supporting leg, the second supporting leg, the third supporting leg and the fourth supporting leg of the robot are all in a grounded state.
[0103] Step 7-2) Based on the fourth rotation angle information, determine the third foot-end force of each leg of the robot; the third foot-end force is used to represent the force corresponding to at least one joint of each leg when the robot is in a semi-tilted posture.
[0104] Specifically, when the robot is in a semi-tilted state, the third foot end force F of the robot's i-th leg can be determined using the following formula (5): i ,in:
[0105]
[0106] Where M represents the robot's own weight and g represents the acceleration due to gravity.
[0107] In practice, the robot's posture is in a semi-inclined state, with each leg of the robot touching the ground, i.e., the third foot end force of each leg is equal.
[0108] Step 7-3) Based on the third foot end force and the fourth joint angle information, determine the fourth action of the robot; the fourth action is used to represent the torque of at least one joint of each leg of the robot when restoring the posture of the robot from a semi-leaning state to a standing state.
[0109] Specifically, according to the third foot end force and the fourth joint angle information, the torque τ of at least one joint of the i-th leg of the robot can be calculated using the above formula (2): i , thus determining the fourth action of the robot, which can be expressed as Among them, τ 31 , τ 32 , τ 33 and τ 34 Respectively represent the torque of at least one joint of the first supporting leg, the second supporting leg, the third supporting leg and the fourth supporting leg.
[0110] In practice, at least one joint of each leg of the robot is given a corresponding torque, and the robot performs the corresponding fourth action. In other words, the robot's posture is restored from a semi-tilted state to a standing state by bouncing.
[0111] Step 7-4) Determine an action sequence in the preset strategy based on the third action, the fourth action and the second action.
[0112] Specifically, according to the third action, the fourth action and the second action, the action sequence is obtained by collecting the third action, the fourth action and the second action. By linearly interpolating the action sequence, we can get the action sequence in the preset strategy, which can be expressed as Here, K is a positive integer; for example, K may be 300.
[0113] The fall self-recovery learning method provided by the present invention obtains the posture information of the robot in the fall state, center of gravity adjustment, and semi-tilted state, respectively determines the corresponding execution actions of the robot, collects the executed actions, and then obtains the action sequence in the preset strategy through linear interpolation, thereby pre-setting the action sequence that can be executed in the fall recovery process for the robot, which is helpful for combining the action sequence in the preset strategy with the action output by the fall recovery learning model during the subsequent training of the fall recovery learning model, thereby realizing the reinforcement learning process of the robot's self-recovery from falling to the standing state, shortening the training time of the fall recovery learning model, and improving the learning efficiency of the robot's fall self-recovery.
[0114] Optionally, after the above step 103, the method further comprises the following steps:
[0115] Step 1) Based on the target action, a control instruction corresponding to the target action is generated; the control instruction is used to control the robot to perform the target action.
[0116] Specifically, according to the target action of the robot, a control instruction corresponding to the target action is generated, and the control instruction is used to control the robot to execute the target action.
[0117] Step 2) Send the control instruction to the robot.
[0118] Specifically, a control instruction is sent to the robot, and after the robot receives the control instruction, it performs the target action according to the control instruction.
[0119] It should be noted that the interaction with the robot is to transmit data using RS485 serial communication technology, so that the robot obtains control instructions, or transmits posture information obtained by the robot's inertial measurement unit and motor.
[0120] Step 3) Obtain the fifth posture information of the robot; the fifth posture information includes fifth rotation angle information and fifth joint angle information; the fifth rotation angle information is used to indicate the corresponding rotation angle after the robot performs the target action; the fifth joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot after the robot performs the target action.
[0121] Specifically, after the robot performs the target action, the fifth posture information of the robot is obtained; the fifth posture information includes fifth rotation angle information and fifth joint angle information; wherein the fifth rotation angle information is used to indicate the corresponding rotation angle after the robot performs the target action; the fifth joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot after the robot performs the target action.
[0122] Step 4) Calculate the reward function of the fall recovery learning model.
[0123] Specifically, according to the fifth rotation angle information, the reward function of the fall recovery learning model is calculated using the following formula (6), where:
[0124]
[0125] Among them, R represents the reward function, θ x represents the rotation angle of the quadruped robot around the x-axis; θ y represents the rotation angle of the quadruped robot around the y-axis.
[0126] Step 5) Based on the reward function, update the parameters of the fall recovery learning model.
[0127] Specifically, according to the reward function, it is determined whether the value of the reward function reaches the target value; if the target value is not reached, the parameters of the fall recovery learning model are updated; wherein the target value can be 0, indicating that θ x and θ y If both are 0, the robot falls and recovers to a standing state; if the target value is reached, the next round of training will be carried out.
[0128] Step 6) Determine whether the fall recovery learning model meets the training stop condition; the training stop condition is that the reward function converges.
[0129] Specifically, the cumulative value of the reward function is calculated to determine whether the fall recovery learning model reaches the training stop condition; if the cumulative value of the reward function no longer changes or changes within a smaller target range, the training of the fall recovery learning model stops; otherwise, the above steps are repeated to continue training the fall recovery learning model until the reward function of the fall recovery learning model converges.
[0130] The fall self-recovery learning method provided by the present invention generates a control instruction corresponding to the target action to enable the robot to perform the target action. After the robot performs the target action, the fifth posture information of the robot is obtained, the value of the reward function is calculated according to the fifth posture information, and the parameters of the fall recovery learning model are updated according to the value of the reward function. Then, it is judged whether the reward function converges to judge whether the fall recovery learning model reaches the training stop condition, thereby realizing the training process of the robot's fall recovery learning model, shortening the total number of training rounds of the fall recovery learning model, thereby shortening the training time of the fall recovery learning model, and improving the learning efficiency of the robot's fall self-recovery.
[0131] Figure 3 FIG. 2 is a flow chart of the fall self-recovery learning method provided by the present invention. Figure 3As shown, the method includes steps 301 to 311, wherein:
[0132] Step 301, obtain the first posture information of the robot; the first posture information includes first rotation angle information and first joint angle information, wherein the first posture information of the robot is obtained in real time, therefore, the first rotation angle information is used to represent the rotation angle corresponding to the current posture of the robot; the first joint angle information is used to represent the joint angle corresponding to at least one joint of each leg of the current posture of the robot.
[0133] Step 302: input the first rotation angle information and the first joint angle information into a fall recovery learning model to obtain a first action output by the fall recovery learning model.
[0134] Step 303, combining the first action and the preset action corresponding to the first action to determine the target action of the robot; wherein the preset action is used to represent the action corresponding to the first action in the preset strategy; the preset strategy includes the action sequence performed by the robot to recover from a falling state to a standing state; the target action is used to represent the torque of at least one joint of each leg when the robot actually performs the action.
[0135] It should be noted that during the training process of the fall recovery learning model, each step of the training process is recorded. The first action output by the fall recovery learning model can be used to determine the step to which the fall recovery learning model has been trained, thereby determining the preset action corresponding to the first action in the preset strategy.
[0136] Step 304: Generate a control instruction corresponding to the target action based on the target action.
[0137] Step 305: Send a control instruction to the robot to enable the robot to perform a target action.
[0138] Step 306, obtain the fifth posture information of the robot; the fifth posture information includes fifth rotation angle information and fifth joint angle information; wherein, the fifth rotation angle information is used to indicate the corresponding rotation angle after the robot performs the target action; the fifth joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot after the robot performs the target action.
[0139] Step 307: Calculate a reward function of the fall recovery learning model based on the fifth rotation angle.
[0140] 308, determine whether the reward function reaches the target value. If yes, go to step 301 to perform the next round of training; otherwise, go to step 309.
[0141] Step 309: Update the parameters of the fall recovery learning model.
[0142] Step 310, determine whether the fall recovery learning model has reached the training stop condition, where the training stop condition is the convergence of the reward function, which means that the cumulative value of the reward function no longer changes or changes within the target range. If yes, go to step 311; otherwise, go to step 301.
[0143] Step 311, the training ends.
[0144] Figure 4 : is a schematic diagram of the rotation angle information change of the robot falling and self-recovering provided by the present invention, such as Figure 4 As shown in the figure, the horizontal axis represents the number of steps (steps) that the robot takes to perform the target action during the self-recovery process, and the vertical axis represents the rotation angle information of the robot after each step of performing the target action. Figure 4 It can be seen that the robot performs a total of 300 target actions during the process of self-recovery after falling; at step 0, the rotation angle information is θ x greater than 0, θ y is equal to 0, at this time the robot is in a falling posture; from step 260 to step 300, θ x and θ y The value of is close to 0, indicating that the robot has successfully recovered to the standing state.
[0145] The following is a description of the fall self-recovery learning device provided by the present invention. The fall self-recovery learning device described below and the fall self-recovery learning method described above can be referenced to each other.
[0146] Figure 5 Schematic diagram of the structure of the fall self-recovery learning device provided by the present invention. Figure 4 As shown, the fall self-recovery learning device 500 includes: a first acquisition module 501, a fall recovery learning module 502 and a first determination module 503; wherein:
[0147] The first acquisition module 501 is used to acquire first posture information of the robot; the first posture information includes first rotation angle information and first joint angle information; the first rotation angle information is used to indicate the rotation angle corresponding to the current posture of the robot; the first joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the current posture of the robot;
[0148] a fall recovery learning module 502, configured to input the first rotation angle information and the first joint angle information into a fall recovery learning model, and obtain a first action output by the fall recovery learning model; the fall recovery learning model is trained based on sample posture information and is used to determine a first action of the robot; the first action is used to represent a torque of at least one joint of each leg of the robot;
[0149] The first determination module 503 is used to determine the target action of the robot based on the first action and the preset action corresponding to the first action; the preset action is used to represent the action corresponding to the first action in the preset strategy; the preset strategy includes the action sequence performed by the robot to recover from a falling state to a standing state; the target action is used to represent the torque of at least one joint of each leg when the robot actually performs the action.
[0150] The fall self-recovery learning device provided by the present invention obtains the first posture information of the robot; the first posture information includes the first rotation angle information and the first joint angle information; the first rotation angle information and the first joint angle information are input into the fall recovery learning model to obtain the first action output by the fall recovery learning model; and the target action of the robot is determined according to the first action and the preset action corresponding to the first action. The method provided by the present invention determines the target action of the robot by combining the first action and the preset action corresponding to the first action, realizes the reinforcement learning process of the robot self-recovering from the fall state to the standing state, shortens the training time of the fall recovery learning model, and improves the learning efficiency of the robot's fall self-recovery.
[0151] Optionally, the first determining module 503 is specifically configured to:
[0152] The first action and the preset action are calculated using formula (1) to determine the target action of the robot;
[0153]
[0154] in, represents the target action of the robot, represents a first action output by the fall recovery learning model; represents the preset action; t represents the tth step; t∈[1,K], K is a positive integer; Represents the Hadamard product.
[0155] Optionally, the fall self-recovery learning device 500 further includes:
[0156] A second acquisition module is used to acquire second posture information of the robot; the second posture information includes second rotation angle information and second joint angle information; the second rotation angle information is used to indicate the rotation angle corresponding to the falling posture of the robot; the second joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot in the falling posture;
[0157] A second determination module is used to determine a first foot-end force of each leg of the robot based on the second rotation angle; the first foot-end force is used to represent a force corresponding to at least one joint of each leg of the robot in a falling posture;
[0158] a third determination module, configured to determine a second action of the robot based on the first foot-end force and the second joint angle information; the second action is used to represent a torque of at least one joint of each leg of the robot when adjusting the center of gravity of the robot;
[0159] a third acquisition module, configured to acquire third posture information of the robot; the third posture information includes third rotation angle information and third joint angle information; the third rotation angle information is used to indicate the rotation angle corresponding to the robot after adjusting the center of gravity of the robot; the third joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot after adjusting the center of gravity of the robot;
[0160] a fourth determination module, configured to determine a second foot-end force of each leg of the robot based on the third rotation angle information; the second foot-end force is used to represent a force corresponding to at least one joint of each leg after adjusting the center of gravity of the robot;
[0161] a fifth determination module, configured to determine a third action of the robot based on the second foot-end force and the third joint angle information; the third action is used to represent the torque of at least one joint of each leg of the robot when restoring the posture of the robot from a falling posture to a semi-inclined posture;
[0162] A sixth determining module is used to determine an action sequence in the preset strategy based on the third action.
[0163] Optionally, the sixth determining module is specifically configured to:
[0164] Based on the third action, fourth posture information of the robot is obtained; the fourth posture information includes fourth rotation angle information and fourth joint angle information; the fourth rotation angle information is used to indicate the rotation angle corresponding to the semi-tilted posture of the robot; the fourth joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot when the robot is in the semi-tilted posture;
[0165] Based on the fourth rotation angle information, determining a third foot-end force of each leg of the robot; the third foot-end force is used to represent a force corresponding to at least one joint of each leg when the robot is in a semi-tilted posture;
[0166] Based on the third foot-end force and the fourth joint angle information, determining a fourth action of the robot; the fourth action is used to represent the torque of at least one joint of each leg of the robot when restoring the posture of the robot from a semi-tilted state to a standing state;
[0167] Based on the third action, the fourth action and the second action, an action sequence in the preset strategy is determined.
[0168] Optionally, the fall self-recovery learning device 500 further includes:
[0169] A generating module, used for generating a control instruction corresponding to the target action based on the target action; the control instruction is used for controlling the robot to perform the target action;
[0170] A sending module, used for sending the control instruction to the robot;
[0171] a fourth acquisition module, configured to acquire fifth posture information of the robot; the fifth posture information includes fifth rotation angle information and fifth joint angle information; the fifth rotation angle information is used to indicate the corresponding rotation angle after the robot performs the target action; the fifth joint angle information is used to indicate the corresponding joint angle of at least one joint of each leg of the robot after the robot performs the target action;
[0172] A calculation module, used for calculating a reward function of the fall recovery learning model;
[0173] An updating module, configured to update parameters of the fall recovery learning model based on the reward function;
[0174] A judgment module is used to judge whether the fall recovery learning model reaches a training stop condition; the training stop condition is that the reward function converges.
[0175] Figure 6 is a schematic diagram of the physical structure of an electronic device provided by the present invention, such as Figure 6 As shown, the electronic device 600 may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the fall self-recovery learning method, which includes:
[0176] Acquire first posture information of the robot; the first posture information includes first rotation angle information and first joint angle information; the first rotation angle information is used to indicate the rotation angle corresponding to the current posture of the robot; the first joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot in the current posture;
[0177] The first rotation angle information and the first joint angle information are input into a fall recovery learning model to obtain a first action output by the fall recovery learning model; the fall recovery learning model is trained based on sample posture information and is used to determine a first action of the robot; the first action is used to represent the torque of at least one joint of each leg of the robot;
[0178] Based on the first action and a preset action corresponding to the first action, a target action of the robot is determined; the preset action is used to represent the action corresponding to the first action in a preset strategy; the preset strategy includes a sequence of actions performed by the robot to recover from a falling state to a standing state; the target action is used to represent the torque of at least one joint of each leg when the robot actually performs the action.
[0179] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0180] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the fall self-recovery learning method provided by the above methods, the method comprising:
[0181] Acquire first posture information of the robot; the first posture information includes first rotation angle information and first joint angle information; the first rotation angle information is used to indicate the rotation angle corresponding to the current posture of the robot; the first joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot in the current posture;
[0182] The first rotation angle information and the first joint angle information are input into a fall recovery learning model to obtain a first action output by the fall recovery learning model; the fall recovery learning model is trained based on sample posture information and is used to determine a first action of the robot; the first action is used to represent the torque of at least one joint of each leg of the robot;
[0183] Based on the first action and a preset action corresponding to the first action, a target action of the robot is determined; the preset action is used to represent the action corresponding to the first action in a preset strategy; the preset strategy includes a sequence of actions performed by the robot to recover from a falling state to a standing state; the target action is used to represent the torque of at least one joint of each leg when the robot actually performs the action.
[0184] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the fall self-recovery learning method provided by the above methods, the method comprising:
[0185] Acquire first posture information of the robot; the first posture information includes first rotation angle information and first joint angle information; the first rotation angle information is used to indicate the rotation angle corresponding to the current posture of the robot; the first joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot in the current posture;
[0186] The first rotation angle information and the first joint angle information are input into a fall recovery learning model to obtain a first action output by the fall recovery learning model; the fall recovery learning model is trained based on sample posture information and is used to determine a first action of the robot; the first action is used to represent the torque of at least one joint of each leg of the robot;
[0187] Based on the first action and a preset action corresponding to the first action, a target action of the robot is determined; the preset action is used to represent the action corresponding to the first action in a preset strategy; the preset strategy includes a sequence of actions performed by the robot to recover from a falling state to a standing state; the target action is used to represent the torque of at least one joint of each leg when the robot actually performs the action.
[0188] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0189] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A fall self-recovery learning method, characterized in that: The method comprises: Acquire first posture information of the robot; the first posture information includes first rotation angle information and first joint angle information; the first rotation angle information is used to indicate the rotation angle corresponding to the current posture of the robot; the first joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot in the current posture; The first rotation angle information and the first joint angle information are input into a fall recovery learning model to obtain a first action output by the fall recovery learning model; the fall recovery learning model is trained based on sample posture information and is used to determine a first action of the robot; the first action is used to represent the torque of at least one joint of each leg of the robot; the network structure of the fall recovery learning model is a four-layer neural network, including an input layer, two hidden layers and an output layer; wherein the number of nodes in the two hidden layers is 256, and the activation function of the output layer is a hyperbolic tangent function, which limits the output result of the fall recovery learning model to the range of [-1, 1]; Determine a target action of the robot based on the first action and a preset action corresponding to the first action; the preset action is used to represent an action corresponding to the first action in a preset strategy; the preset strategy includes a sequence of actions performed by the robot to recover from a falling state to a standing state; the target action is used to represent a torque of at least one joint of each leg when the robot actually performs the action; The determining the target action of the robot based on the first action and a preset action corresponding to the first action includes: The first action and the preset action are calculated using formula (1) to determine the target action of the robot; in, represents the target action of the robot, represents a first action output by the fall recovery learning model; represents the preset action; t represents the tth step; t∈[1,K], K is a positive integer; represents the Hadamard product; The method further comprises: Acquire second posture information of the robot; the second posture information includes second rotation angle information and second joint angle information; the second rotation angle information is used to indicate the rotation angle corresponding to the falling posture of the robot; the second joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot in the falling posture; Based on the second rotation angle, determining a first foot-end force of each leg of the robot; the first foot-end force is used to represent a force corresponding to at least one joint of each leg of the robot in a falling posture; Based on the first foot-end force and the second joint angle information, determining a second action of the robot; the second action is used to represent the torque of at least one joint of each leg of the robot when adjusting the center of gravity of the robot; Acquire third posture information of the robot; the third posture information includes third rotation angle information and third joint angle information; the third rotation angle information is used to indicate the rotation angle corresponding to the robot after adjusting the center of gravity of the robot; the third joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot after adjusting the center of gravity of the robot; Based on the third rotation angle information, determining a second foot-end force of each leg of the robot; the second foot-end force is used to represent the force corresponding to at least one joint of each leg after adjusting the center of gravity of the robot; Based on the second foot-end force and the third joint angle information, determining a third action of the robot; the third action is used to represent the torque of at least one joint of each leg of the robot when restoring the posture of the robot from a falling posture to a semi-inclined posture; Based on the third action, determining an action sequence in the preset strategy; The determining, based on the third action, an action sequence in the preset strategy includes: Based on the third action, fourth posture information of the robot is obtained; the fourth posture information includes fourth rotation angle information and fourth joint angle information; the fourth rotation angle information is used to indicate the rotation angle corresponding to the semi-tilted posture of the robot; the fourth joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot when the robot is in the semi-tilted posture; Based on the fourth rotation angle information, determining a third foot-end force of each leg of the robot; the third foot-end force is used to represent a force corresponding to at least one joint of each leg when the robot is in a semi-tilted posture; Based on the third foot-end force and the fourth joint angle information, determining a fourth action of the robot; the fourth action is used to represent the torque of at least one joint of each leg of the robot when restoring the posture of the robot from a semi-tilted state to a standing state; Based on the third action, the fourth action and the second action, an action sequence in the preset strategy is determined.
2. The fall self-recovery learning method according to claim 1, characterized in that: After determining the target action of the robot, the method further includes: Based on the target action, generating a control instruction corresponding to the target action; the control instruction is used to control the robot to perform the target action; sending the control instruction to the robot; Acquire fifth posture information of the robot; the fifth posture information includes fifth rotation angle information and fifth joint angle information; the fifth rotation angle information is used to indicate the corresponding rotation angle after the robot performs the target action; the fifth joint angle information is used to indicate the corresponding joint angle of at least one joint of each leg of the robot after the robot performs the target action; Calculating a reward function of the fall recovery learning model; Based on the reward function, updating parameters of the fall recovery learning model; Determine whether the fall recovery learning model meets the training stop condition; the training stop condition is that the reward function converges.
3. A fall self-recovery learning device, characterized in that: The device comprises: A first acquisition module is used to acquire first posture information of the robot; the first posture information includes first rotation angle information and first joint angle information; the first rotation angle information is used to indicate the rotation angle corresponding to the current posture of the robot; the first joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot in the current posture; A fall recovery learning module, used for inputting the first rotation angle information and the first joint angle information into a fall recovery learning model, and obtaining a first action output by the fall recovery learning model; the fall recovery learning model is trained based on sample posture information, and is used to determine the first action of the robot; the first action is used to represent the torque of at least one joint of each leg of the robot; the network structure of the fall recovery learning model is a four-layer neural network, including an input layer, two hidden layers and an output layer; wherein the number of nodes in the two hidden layers is 256, and the activation function of the output layer is a hyperbolic tangent function, which limits the output result of the fall recovery learning model to the range of [-1, 1]; a first determining module, configured to determine a target action of the robot based on the first action and a preset action corresponding to the first action; the preset action is used to represent an action corresponding to the first action in a preset strategy; the preset strategy includes a sequence of actions performed by the robot to recover from a falling state to a standing state; the target action is used to represent a torque of at least one joint of each leg when the robot actually performs the action; The first determining module is specifically configured to: The first action and the preset action are calculated using formula (1) to determine the target action of the robot; in, represents the target action of the robot, represents a first action output by the fall recovery learning model; represents the preset action; t represents the tth step; t∈[1,K], K is a positive integer; represents the Hadamard product; The fall self-recovery learning device also includes: A second acquisition module is used to acquire second posture information of the robot; the second posture information includes second rotation angle information and second joint angle information; the second rotation angle information is used to indicate the rotation angle corresponding to the falling posture of the robot; the second joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot in the falling posture; A second determination module is used to determine a first foot-end force of each leg of the robot based on the second rotation angle; the first foot-end force is used to represent a force corresponding to at least one joint of each leg of the robot in a falling posture; a third determination module, configured to determine a second action of the robot based on the first foot-end force and the second joint angle information; the second action is used to represent a torque of at least one joint of each leg of the robot when adjusting the center of gravity of the robot; a third acquisition module, configured to acquire third posture information of the robot; the third posture information includes third rotation angle information and third joint angle information; the third rotation angle information is used to indicate the rotation angle corresponding to the robot after adjusting the center of gravity of the robot; the third joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot after adjusting the center of gravity of the robot; a fourth determination module, configured to determine a second foot-end force of each leg of the robot based on the third rotation angle information; the second foot-end force is used to represent a force corresponding to at least one joint of each leg after adjusting the center of gravity of the robot; a fifth determination module, configured to determine a third action of the robot based on the second foot-end force and the third joint angle information; the third action is used to represent the torque of at least one joint of each leg of the robot when restoring the posture of the robot from a falling posture to a semi-inclined posture; A sixth determining module, configured to determine an action sequence in the preset strategy based on the third action; The sixth determination module is specifically used for: Based on the third action, fourth posture information of the robot is obtained; the fourth posture information includes fourth rotation angle information and fourth joint angle information; the fourth rotation angle information is used to indicate the rotation angle corresponding to the semi-tilted posture of the robot; the fourth joint angle information is used to indicate the joint angle corresponding to at least one joint of each leg of the robot when the robot is in the semi-tilted posture; Based on the fourth rotation angle information, determining a third foot-end force of each leg of the robot; the third foot-end force is used to represent a force corresponding to at least one joint of each leg when the robot is in a semi-tilted posture; Based on the third foot-end force and the fourth joint angle information, determining a fourth action of the robot; the fourth action is used to represent the torque of at least one joint of each leg of the robot when restoring the posture of the robot from a semi-tilted state to a standing state; Based on the third action, the fourth action and the second action, an action sequence in the preset strategy is determined.
4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the fall self-recovery learning method as described in claim 1 or 2 is implemented.
5. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the fall self-recovery learning method as claimed in claim 1 or 2 is implemented.
6. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the fall self-recovery learning method as claimed in claim 1 or 2 is implemented.
