Adaptive gait planning method for biped robot on complex terrain and biped robot

Abstract

The application relates to a complex terrain adaptive gait planning method for a biped robot and the biped robot, and belongs to the technical field of robots, and comprises the following steps: a fractional order PID controller is constructed, the fractional order PID controller comprises a multi-sensor network module, an adaptive module and a fractional order calculus control module; a reinforcement learning model for gait planning of the biped robot is constructed, a state, an action, a decision and a reinforcement learning reward function of the reinforcement learning are set, a policy gradient method is used to continuously iterate and optimally adjust the reinforcement learning model, policy parameters are updated, and control parameters of the biped robot are output; and the control parameters output by the fractional order PID controller and the reinforcement learning model are decisionally fused through a comprehensive loss function, so that globally optimal control parameters of the biped robot are obtained. The application can provide more accurate gait planning for the biped robot in complex terrain, and better walking flexibility and adaptability are achieved.

Description

Technical Field

[0001] The present invention relates to the field of robotics, and in particular to a complex terrain adaptive gait planning method for a bipedal robot and the bipedal robot. Background Art

[0002] For model-based control of bipedal robots, their motion has traditionally been modeled through concepts of gait stability, such as the ZMP criterion, which can be implemented using a reduced-order model of the robot. While these approaches can effectively control flat-legged humanoid robots, their walking styles tend to be conservative. Alternatively, techniques based on Hybrid Zero Dynamics (HZD) can also utilize a full-order model of the robot and then generate stable periodic walks based on input-output linearization. However, HZD-based 3D robot controllers typically require extensive parameter tuning in both simulation and the real world and have difficulty adapting to environmental changes.

[0003] Existing gait planning methods for bipedal robots can achieve good results on flat ground, but on inclined and uneven roads, they are prone to problems such as unstable center of gravity and inability to adapt to tilt, causing the robot to become unstable or even fall. In complex terrain environments, the robot's gait planning needs to comprehensively consider multiple factors (terrain characteristics, obstacles, the robot's own state, etc.), resulting in a significant increase in the algorithm's computational complexity. Existing algorithms cannot respond to environmental changes in real time and are therefore unable to adapt to complex terrain. Currently, various advanced gait planning algorithms represented by deep reinforcement learning have problems such as excessive computational complexity. In particular, they suffer from low computational efficiency in optimizing gait strategies. This is especially true in complex terrain environments, where robots must make correct choices within a limited time. The shortcomings of existing gait planning algorithms are even more obvious.

[0004] At the same time, existing technologies often suffer from inaccuracies in bipedal robots' sensor data, which can affect gait planning accuracy, especially in complex terrain or low-visibility conditions, leading to erroneous decisions. Furthermore, existing gait planning methods lack the ability to automatically adjust gait parameters to environmental changes. This makes it difficult for bipedal robots to maintain a stable gait plan on complex surfaces, potentially leading to falls or slips. Summary of the Invention

[0005] In order to solve at least one of the above technical problems, the application provides a complex terrain adaptive gait planning method for a biped robot and the biped robot. The application uses a fractional order PID controller with higher freedom and better dynamic performance for the control of the biped robot of a complex and nonlinear system, combines a deep reinforcement algorithm optimization strategy, and performs decision-level fusion of the fractional order calculus control and the reinforcement learning algorithm based on a loss function, so that the biped robot can generate stable walking behaviors suitable for various complex terrains.

[0006] The technical solutions adopted by the application are as follows:

[0007] The complex terrain adaptive gait planning method for the biped robot comprises the following steps:

[0008] Step S1: constructing a fractional order PID controller, wherein the fractional order PID controller comprises a multi-sensor network module, an adaptive module and a fractional order calculus control module, the multi-sensor network module fuses and encodes and compresses external environment data collected by multiple sensor nodes on the biped robot, generates an external vector and transmits the external vector into the adaptive module, the adaptive module online estimates a current external vector according to a history state sequence and a history action sequence from proprioception, outputs a corresponding control component, and the fractional order calculus control module finely adjusts the control component obtained by the adaptive module and finally outputs a control parameter of the biped robot;

[0009] Step S2: constructing a reinforcement learning model for gait planning of the biped robot, setting a state, an action, a decision and a reinforcement learning reward function of the reinforcement learning, and using a policy gradient method to continuously iterate and optimize the reinforcement learning model, update a policy parameter, and output a control parameter of the biped robot, wherein an expression of the reinforcement learning reward function is as follows:

[0010] R=R stability +R energy +R speed +R smoothness +R fall (18)

[0011] wherein R stability is a walking stability reward function, R energy is an energy efficiency reward function, R speed is a walking speed reward function, R smoothness is a gait smoothness reward function, and R fall is a fall penalty function;

[0012] Step S3: weighting and fusing the control parameters of the biped robot output by the fractional order PID controller and the reinforcement learning model, and the fused control signal is as follows:

[0013] u(t) = b1 u fractional (t) + b2 u RL (t) (22)

[0014] Wherein, u fractional (t) is the control parameter of the fractional order PID controller output, u RL (t) is the control parameter of the reinforcement learning model output, b1 and b2 are weight parameters;

[0015] Set the comprehensive loss function L, the control parameters output by the fractional order PID controller and the reinforcement learning model are decision fusion through the comprehensive loss function, and the globally optimal biped robot control parameter is obtained, and the expression of the comprehensive loss function L is as follows:

[0016] L = a1 u fractional (t) + a2 u RL (t) + a3 u(t) (23)

[0017] Wherein, a1, a2 and a3 are weight coefficients.

[0018] The beneficial effects of the present application: the external sensor node obtains the external environment data into the fractional order PID controller, the characteristics of fractional calculus can better process complex and nonlinear data, the external environment data is accurately processed, and the parameters obtained by reinforcement learning and the parameters obtained by fractional order PID controller are fused and decided by using the loss function, which provides more accurate gait planning for biped robots in complex terrain, and achieves better walking flexibility and adaptability.

[0019] Compared with the existing method, the adaptive module of the external vector can better, faster and more accurately monitor the external environment and input it into the controller, so as to better process nonlinear data and better adjust the gait of the biped robot. The loss function can better make the biped robot select the appropriate gait in different terrain environments and achieve more accurate gait control. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The flowchart of the biped robot complex terrain adaptive gait planning method described in the embodiment of the present application. DETAILED DESCRIPTION

[0021] The technical solutions of the present application will be described in detail below with reference to the preferred embodiments and the accompanying drawings.

[0022] Referring to Figure 1The embodiment provides a biped robot complex terrain adaptive gait planning method based on fractional calculus and reinforcement learning.

[0023] Step S1: constructing a fractional order PID controller;

[0024] Step S2: constructing a reinforcement learning model for biped robot gait planning;

[0025] Step S3: designing a comprehensive loss function to fuse the decision-making of fractional calculus control and reinforcement learning, and realizing more natural human gait walking. It should be pointed out that the step labels above do not necessarily represent the order of implementation of each step, and the order of step S1 and step S2 can be changed, and the technical solution after the change still belongs to the protection scope of the present application.

[0026] In the embodiment, step S1 first constructs a fractional order proportional integral differential (PID) controller, which includes a multi-sensor network module for generating a basic control component of a target joint node of a biped robot, an adaptive module for continuously estimating an external vector using proprioception, and a fractional order calculus control module. First, the external environment data collected by the multi-sensor nodes on the biped robot is fused and encoded and compressed by the multi-sensor network module to generate an external vector, and then the external vector is transmitted to the evaluation model in the adaptive module. The adaptive module estimates the current external vector online according to the historical state sequence and historical action sequence from proprioception, and outputs the corresponding control component. However, a single evaluation model cannot achieve the expected resolution of the control component, and can only adjust integers, so a fractional order calculus control module is introduced to enable the control component to adjust a fractional order or a larger range.

[0027] In a complex environment, the robot will receive a series of special information, so-called "outside" information. These information include environmental parameters such as terrain friction coefficient, mass distribution, inertia, which have important influence on the walking of the robot. On this basis, the embodiment adopts a multi-sensor network module, which fuses the external environment data collected by multiple sensor nodes on the biped robot based on a convolutional neural network model. These fused external information is encoded into a compressed external vector, and then the obtained external vector is transmitted to the adaptive module. The adaptive module analyzes the difference between the robot instructions and the real joint position to infer the specific information of the external environment, and estimates the current external vector using the action and state information of the robot in the past period of time.

[0028] However, the accuracy of the evaluation model of the adaptive module is not high enough because part of the external information may not be completely restored in the observed state. On this basis, a fractional calculus control module is introduced to fine-tune the control component.

[0029] The control component π takes two parameters as input, which are the current state x t and the current external vector z t ∈R 3 , where R 3 is a 3-dimensional real vector containing specific information about the environment to help the robot quickly adapt to different walking environments. Then the next action a t is predicted as shown in equation (2). The predicted action a t is the target position of the 10 driving joints of the robot, which is converted into torque using a fractional order PID controller. The environment vector e t includes friction, inertia and center of mass of the robot, etc. But the external vector z t only retains the information from the environment vector e t that is needed for the control component to adapt.

[0030] z t =μ(e t ) (1)

[0031] a t =π(x t ,a t-1 ,g t ,z t ) (2)

[0032] where a t-1 is the last action, g t is the real inclination angle of the ground, and μ is a rule-based control strategy.

[0033] Multilayer Perceptron (MLPs) is composed of multiple layers, including input layer, multiple hidden layers and output layer. Each layer is composed of multiple neurons, and the neurons are connected through weights. μ and control component π are transmitted into MLPs, and they are jointly trained end-to-end using reinforcement learning to maximize the expected return of control component π as follows:

[0034]

[0035] where τ = {(x0, a0, r0), (x1, a1, r1),...} represents the trajectory of the robot when executing the control component p, x0 is the specific situation of the environment at this time or the perception input of the robot, a0 is the action behavior selected in this state, r0 is the feedback obtained from the environment after executing the action, x1 is the state of the next time step, a1 is the action behavior of the next time step, and r1 is the reward feedback of the next time step; p(τ|p) represents the possibility of the trajectory under the control component p; is the expected value symbol, representing the long-term average value of a certain random variable; γ is a time-dependent discount factor used in reinforcement learning to measure the importance of future rewards relative to current rewards; r t is the immediate reward obtained at time step t; T is the total number of time steps of the training period.

[0036] Adaptive module for estimating external environment According to the history of proprioception (x t-k:t-1 , a t-k:t-1 ) online estimate the external vector Estimated external vector is given by:

[0037]

[0038] where x t-k:t-1 is the state sequence from time step t-k to time step t-1, a t-k:t-1 is the action sequence from time step t-k to time step t-1, and φ() is the adaptive module function.

[0039] The adaptive module can be trained in simulation by supervised learning to minimize:

[0040]

[0041] where z t = μ(e t ), the adaptive module φ is modeled as a 1-D CNN to capture temporal correlation.

[0042] Fractional calculus is an extension of traditional integer-order calculus, which can better describe the dynamic behavior of systems. Compared with integer-order controllers, fractional-order controllers have more degrees of freedom and better dynamic characteristics, and are particularly suitable for modeling and control of complex nonlinear systems. Fractional calculus control is divided into the following steps:

[0043] 1.1. Dynamics modeling of biped robot:

[0044] The dynamics equation of a biped robot with n joints is obtained by Lagrange equation:

[0045]

[0046] where q is the n-dimensional joint position vector, and are the velocity and acceleration vectors, respectively, M(q) is the n x n mass matrix, is the Coriolis and centrifugal force matrix, G(q) is the gravity vector, is the control input torque, F ext is the external disturbance torque.

[0047] 1.2. Dynamic system introduces fractional calculus:

[0048] In the traditional dynamics equation, the velocity and acceleration are usually integer order derivatives. In order to better capture the complexity and memory effect of the system, fractional order derivatives are introduced here, and the definition of Caputo fractional derivative is adopted here:

[0049]

[0050] where α is the order of the derivative, that is, α is a non-negative real number, n is the smallest positive integer such that n-1 < α ≤ n, used to determine the integer order derivative required for fractional order derivative, Γ(·) is the Gamma function, which is an extension of the factorial function, f (n) (τ) is the nth derivative of the function f(τ) with respect to τ, τ is the integral variable, and is integrated from 0 to t.

[0051] 1.3. Establish fractional order dynamics equation:

[0052] Introduce fractional calculus into the dynamics model of biped robot, assume that the velocity and acceleration part adopts fractional order form, the fractional order dynamics equation of biped robot can be expressed as:

[0053]

[0054] where, is the fractional derivative of the joint position vector q(t), the order is α, is the fractional derivative of the joint acceleration vector, the order is β, generally β = α + 1, α > 0.

[0055] 1.4. Fractional calculus control module:

[0056] The fractional calculus control module is mainly divided into three parts: proportional (P) control module, fractional integral (I λ ) control module, fractional differential (D μ ) control module.

[0057] 1.4.1. Proportional (P) control module:

[0058] The proportional control module generates a control signal in response to the current error value (the difference between the set value and the actual value) to make timely corrections to the system's condition. For biped robots, the proportional control module is used to adjust the errors generated during walking, such as deviations from the predetermined track, in real time. The transfer function of the proportional control module is:

[0059] C P (s) = K p · e(t) (9)

[0060] where K p is the proportional gain, and e(t) is the error signal.

[0061] 1.4.2. Fractional-order integral (I λ ) control module:

[0062] The fractional-order integral control module eliminates the steady-state error of the system by integrating the error at each time. The use of fractional-order integral control provides greater flexibility to the system. The fractional-order integral control module can better address the continuous errors that occur during the walking process of a biped robot, such as forward movement and turning. The adjustment of the fractional order λ provides more precise control over the accumulation speed of the error. The transfer function of the fractional-order integral control module is:

[0063]

[0064] where λ is the fractional order of integration, K i is the integral gain, and s is a complex variable in Laplace transform, commonly used to describe the frequency domain behavior of linear time-invariant systems.

[0065] 1.4.3. Fractional-order derivative (D μ ) control module:

[0066] The fractional-order derivative control module is used to control the rate of change of the error to predict the dynamic changes of the system and reduce overshoot or oscillation. At the same time, the use of fractional-order derivative control improves the dynamic characteristics of the system. For biped robots, the use of fractional-order derivative control allows for better capture of the robot's gait and posture and more stable adjustment, avoiding situations such as shaking and falling during walking. The transfer function of the fractional-order derivative control module is:

[0067] C D (s) = K d · s u (11)

[0068] where u is the fractional order of differentiation, K d is the differential gain.

[0069] The overall transfer function of the fractional calculus control module can be expressed as:

[0070]

[0071] where K p , K i , K d is also called the control gain.

[0072] The fractional calculus control module is composed of three core parts: proportion, fractional integral, and fractional derivative. By introducing fractional order integral and derivative operations, the control parameters of the biped robot are fine-tuned more accurately, and the control parameters of the biped robot are finally output.

[0073] In step S2, a reinforcement learning model for biped robot gait planning is constructed, the state, action, decision, and reinforcement learning reward function of reinforcement learning are set, and the policy gradient method is used to continuously iterate and optimize the reinforcement learning model, update the policy parameters, and finally output the control parameters of the biped robot.

[0074] Reinforcement learning enables the biped robot to learn multiple gaits in complex terrains and environments. Through interaction with the environment, the robot can optimize its gait strategy to maintain balance and stability. Step S2 mainly includes the following steps:

[0075] 2.1. Set the state and action space of the reinforcement learning model:

[0076] Let the state space be S and the action space be A. At each time step t, the agent observes the state s t and takes action a t to maximize the cumulative reward R.

[0077] 2.2. Establish a reinforcement learning reward function for biped robot gait planning, which needs to consider multiple factors to ensure that the robot can learn stable and efficient gaits. The reinforcement learning reward function mainly consists of the following key components:

[0078] 2.2.1. Walking stability reward

[0079] Ensuring that the robot can maintain balance and stability while walking is the basic goal of gait planning. The walking stability reward function is:

[0080] R stability = -λ1 × CoM deviation -λ2 × angulardeviation (13)

[0081] where CoM deviation is the distance of the center of mass from the body's centerline, keeping the center of mass stable can reduce the risk of falling; angular deviation is the angle between the robot's body centerline and the ground, maintaining an upright posture can help biped robots walk more smoothly; λ1 and λ2 are weight coefficients.

[0082] 2.2.2. Energy efficiency reward

[0083] Encourage the robot to reduce energy consumption as much as possible when walking, improve the economy of walking, and the energy efficiency reward function is:

[0084] R energy = -λ3 x energy consumption (14)

[0085] where energy consumption is the energy consumption of the robot at each time, this value can be calculated according to the joint torque and walking distance completed by the robot; λ3 is the weight coefficient.

[0086] 2.2.3. Walking speed reward

[0087] Encourage the robot to maintain the correct pace, not too slow or too fast, and the walking speed reward function is:

[0088]

[0089] where v actual is the current walking speed of the robot, v target is the predetermined target walking speed, and λ4 is the weight coefficient. Reward the robot for walking close to the target speed.

[0090] 2.2.4. Gait smoothness reward

[0091] Reward the robot to produce a smooth and natural gait to avoid sudden or intermittent motion. The gait smoothness reward function is:

[0092] R smoothness = -λ5 x ∑ i (Δu i ) 2 (16)

[0093] where u i is the change in torque, angle, etc. control signal of each movement pair in adjacent time steps. Subtle changes indicate smoother walking. λ5 is the weight coefficient.

[0094] 2.2.5. Fall penalty

[0095] The robot is punished when it falls or shows obvious unstable behavior, and is forced to avoid unstable gait. The fall punishment function is:

[0096]

[0097] where λ6is a weight coefficient.

[0098] The above functions are combined into a comprehensive reward function, which is the reinforcement learning reward function:

[0099] R = R stability + R energy + R speed + R smoothness + R fall (18)

[0100] The weight coefficients λ1, λ2, …, λ6can be adjusted according to the priority of different tasks.

[0101] 2.3. Policy optimization:

[0102] The policy gradient method is used to optimize and adjust the reinforcement learning model: first, initialize the policy function (π θ (a|s)), then run the simulation under the current policy, collect state-action pairs (s t ,a t ) and corresponding rewards R t , and use the collected samples to calculate the policy gradient, and update the policy parameters according to the following formula:

[0103]

[0104] where α is the learning rate, θ is the weight of the control policy model, is the core of optimization, and J(θ) is the goal of policy optimization, which is usually the expected cumulative reward.

[0105] Further, based on the Actor-Critic network architecture, first estimate the value V t (s φ ) of each state S t using the Critic network, and calculate the advantage function through the TD error:

[0106] A(s t ,a t ) = r t + γV φ (s t+1 ) - V φ (s t ) (20)

[0107] where A(s t , a t ) represents the state s t under action a t , r t is the immediate reward obtained by the robot from the environment at time step t, and γ is a value between 0 and 1, indicating the discount rate of future rewards.

[0108] Further, the proximal policy optimization (PPO) algorithm is used to stabilize policy updates. PPO limits the magnitude of each update by introducing a clipping range, preventing the update of policy parameters from being too large and causing instability. The objective function of the PPO algorithm is:

[0109]

[0110] where V (s ) is the estimated advantage function; ∈ is the clipping range;

[0111] The entropy of the current policy at time step t is denoted by H θ (a t |s t ), which measures the randomness or uncertainty of the policy π t (a t |s θ ) in selecting action a t in state s t . The probability distribution of the robot selecting action a t according to the current state s t is controlled by the parameter θ.

[0112] In step S3, the fractional calculus control and reinforcement learning are combined through a loss function, providing an effective strategy that combines classical control theory with modern intelligent methods. Fractional calculus control has advantages in dealing with complex dynamics, memory, etc., while reinforcement learning learns the optimal strategy autonomously based on interaction with the outside world. Combining these two methods can be evaluated comprehensively using a loss function, making it more intelligent. By combining the fractional PID controller and reinforcement learning model outputs through a comprehensive loss function, the globally optimal biped robot control parameters are obtained, achieving efficient and stable dynamic adaptive gait planning for biped robots on complex terrain. The following are the specific steps and methods:

[0113] 3.1. Fusion of fractional calculus control and reinforcement learning:

[0114] A parallel fusion method is used to realize the fusion of fractional calculus control and reinforcement learning. First, the fractional PID controller and reinforcement learning are run independently, each calculating control signals and action strategies. Then, the control parameters output by the fractional PID controller and reinforcement learning are fused by weighting. The fused control signal can be represented by the following formula:

[0115] u(t) = β1·u fractional (t) + β2·u RL (t) (22)

[0116] Where u fractional (t) is the control parameter output by the fractional PID controller, u RL (t) is the control parameter output by the reinforcement learning model, and β1 and β2 are weight parameters.

[0117] 3.2. Set the comprehensive loss function:

[0118] The loss function is an important indicator of system performance. Combining fractional calculus with reinforcement learning makes it have better global optimality. Combining the above objectives, a comprehensive loss function can be defined, as shown in the following formula:

[0119] L = α1·u fractional (t) + α2·u RL (t) + α3·u(t) (23)

[0120] Where α1, α2, and α3 are weight coefficients.

[0121] The output of fractional calculus control and reinforcement learning is fused by a comprehensive loss function to realize the respective advantages of fractional differential control and reinforcement learning, and improve the control effect and intelligence level of the system.

[0122] The external environment data obtained by the external sensor node is transmitted into the fractional order PID controller, the characteristics of the fractional calculus that can better process complex and nonlinear data are utilized to accurately process the external environment data, and the parameters obtained by the reinforcement learning and the parameters obtained by the fractional order PID controller are fused and decided by the loss function, so that more accurate gait planning is provided for the biped robot in the complex terrain, and better walking flexibility and adaptability are achieved.

[0123] Compared with the existing method, the adaptive module of the external vector can better, faster and more accurately monitor the external environment and input it into the controller, so as to better process nonlinear data and better adjust the gait of the biped robot. The loss function can better make the biped robot select a suitable gait in different terrain environments and achieve more accurate gait control.

[0124] The technical features of the above-described embodiments can be combined arbitrarily, and to make the description concise, all possible combinations of the technical features in the above-described embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0125] The above-described embodiments only express several embodiments of the present application, the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of variations and improvements can be made, which belong to the protection scope of the present application. Therefore, the protection scope of the present application patent should be subject to the appended claims.

Claims

1. A method for adaptive gait planning for bipedal robots on complex terrains, characterized by: The following steps are involved: Step S1: Constructing a fractional-order PID controller, the fractional-order PID controller includes a multi-sensor network module, an adaptive module, and a fractional-order calculus control module. The multi-sensor network module fuses and encodes the external environment data collected by multiple sensor nodes on the bipedal robot to generate an external vector and transmits it to the adaptive module. The adaptive module estimates the current external vector online based on the historical state sequence and historical action sequence sensed from the proprioception, and outputs the corresponding control component. The fractional-order calculus control module finely adjusts the control component obtained by the adaptive module and finally outputs the control parameters of the bipedal robot. Step S2: Construct a reinforcement learning model for bipedal robot gait planning, set the reinforcement learning state, action, decision and reinforcement learning reward function, and use the policy gradient method to continuously iterate and optimize the reinforcement learning model, update the policy parameters, and output the control parameters of the bipedal robot. The expression of the reinforcement learning reward function is as follows: R=R stability +R energy +R speed +R smoothness +R fall (18) Among them, R stability is the walking stability reward function, R energy is the energy efficiency reward function, R speed is the walking speed reward function, R smoothness is the gait smoothness reward function, R fall is the fall penalty function; Step S3: weighted fusion of the biped robot control parameters output by the fractional-order PID controller and the reinforcement learning model. The fused control signal is: u(t)=β1·u fractional (t)+β2·u RL (t) (22) Among them, u fractional (t) is the control parameter output by the fractional-order PID controller, u RL (t) is the control parameter output by the reinforcement learning model, and β1 and β2 are weight parameters; A comprehensive loss function L is set, and the control parameters output by the fractional-order PID controller and the reinforcement learning model are fused by the comprehensive loss function to obtain the globally optimal control parameters of the biped robot. The expression of the comprehensive loss function L is as follows: L=α1·u fractional (t)+α2·u RL (t)+α3·u(t) (23) Among them, α1, α2 and α3 are weight coefficients.

2. The complex terrain adaptive gait planning method for a bipedal robot according to claim 1, characterized in that: The control component is based on the current state x t and the current external vector z t Two parameters are used as input to predict the next action a t , which can be expressed as: With t =μ(e t ) (1) a t =π(x t ,a t-1 ,g t ,z t ) (2) Among them, a t-1 For the previous action, g t is the ground true tilt angle, e t is the environment vector, and μ is the rule-based control strategy.

3. The complex terrain adaptive gait planning method for a bipedal robot according to claim 2, characterized in that: The predicted next action a t is the target position of the 10 driven biped robot joints, the environment vector e t This includes friction, the robot's inertia, and its center of mass.

4. The complex terrain adaptive gait planning method for a bipedal robot according to claim 2 or 3, characterized in that: The adaptive module is trained via supervised learning to minimize: in, is the estimated current external vector, x t-k：t-1 is the state sequence from time step tk to time step t-1, a t-k：t-1 is the action sequence from time step tk to time step t-1, and φ() is the adaptive module function.

5. The complex terrain adaptive gait planning method for a bipedal robot according to claim 1, characterized in that: The fractional-order calculus control module includes a proportional control module for real-time adjustment of the error generated when the biped robot walks, a fractional-order integral control module for integrating the error at each moment, and a fractional-order differential control module for controlling the rate of change of the error.

6. The complex terrain adaptive gait planning method for a bipedal robot according to claim 5, characterized in that: The transfer function of the proportional control module is: C P (s)=K p ·e(t) (9) The transfer function of the fractional-order integral control module is: The transfer function of the fractional-order differential control module is: C D (s)=K d ·s u (11) The overall transfer function of the fractional-order calculus control module is: Among them, K p , K i , K d are proportional gain, integral gain, and differential gain respectively, e(t) is the error signal, s is the complex variable in Laplace transform, λ is the fractional order of integration, and u is the fractional order of differentiation.

7. The complex terrain adaptive gait planning method for a bipedal robot according to claim 1, characterized in that: The policy gradient method uses the Actor-Critic network architecture for optimization.

8. The complex terrain adaptive gait planning method for a bipedal robot according to claim 1, characterized in that: The policy gradient method uses a proximal policy optimization algorithm to stabilize policy updates. The objective function of the proximal policy optimization algorithm is: in, is the estimated advantage function; ∈ is the shear range; Represents the entropy of the current strategy at time step t, which measures the strategy π θ (a t |s t ) in state s t Next select action a t The randomness or uncertainty of θ (a t |s t ) is the robot according to the current state s t Select action a t The probability distribution of , which is controlled by the parameter θ.

9. The complex terrain adaptive gait planning method for a bipedal robot according to claim 1, characterized in that: R stability =-λ1×COM deviation -λ2×angular deviation (13) Among them, CoM deviation is the distance that the center of gravity deviates from the body's central axis, angular deviation is the angle between the robot's body axis and the ground, λ1 and λ2 are weight coefficients; R energy =-λ3×energy consumption (14) Among them, energy consumption is the energy consumption of the robot at each moment, λ3 is the weight coefficient; Among them, v actual is the current walking speed of the robot, v target is the predetermined target walking speed, λ4 is the weight coefficient; R smoothness =-λ5×∑ i (D) i ) 2 (16) Among them, u i It refers to the change of the control signal of each kinematic pair in adjacent time steps, and λ5 is the weight coefficient; Among them, λ6 is the weight coefficient.

10. A bipedal robot, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, the method for complex terrain adaptive gait planning of a bipedal robot according to any one of claims 1 to 9 is implemented.

Images