Alligator-imitated robot gait control method

Abstract

The invention belongs to the technical field of foot-type robot gait control, and particularly discloses an alligator-imitated robot gait control method, which comprises the following steps: acquiring state data and terrain depth map data of an alligator-imitated robot; inputting the obtained data into a gait control model to obtain a predicted gait; controlling the crocodile-imitating robot to execute gait motion based on the predicted gait, and recording a motion track of the crocodile Sampling track data to iteratively update parameters of the gait control model; wherein the gait control model outputs a diffusion strategy gait through a diffusion actor-commentator network, generates a prior gait through a central pattern generator, and fuses the diffusion strategy gait and the prior gait through a weight network to obtain a predicted gait; the method effectively solves the problems that a traditional alligator-imitated robot gait control algorithm based on reinforcement learning is insufficient in action exploration ability, low in convergence speed and insufficient in strategy generalization ability under complex terrains.

Description

Technical Field

[0001] This invention belongs to the field of legged robot gait control technology, specifically relating to a gait control method for an alligator-like robot. Background Technology

[0002] Crocodile-like robots have unique advantages in complex terrain scenarios such as space exploration and power line inspection due to their discrete footholds, autonomous obstacle crossing, and low requirements for ground flatness. Gait control of crocodile-like robots is an important prerequisite for realizing robot motion control.

[0003] In the prior art, the gait control algorithm of legged robots usually adopts reinforcement learning-based gait control algorithms such as SAC algorithm and PPO algorithm. The above methods can effectively deal with the feature representation problem of high-dimensional continuous state-action space through end-to-end perception-decision mechanism. Moreover, based on the Markov decision process and continuous interaction with the environment to learn the optimal strategy, it shows good adaptability to unstructured terrain. Theoretically, the optimal gait controller for a specific terrain can be obtained through the same learning process.

[0004] However, the aforementioned methods still have significant limitations in gait control in complex terrain: the policy functions of these methods mostly adopt Gaussian policy distribution and multilayer perceptron architecture. Among them, the Gaussian policy distribution is a unimodal policy distribution, which is insufficient in exploring actions in complex terrain and is prone to getting trapped in local optima; the fully connected structure of the multilayer perceptron is difficult to effectively capture the complex nonlinear relationship between the robot's state and actions in gait control in complex terrain, and its feature extraction capability is limited under the architecture of directly outputting actions, resulting in suboptimal policy expression. These architectural deficiencies not only lead to slow algorithm convergence speed, but also to problems such as poor policy generalization ability and weak adaptability, making it difficult to meet the requirements of alligator-like robots for real-time performance and stability in gait decision-making in complex terrain environments. Summary of the Invention

[0005] The purpose of this invention is to provide a gait control method for crocodile-like robots, in order to solve the technical problems in the existing technology where reinforcement learning-based gait control algorithms for crocodile-like robots have insufficient action exploration capabilities in complex terrain, suboptimal policy expression, slow convergence speed, and poor generalization ability, and thus cannot meet the requirements of crocodile-like robots for real-time performance and stability in complex terrain environments.

[0006] To achieve the above objectives, the present invention provides a gait control method for a crocodile-like robot, comprising:

[0007] Acquire state data and terrain depth map data of the alligator-like robot;

[0008] The state data and terrain depth map data of the crocodile-like robot are input into the gait control model to obtain the predicted gait. The gait control model includes a diffusion actor-commentator network, a weighted network, and a central pattern generator. The gait control model outputs a diffusion strategy gait through the diffusion actor-commentator network, generates a prior gait through the central pattern generator, and then the weighted network fuses the diffusion strategy gait and the prior gait to obtain the predicted gait. The diffusion actor-commentator network uses the diffusion model as its actor network. The diffusion strategy gait includes the diffusion strategy rotation angles of the hip joint and knee joint of the crocodile-like robot. The prior gait includes the prior rotation angles of the hip joint and knee joint of the crocodile-like robot. The predicted gait includes the predicted rotation angles of the hip joint and knee joint of the crocodile-like robot.

[0009] Based on the predicted gait control, the alligator-like robot performs gait movements, the trajectory data of the alligator-like robot is collected, and the parameters of the gait control model are iteratively updated.

[0010] The trajectory data includes the current state data, the predicted gait action at the current moment, the action reward at the current moment, and the state data for the next moment.

[0011] Furthermore, the process of generating the diffusion strategy gait through the diffusion actor-critic network includes:

[0012] The diffusion actor-critic network includes a noise prediction network;

[0013] Based on the state data and terrain depth map data of the crocodile-like robot, as well as random Gaussian noise, a noise prediction network is used to obtain predicted Gaussian noise; wherein, the random Gaussian noise is randomly sampled from the Gaussian distribution N(0,I); based on the predicted Gaussian noise and the random Gaussian noise, the diffusion model performs a reverse denoising process step by step to obtain the diffusion strategy gait action;

[0014] Repeat the process k times to obtain k gait actions for the diffusion policy. Select the action sample with the highest Q value as the output gait of the diffusion policy, as shown in the following expression:

[0015]

[0016] In the formula, Q(s,a) represents the action value of the diffusion strategy gait.

[0017] Furthermore, the process of generating the diffusion strategy gait through the diffusion actor-critic network includes:

[0018] Define the reward function R for the diffusion actor-critic network, its expression is:

[0019] R = R f +R e +R stable (2)

[0020] In the formula, R f R represents positive motivational reward. e R represents the energy consumption penalty. stable This indicates a stability bonus.

[0021] Furthermore, the process of generating prior gait via the central pattern generator includes:

[0022] Four rhythmic signals with different phase differences are generated by four Hopf harmonic oscillators;

[0023] The four rhythmic signals with different phase differences are coupled by using a coupled oscillator model to obtain four coupled rhythmic signals with different phase differences.

[0024] The prior gait of the crocodile-like robot is obtained by mapping and calculating the four rhythmic signals with different phase differences after coupling.

[0025] Furthermore, the expression for the coupled oscillator model is:

[0026]

[0027] In the formula, x and y are the angular position control signals of the hip and knee joints, respectively, μ is the square of the amplitude A, α determines the speed at which the oscillator converges to the limit cycle, and ω is the frequency of the oscillator. For rotation matrix, This represents the relative phase difference between the i-leg oscillator and the j-leg oscillator.

[0028] Furthermore, the prior gait is a diagonal trotting gait.

[0029] Furthermore, the weighted network fuses the diffusion policy gait and the prior gait to obtain...

[0030] The process of predicting gait includes:

[0031] Based on the state data of the crocodile-like robot and the terrain depth map data, the adaptive weight coefficients of the weight network are obtained.

[0032] Based on the adaptive weight coefficients of the weighted network, the predicted gait action is obtained, calculated as follows:

[0033] a hybrid =αa t +(1-α)a d (4)

[0034] In the formula, ahybrid Indicates the action for predicting gait, a t The action representing the gait of the diffusion strategy, a d α represents the action of the prior gait, and α represents the adaptive weight coefficient, α∈[0,1].

[0035] Furthermore, the training steps for the noise prediction network include:

[0036] Based on the actions in the random Gaussian noise and the sampled trajectory data, the diffusion model performs a forward diffusion process to obtain actions containing Gaussian noise;

[0037] Based on the state in the action and sampled trajectory data containing Gaussian noise, the predicted Gaussian noise of the noise prediction network is obtained;

[0038] The noise prediction network is updated by minimizing the loss between the random Gaussian noise and the predicted Gaussian noise, resulting in a trained noise prediction network. The expression for the loss function is as follows:

[0039]

[0040] In the formula, Let ω(s,a) represent the expectation operator, ω(s,a) represent the truncation dominance function, and ∈ represent random Gaussian noise. θ (x t (s,t) represents the predicted Gaussian noise, x t This indicates actions containing Gaussian noise.

[0041] Furthermore, the training steps for the weighted network include:

[0042] Based on the reward function R, the action reward of the predicted gait is obtained;

[0043] The weight network is updated to minimize the expected negative cumulative action reward of the predicted gait, resulting in a trained weight network. The training objective function of the weight network is expressed as follows:

[0044]

[0045] In the formula, R(s) t ,a hybrid ) indicates the use of fused gait a hybrid The corresponding state s t The rewards obtained below.

[0046] Furthermore, the alligator-like robot state data includes the robot's posture data, hip joint angles, and knee joint angles. The posture data includes roll angle, pitch angle, yaw angle and their rate of change, as well as the displacement of the body's center of mass in the x, y, and z axes. The alligator-like robot is a quadruped robot.

[0047] Compared with existing technologies, the present invention has the following advantages:

[0048] 1. The gait control model of this invention generates a diffusion strategy gait through the inverse denoising process of the diffusion model in a diffusion actor network. It leverages the modeling advantages of the diffusion model in complex probability distributions, exhibiting excellent representation and action exploration capabilities for complex terrain features in complex environments. The prior gait generated by the central pattern generator is a gait that simulates the rhythmic characteristics of biological movement and conforms to the laws of dynamics. Using it as a guide for the diffusion strategy gait can effectively improve the stability of the initial gait training of the gait control model and accelerate the convergence of gait training. The predicted gait obtained by adaptively fusing the prior gait and the action generated by the diffusion model through a weighted network balances the real-time performance and stability of gait decision-making for the crocodile robot in complex terrain environments, while improving the action exploration capability, convergence speed, and policy generalization capability of the crocodile robot control algorithm in complex terrain environments. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating the gait control method for the alligator-like robot provided by the present invention.

[0050] Figure 2 The block diagram of the gait control model provided by this invention.

[0051] Figure 3 This is a schematic diagram of the diffusion actor-critic network structure provided by the present invention.

[0052] Figure 4 This is a schematic diagram of the central pattern generator network structure provided by the present invention;

[0053] Figure 5 This is a schematic diagram of the crocodile-like robot model provided by the present invention crawling on unstructured terrain such as rugged and stepped terrain;

[0054] Figure 6 The simulation results of the gait control method for the crocodile robot provided by this invention are compared with those of existing methods. Detailed Implementation

[0055] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but it should be understood that the scope of protection of the invention is not limited to the specific embodiments.

[0056] refer to Figures 1 to 4 A gait control method for an alligator-like robot includes the following steps:

[0057] Step S1: Obtain the crocodile-like robot's status data and terrain depth map data;

[0058] Step S2: Input the crocodile-like robot state data and terrain depth map data into the gait control model to obtain the predicted gait;

[0059] The gait control model includes a diffusion actor-critic network, a weight network, and a central pattern generator. The gait control model outputs the diffusion strategy gait through the diffusion actor-critic network, generates the prior gait through the central pattern generator, and then the weight network fuses the diffusion strategy gait and the prior gait to obtain the predicted gait.

[0060] Among them, the diffusion strategy gait includes the diffusion strategy rotation angle of the hip joint and the diffusion strategy rotation angle of the knee joint of the crocodile robot.

[0061] Prior gait includes the prior rotation angles of the hip joint and the prior rotation angles of the knee joint of the crocodile robot.

[0062] The predicted gait includes the predicted rotation angles of the hip joint and the knee joint of the alligator-like robot.

[0063] Step S3: Based on the predicted gait control, the crocodile-like robot performs gait motion, collects the trajectory data of the crocodile-like robot, and iteratively updates the gait control model parameters;

[0064] The trajectory data includes the current state data, the predicted gait action at the current moment, the action reward at the current moment, and the state data for the next moment.

[0065] Specifically, in step S2, the diffusion actor-critic network includes a noise prediction network, and the process of generating the diffusion strategy gait through the diffusion actor-critic network includes the following steps:

[0066] Step S201: Based on the state data and terrain depth map data of the crocodile-like robot, and random Gaussian noise, a noise prediction network is used to obtain predicted Gaussian noise; wherein, the random Gaussian noise is randomly sampled from the Gaussian distribution N(0,I); Step S202: Based on the predicted Gaussian noise and random Gaussian noise, the diffusion model performs a reverse denoising process step by step to obtain the diffusion strategy gait action.

[0067] Step S203: Repeat step S202 k times to obtain k diffusion policy gait actions. Select the action sample with the highest Q value as the output gait of the diffusion policy, as shown in the following expression:

[0068]

[0069] In the formula, Q(s,a) represents the action value of the diffusion strategy gait.

[0070] Step S204: Define the reward function R. The expression for the reward function R is:

[0071] R = R f +R e +R stable (2)

[0072] In the formula, R f R represents positive motivational reward. e R represents the energy consumption penalty. stable This indicates a stability bonus.

[0073] Specifically, while diffusion models can generate effective policy representations through a reverse denoising process, they also introduce significant policy variance, leading to low sample efficiency when behavioral policies interact with the environment online. Therefore, step S203 involves action selection to construct efficient behavioral policies to improve sample efficiency. The core logic is that the action sample efficiency of a behavioral policy depends on the action value Q; that is, action samples with high Q often correspond to larger potential rewards, providing more valuable feedback for policy optimization.

[0074] Specifically, in step S2, the process of generating the prior gait through the central pattern generator includes the following:

[0075] step:

[0076] Step S205: Generate four rhythmic signals with different phase differences using four Hopf harmonic oscillators;

[0077] Step S206: Couple four rhythmic signals with different phase differences using a coupled oscillator model to obtain four coupled rhythmic signals with different phase differences.

[0078] Step S207: Map and calculate the four rhythmic signals with different phase differences after coupling to obtain the prior gait of the crocodile robot.

[0079] The expression for the coupled oscillator model is as follows:

[0080]

[0081] In the formula, x and y are the angular position control signals of the hip and knee joints, respectively, μ is the square of the amplitude A, α determines the speed at which the oscillator converges to the limit cycle, and ω is the frequency of the oscillator. For rotation matrix, This represents the relative phase difference between the i-leg oscillator and the j-leg oscillator.

[0082] Specifically, the prior gait is a diagonal trotting gait with a duty cycle of 0.5, which can achieve a good balance between walking speed and stability, providing a foundation for the robot's efficient and stable movement.

[0083] In this embodiment, reference Figure 4The central pattern generator generates periodic signals to drive robot movement by simulating the structure of a biological neural network, thereby enabling the bionic robot to achieve natural and stable gait control. The Hopf harmonic oscillator generates periodic oscillation signals based on nonlinear differential equations, which facilitates modeling analysis and stability assessment. It also features a simple model structure and adjustable parameters (such as amplitude, phase, and frequency), and can independently control the support phase and oscillation phase during motion.

[0084] Specifically, in step S2, the process of fusing the diffusion strategy gait and the prior gait to obtain the predicted gait includes the following steps:

[0085] Step S208: Based on the state data of the crocodile-like robot and the terrain depth map data, obtain the adaptive weight coefficients of the weight network;

[0086] Step S209: Based on the adaptive weight coefficients of the weight network, obtain the predicted gait action and calculate...

[0087] The formula is as follows:

[0088] a hybrid =αa t +(1-α)a d (4)

[0089] In the formula, a hybrid Indicates the action for predicting gait, a t The action representing the gait of the diffusion strategy, a d α represents the action of the prior gait, and α represents the adaptive weight coefficient, α∈[0,1].

[0090] Specifically, the training steps for the noise prediction network are as follows:

[0091] Step S201.1: Based on the random Gaussian noise and the actions in the sampled trajectory data, the diffusion model performs a forward diffusion process to obtain actions containing Gaussian noise;

[0092] Step S201.2: Based on the state in the action and sampled trajectory data containing Gaussian noise, obtain the predicted Gaussian noise of the noise prediction network;

[0093] Step S201.3: Update the noise prediction network by minimizing the loss between the random Gaussian noise and the predicted Gaussian noise, thus obtaining the trained noise prediction network. The expression for the loss function is:

[0094]

[0095] In the formula, Let ω(s,a) represent the expectation operator, ω(s,a) represent the truncation dominance function, and ∈ represent random Gaussian noise. θ(xt,s,t) represents the predicted Gaussian noise, x t This indicates actions containing Gaussian noise.

[0096] Specifically, the training steps for the weighted network include:

[0097] Step S209.1: Obtain the action reward for the predicted gait based on the reward function R;

[0098] Step S209.2: Update the weight network to minimize the expected negative cumulative action reward of the predicted gait, obtaining a trained weight network. The training objective function expression of the weight network is:

[0099]

[0100] In the formula, R(s) t ,a hybrid ) indicates the use of fused gait a hybrid The corresponding state s t The rewards obtained below.

[0101] Specifically, in step S1, the alligator-like robot's state data includes body posture data and angle data of each joint. The body posture data includes roll angle, pitch angle, yaw angle and their rate of change, as well as the displacement of the body's center of mass in the x, y, and z axes. The alligator-like robot is a quadruped robot. The terrain depth map data includes obstacle height, pit depth, slope angle, etc. In this embodiment, the alligator-like robot is a quadruped with eight degrees of freedom, with two degrees of freedom at the hip and knee joints of each leg. The alligator-like robot is equipped with a depth camera, an IMU attitude sensor, and joint sensors. The depth camera collects terrain environment information to obtain terrain depth map data, and the joint sensors collect joint angle data of the alligator-like robot, including the hip and knee joint angles of the four legs.

[0102] In specific implementation, refer to Figure 2 The terrain depth map data is processed by a CNN (Convolutional Neural Network) to obtain terrain feature vectors, namely obstacle height vectors, pothole depth vectors, and slope angle vectors. After matrix operations, the joint angle data and body posture data are mapped into the crocodile robot state vector.

[0103] Specifically, in step S2, to avoid differences in the value ranges of various feature information in the state space and action space, and to prevent the loss function from being dominated by large-scale features and causing an imbalance in the evaluation, the variables in the state space and action space are normalized. The normalization expression is as follows:

[0104]

[0105] In the formula, x is the value before normalization, x∈(xmin ,x max ), x′∈[-1,1] is the normalized value.

[0106] In specific implementation, in step S204, the energy consumption penalty R e The energy consumed by all motors within one time step, used to guide the robot to complete gait movements with low energy consumption, is calculated using the following expression:

[0107]

[0108] In the formula, i is the motor number, k e τ is the weighting parameter. i ω is the motor torque. i This represents the motor speed.

[0109] Stability Bonus R stable It is used to improve the stability of robot gait and includes three sub-items: centroid height fluctuation penalty, fall penalty, and posture angle change penalty.

[0110] In specific implementation, refer to Figure 3 In steps S2 and S3, the state space S is initially set as the crocodile-like robot's state data and terrain depth map data, and the gait of the crocodile-like robot is set as a. The training process of the diffusion actor-critic network is as follows:

[0111] 1: Initialize the parameters μ and θ of the Critic and Actor networks.

[0112] 2: Copy the network parameters to obtain the target Critic network and the target Actor network μ′←μ,θ′←θ

[0113] 3: Initialize the experience playback buffer D

[0114] 4: for episode = 1, M do

[0115] 5: For each episode, initialize the randomization process for action exploration (initialize the environment).

[0116] 6: Take the initial state s t

[0117] 7: for t=1,T do

[0118] 8: Based on the initial state s t Action a is obtained from the Actor network. t =π(s) t |θ)

[0119] 9: Perform action a t r is obtained from the reward function tAnd observe the new state s t+1

[0120] 10: The trajectory of motion (s) t a t r t s t+1 ) Stored in the experience playback buffer D

[0121] 11: Randomly sample small batches of trajectory samples (s) from D. t a t r t s t+1 N

[0122] 12: Update the Critic network:

[0123] Calculate TD_target:

[0124] Calculate TD_terror:

[0125] Update the Critic network by minimizing the loss:

[0126] 13: Update the Actor network:

[0127] x0 = a t ,t~Uniform({1,…,T}),∈~N(0,I)

[0128] Forward diffusion process:

[0129] Through Q(s) t ,a t |μ) calculates the weight ω(s,a)

[0130] Update the Diffusion network by minimizing the loss:

[0131]

[0132] Diffusion networks are incorporated into the inverse noise reduction process and combined as Actor networks.

[0133] 14: Soft update target network:

[0134] μ′ t ←τμ t +(1-τ)μ′ t

[0135] θ′ t ←τθ t +(1-τ)θ′ t

[0136] 15: end for

[0137] 16: end for

[0138] As shown above, the diffusion actor-critic network employs a diffusion model as its actor network. The diffusion model involves two processes: forward diffusion and backward denoising. The forward diffusion process gradually adds noise to the data, transforming the original data distribution into a tractable Gaussian distribution. The backward denoising process learns to inversely deduce the original data distribution from the Gaussian noise. Specifically, by training a noise prediction network, it progressively denoises from random noise to recover the original data distribution. In this process, the diffusion model demonstrates significant effectiveness in modeling complex data distributions through iterative denoising. This powerful generative capability enables it to generate effective policy representations for robot gait control in complex terrain during deep reinforcement learning through the backward denoising process, thereby guiding the robot to better explore and learn.

[0139] refer to Figure 5 and Figure 6 In specific implementation, this invention constructs a SolidWorks 3D model of a quadrupedal crocodile-like robot. This model has 8 degrees of freedom, with 2 degrees of freedom set at the hip and knee joints for each leg. The 3D model is then imported into the PyBullet simulation simulator for simulation. Figure 6In the diagram, curve 1 represents the reward curve for gait actions generated using the method of this invention; curve 2 represents the reward curve for prior gait actions generated using the Central Pattern Generator (CPG); curve 3 represents the reward curve for policy gait actions generated using an actor-critic training network with a Gaussian policy; and curve 4 represents the reward curve for policy gait actions generated using a diffusion actor-critic training network. Simulation results show that in rugged terrain, the gait control model fused with the diffusion actor-critic training network and the central pattern generator proposed in this invention improves the efficiency of initial training samples in the prior gait, leading to a rapid increase in early rewards. Later, it leverages the excellent modeling ability of the diffusion strategy for rugged terrain to achieve high convergence rewards, and has the fewest training epochs within the total training steps, demonstrating strong adaptability to rugged terrain and exhibiting the best performance. The diffusion actor-critic training network (curve 4) has low initial sample efficiency and slow reward growth, but it relies on the diffusion model to handle complex decision spaces under high-dimensional noisy input data. The diffusion strategy demonstrates its ability to capture complex nonlinear relationships and explore actions in the high-dimensional state-action space of rugged terrain, with considerable convergence rewards in the later stages and fewer training rounds than the Gaussian strategy. The actor-commentator training network (curve 3) using the Gaussian strategy suffers from low sample efficiency in the early stages and insufficient ability to model rugged terrain, resulting in slow reward growth in the early stages and low convergence rewards in the later stages. It only slightly outperforms the fixed gait strategy in terms of the number of training rounds and has weak adaptability. The central pattern generator (curve 2) uses a fixed gait and cannot update its strategy according to terrain changes. Its rewards are the lowest and almost unchanged, making it completely unadaptable to terrain changes.

[0140] In summary, the gait control method for the crocodile-like robot proposed in this invention generates a diffusion strategy gait through a diffusion model of a diffusion actor network. This leverages the modeling advantages of diffusion models in complex probability distributions, demonstrating excellent representation and action exploration capabilities for complex terrain features in complex environments. The prior gait generated by the central pattern generator simulates the rhythmic characteristics of biological movement and conforms to dynamic laws. Using this gait as a guide for the diffusion strategy gait effectively improves the stability of the initial gait training of the gait control model and accelerates gait training convergence. The predicted gait obtained by adaptively fusing the prior gait and the actions generated by the diffusion model through a weighted network balances the real-time performance and stability of gait decision-making for the crocodile-like robot in complex terrain environments, while also enhancing the crocodile-like robot control algorithm's action exploration capability, convergence speed, and policy generalization capability in complex terrain environments.

[0141] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A gait control method for an alligator-like robot, characterized in that, include: Acquire state data and terrain depth map data of the alligator-like robot; The state data and terrain depth map data of the crocodile-like robot are input into the gait control model to obtain the predicted gait. The gait control model includes a diffusion actor-commentator network, a weighted network, and a central pattern generator. The gait control model outputs a diffusion strategy gait through the diffusion actor-commentator network, generates a prior gait through the central pattern generator, and then the weighted network fuses the diffusion strategy gait and the prior gait to obtain the predicted gait. The diffusion actor-commentator network uses the diffusion model as its actor network. The diffusion strategy gait includes the diffusion strategy rotation angles of the hip joint and knee joint of the crocodile-like robot. The prior gait includes the prior rotation angles of the hip joint and knee joint of the crocodile-like robot. The predicted gait includes the predicted rotation angles of the hip joint and knee joint of the crocodile-like robot. Based on the predicted gait control, the alligator-like robot performs gait movements, the trajectory data of the alligator-like robot is collected, and the parameters of the gait control model are iteratively updated. The trajectory data includes the current state data, the predicted gait action at the current moment, the action reward at the current moment, and the state data for the next moment.

2. The gait control method for an alligator-like robot according to claim 1, characterized in that, The process of generating diffusion strategy gait via a diffusion actor-critic network includes: The diffusion actor-critic network includes a noise prediction network; Based on the state data and terrain depth map data of the crocodile-like robot, as well as random Gaussian noise, a noise prediction network is used to obtain predicted Gaussian noise; wherein, the random Gaussian noise is randomly sampled from the Gaussian distribution N(0,I); based on the predicted Gaussian noise and the random Gaussian noise, the diffusion model performs a reverse denoising process step by step to obtain the diffusion strategy gait action; Repeat the process k times to obtain k gait actions for the diffusion policy. Select the action sample with the highest Q value as the output gait of the diffusion policy, as shown in the following expression: In the formula, Q(s,a) represents the action value of the diffusion strategy gait.

3. The gait control method for an alligator-like robot according to claim 2, characterized in that, The process of generating diffusion strategy gait via a diffusion actor-critic network includes: Define the reward function R for the diffusion actor-critic network, its expression is: T=R f +R e +R stable (2) In the formula, R f R represents positive motivational reward. e R represents the energy consumption penalty. stable This indicates a stability bonus.

4. The gait control method for an alligator-like robot according to claim 1, characterized in that, The process of generating prior gait via the central pattern generator includes: Four rhythmic signals with different phase differences are generated by four Hopf harmonic oscillators; The four rhythmic signals with different phase differences are coupled by using a coupled oscillator model to obtain four coupled rhythmic signals with different phase differences. The prior gait of the crocodile-like robot is obtained by mapping and calculating the four rhythmic signals with different phase differences after coupling.

5. The gait control method for an alligator-like robot according to claim 4, characterized in that, The expression for the coupled oscillator model is: In the formula, x and y are the angular position control signals of the hip and knee joints, respectively, μ is the square of the amplitude A, α determines the speed at which the oscillator converges to the limit cycle, and ω is the frequency of the oscillator. For rotation matrix, This represents the relative phase difference between the i-leg oscillator and the j-leg oscillator.

6. The gait control method for an alligator-like robot according to claim 4, characterized in that, The prior gait is a diagonal trotting gait.

7. The gait control method for an alligator-like robot according to claim 3, characterized in that, The process by which the weighted network fuses the diffusion strategy gait and the prior gait to obtain the predicted gait includes: Based on the state data of the crocodile-like robot and the terrain depth map data, the adaptive weight coefficients of the weight network are obtained. Based on the adaptive weight coefficients of the weighted network, the predicted gait action is obtained, calculated as follows: a hybrid =aaa t +(1-a)a d (4) In the formula, a hybrid Indicates the action for predicting gait, a t The action representing the gait of the diffusion strategy, a d α represents the action of the prior gait, and α represents the adaptive weight coefficient, α∈[0,1].

8. The gait control method for an alligator-like robot according to claim 2, characterized in that, The training steps for the noise prediction network include: Based on the actions in the random Gaussian noise and the sampled trajectory data, the diffusion model performs a forward diffusion process to obtain actions containing Gaussian noise; Based on the state in the action and sampled trajectory data containing Gaussian noise, the predicted Gaussian noise of the noise prediction network is obtained; The noise prediction network is updated by minimizing the loss between the random Gaussian noise and the predicted Gaussian noise, resulting in a trained noise prediction network. The expression for the loss function is as follows: In the formula, Let ω(s,a) represent the expectation operator, ω(s,a) represent the truncation dominance function, and ∈ represent random Gaussian noise. θ (x t (s,t) represents the predicted Gaussian noise, x t This indicates actions containing Gaussian noise.

9. The gait control method for an alligator-like robot according to claim 7, characterized in that, The training steps for the weighted network include: Based on the reward function R, the action reward of the predicted gait is obtained; The weight network is updated to minimize the expected negative cumulative action reward of the predicted gait, resulting in a trained weight network. The training objective function of the weight network is expressed as follows: In the formula, R(s) t ,a hybrid ) indicates the use of fused gait a hybird The corresponding state s t The rewards obtained below.

10. The gait control method for an alligator-like robot according to claim 1, characterized in that, The alligator-like robot's state data includes the robot's posture data, hip joint angles, and knee joint angles. The posture data includes roll angle, pitch angle, yaw angle and their rate of change, as well as the displacement of the body's center of mass in the x, y, and z axes. The alligator-like robot is a quadruped robot.

Images