A hierarchical reinforcement learning motion planning and control method for a legged robot facing sparse terrain

Abstract

This invention discloses a hierarchical reinforcement learning motion planning and control method for legged robots in sparse terrain. It extracts semantically rich terrain features from elevation maps using a multi-scale perceptual encoder. These terrain features, along with the robot's real-time state and target commands, serve as input to a conditional denoising network. This network models landing point planning as a parallel, progressive denoising process, generating a sequence of robot landing points. During this process, an inverse kinematics feasibility verification layer evaluates the kinematic reachability of candidate landing points in real time, thus determining the planned landing point sequence. Based on the planned landing point sequence and the robot's real-time state, robust joint motion commands are output using reinforcement learning to control the legged robot's movements. This invention effectively solves the problems of short-sighted planning, low computational efficiency, and poor robustness in traditional methods in discontinuous, high-elevation, sparse terrain, achieving highly dynamic and stable motion of legged robots in complex environments.

Description

Technical Field

[0001] This invention belongs to the field of legged robot technology, specifically relating to a hierarchical reinforcement learning motion planning and control method for legged robots in sparse terrain. Background Technology

[0002] With the increasing demand for legged robots in complex environments such as field exploration and disaster relief, humanoid robots have attracted much attention due to their humanoid structure and operational potential. However, in sparse terrain with discontinuous sections, high elevation differences, or narrow support surfaces, robots face challenges such as difficulty in choosing footholds, complex whole-body coordination, and sensitivity to environmental disturbances. Currently, the mainstream dynamic motion schemes can be mainly divided into the following categories: Model-based optimization methods, such as nonlinear model predictive control, generate motions that satisfy dynamic constraints through trajectory optimization, but they have extremely high requirements for model accuracy and real-time perception. End-to-end reinforcement learning methods directly map sensor inputs to joint control, exhibiting good robustness but poor interpretability and difficulty in exploring sparse terrain. Hybrid approaches, which combine learning and model planning, utilize learning components to improve tracking performance or generate reference trajectories, but these systems are complex and computationally burdensome. Furthermore, existing methods are mostly designed for single robot platforms and lack generalization capabilities for legged robots with different configurations (such as quadrupeds and humanoids), and most learning frameworks fail to effectively achieve zero-shot transfer from simulation to real-world applications. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a hierarchical reinforcement learning motion planning and control method for legged robots in sparse terrain. The upper-layer network learns a forward-looking footpoint generation strategy, while the lower-layer network learns precise footpoint tracking and whole-body motion control. This method ensures the agility and robustness of the legged robot's motion, while achieving cross-platform, generalizable, and interpretable dynamic motion capabilities.

[0004] The present invention achieves the above-mentioned technical objectives through the following technical means.

[0005] A hierarchical reinforcement learning method for motion planning and control of legged robots in sparse terrain:

[0006] Upper-layer landing point generation network: Based on a multi-scale perceptual encoder, terrain features rich in semantic information are extracted from the elevation map. These terrain features, along with the robot's real-time state and target commands, serve as inputs to a conditional denoising network. This network models landing point planning as a parallel, progressive denoising process, generating a robot landing point sequence. During the landing point sequence process, the kinematic reachability of candidate landing points is evaluated in real time through an inverse kinematics feasibility verification layer, thereby determining the planned landing point sequence.

[0007] The lower-level motion control network receives the planned landing point sequence and the robot's real-time status, and outputs robust joint motion commands based on reinforcement learning to control the legged robot's movements.

[0008] Furthermore, the conditional denoising network models the landing point planning as a parallel progressive denoising process to generate a robot landing point sequence. Specifically, this includes: sampling a landing point sequence composed of standard Gaussian noise as the initial trajectory; and the conditional denoising network performing... arrive The iterative loop.

[0009] Furthermore, in each iteration In the middle, the conditional denoising network receives the current noisy landing point sequence. Current iteration step and conditional input ,predict Noise components included The predicted noise components are used to perform a "denoising" operation on the noisy landing point sequence to calculate the next trajectory.

[0010] Furthermore, the kinematic reachability of candidate landing points is evaluated in real time through an inverse kinematics feasibility verification layer. Specifically, this is achieved using a feedforward neural network, whose input is the three-dimensional position of the candidate landing point relative to the robot's hip joint coordinate system. The robot's current leg configuration and the robot's body posture The output of the feedforward neural network is a scalar reachability score. The rating is in Within the specified range, determine if the landing point is feasible; otherwise, exclude the candidate landing point.

[0011] Furthermore, the construction of the training data for the feedforward neural network is specifically as follows:

[0012] In a simulation environment or geometric space, multiple combinations of robot states are randomly sampled. For each sampled data sample... The corresponding input vector Including three-dimensional position Leg structure and fuselage attitude ;

[0013] The Input into a physics simulation engine for solving and generate labels. ,Label This represents the feasibility score.

[0014] Furthermore, At that time, the label is a positive sample, which simultaneously satisfies joint angle constraints, singular configuration avoidance, and foot collision risk. When the condition is met, the label is negative, meaning that the condition does not meet any of the following conditions: joint angle constraint, singular configuration avoidance, and foot collision risk.

[0015] Furthermore, the composite reward function of the upper-layer landing point generation network. Progress Rewards Feasibility constraint rewards Sequence smoothness reward Terminal feasibility reward ,in, , , , Assigning weights to each reward Let the velocity vector be the center of mass of the robot. Let the target direction be the unit vector. This indicates the robot's maximum linear velocity. This indicates that the input value will be truncated within a specified range. To avoid decimals in infeasible values, Therefore, the landing point The total effective feasible trampling area or volume within a preset search radius, centered on the target area. This is a preset threshold.

[0016] Furthermore, the reward function of the lower-level motion control network is: Landing point tracking accuracy bonus Speed ​​tracking performance bonus Motion smoothness reward Energy efficiency reward Attitude stability reward ;in, Location of the landing point The location of the target landing point. For the desired linear velocity, For the desired steering speed, This represents the difference in action output between two moments. Indicates the output torque. This represents the difference between the target pose and the current pose.

[0017] The beneficial effects of this invention are as follows:

[0018] (1) "Perception-Planning-Control" Layered Fusion Architecture: A three-layer deep reinforcement learning architecture including a multi-scale perception encoder, an upper-layer look-ahead planning network and a lower-layer robust control network was designed to achieve effective decoupling and synergy between long-term landing point planning and high-frequency whole-body dynamics control.

[0019] (2) Parallel optimization across the entire sequence to ensure gait consistency and global reachability: This invention abandons the short-sighted decision-making of the traditional "one step at a time" approach and adopts a parallel denoising architecture. At the initial stage of iteration, it simultaneously observes and optimizes all landing points within the future time window, explicitly capturing gait dependencies in the long time domain. Combined with the terminal feasibility reward mechanism, the model can not only effectively prevent the generation of physically infeasible sequences such as crossed legs, but also predict and avoid "dead end" areas in sparse terrain in advance, significantly improving the success rate of traversing complex terrain.

[0020] (3) Built-in inverse kinematics feasibility verification layer: A differentiable lightweight inverse kinematics (IK) verification module is embedded inside the conditional denoising network to evaluate the kinematic feasibility and safety of candidate points in real time during the generation of landing point sequence, ensuring that the planning results meet the physical constraints from the source.

[0021] (4) Implicit dynamic constraint encoding, balancing long-term planning and real-time reasoning: Complex robot limb dynamic constraints and environmental geometric constraints are implicitly encoded in the generation distribution of the conditional denoising network; unlike traditional nonlinear model predictive control (MPC), which requires online iterative solving of complex optimization problems, this invention only needs to generate high-quality trajectories that conform to physical constraints through feedforward reasoning. This significantly reduces the computational burden while ensuring long-term look-ahead planning capabilities, meeting the real-time control requirements of legged robots under high dynamic motion. Attached Figure Description

[0022] Figure 1 This is a block diagram of the hierarchical reinforcement learning motion planning and control of a legged robot for sparse terrain as described in this invention;

[0023] Figure 2 A network block diagram is generated for the landing point described in this invention. Detailed Implementation

[0024] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0025] This invention employs a deeply integrated "perception-planning-control" hierarchical architecture: the upper-layer network extracts rich environmental semantic features based on a multi-scale terrain perception encoder, and a conditional denoising network models landing point planning as a parallel progressive denoising process, generating a robot landing point sequence. Simultaneously, a built-in inverse kinematics feasibility verification layer evaluates the physical feasibility of candidate landing points in real time. The lower-layer network receives the landing point sequence planned by the upper layer and the robot's high-frequency real-time state, and outputs robust joint motion commands based on reinforcement learning. By decoupling long-term planning and high-frequency control, this invention effectively solves the problems of short-sighted planning, low computational efficiency, and poor robustness in traditional methods in discontinuous, high-elevation, and sparse terrain, achieving highly dynamic and stable motion of legged robots in complex environments.

[0026] I. System Overall Framework and Core Innovations

[0027] The core of this invention lies in proposing a hierarchical reinforcement learning architecture that deeply integrates perception, planning, and control. This architecture effectively addresses the challenge of dynamic movement of humanoid robots on sparse terrain by introducing a spatiotemporal look-ahead planning mechanism. This framework uses an upper-layer network to perform joint reasoning based on terrain geometric features and the robot's dynamic state, generating a sequence of dynamically feasible landing points to achieve a feedforward planning closed loop. The overall system framework is as follows: Figure 1 As shown.

[0028] II. Upper-level landing point generation network with spatiotemporal foresight capabilities

[0029] The core innovation of this invention lies in the upper-layer network, which no longer makes single, instantaneous landing point decisions, but instead models landing point planning as a parallel, progressive denoising process. This enables the robot to perform multi-step, forward-looking planning, much like a human walking in complex terrain. The flowchart for this part is as follows... Figure 2 As shown.

[0030] 1. Multi-scale terrain sensing encoder

[0031] The input elevation map is first processed through a parallel convolutional path, using convolutional kernels of different sizes (3×3, 5×5, 7×7) to simultaneously extract local detail features and global structural features. The output multi-scale feature map is then adaptively fused using a gating mechanism to form terrain features rich in semantic information. .

[0032] Adaptive feature fusion includes:

[0033] (1) Calculation of gating weights:

[0034]

[0035] (2) Feature fusion:

[0036]

[0037] Among them, gating network The contribution of features at each scale is dynamically adjusted based on the robot's real-time motion state. , , Indicates the feature weights at each scale. Indicates the robot's linear velocity. Represents the robot's angular velocity. , These represent the robot's pitch angle and yaw angle, respectively. Indicates microscale characteristics, Indicates mesoscale characteristics, It indicates macroscopic scale characteristics.

[0038] 2. Landing Point Generation Task Modeling and Forward-Looking Planning

[0039] This invention views landing point planning as a parallel, progressive denoising process from a "random noise trajectory" to a "feasible target trajectory".

[0040] A conditional denoising network is constructed as the upper-level planner. This conditional denoising network adopts the Transformer architecture, treats the noisy landing point sequence as a time-series token, and optimizes all landing points in parallel within the entire future time window through an iterative refinement mechanism, thereby achieving globally optimal planning.

[0041] (1) Input and output of conditional denoising network

[0042] Planning Goal: Generating the Future The sequence of foot placement points of the robot in each step is denoted as the trajectory. .

[0043] Conditional input To ensure the generated trajectory is suitable for the current environment and task, the denoising process accepts the following input conditions:

[0044] Environmental perception information: Topographic features extracted by a multi-scale terrain perception encoder. ;

[0045] Robot's current state: including the robot's linear velocity angular velocity Attitude angle ( , The current position of the foot (i.e., the position of the robot's foot relative to the body, calculated from the joint angles);

[0046] Advanced target instructions: desired forward speed, target direction vector.

[0047] (2) Trajectory generation mechanism based on diffusion model

[0048] The generation process of this invention is a reverse diffusion process, comprising: This is the next iteration step. The process is as follows:

[0049] Step 1: Initialization (Noise Injection)

[0050] At the start of inference, random points are first initialized within the robot's planning space. Specifically, a sequence of landing points consisting entirely of standard Gaussian noise is sampled as the initial trajectory assumption.

[0051]

[0052] At this time It is a random collection of points that has no physical meaning.

[0053] Step 2: Parallel progressive iterative denoising

[0054] Conditional denoising network performs from arrive The iterative loop. In each iteration... Conditional denoising network Perform the following operations:

[0055] Parallel observation and evaluation: Conditional denoising networks simultaneously receive the current noisy trajectory. Current iteration step and conditional input The network's architecture enables it to process all points in the sequence in parallel. The network is evaluated with a global perspective. For all points, determine which points are physically infeasible (e.g., landing on an obstacle or beyond the range of leg movement) and which adjacent points are disjointed (e.g., taking too large a step or experiencing a sudden change in speed).

[0056] Predicting noise and fine-tuning: Conditional denoising networks do not directly output the final trajectory, but instead predict the current trajectory. Noise components included .

[0057] Based on the sampling formula of the diffusion model (such as the DDPM or DDIM sampler), the system uses the predicted noise to perform a "denoising" operation on the current trajectory and calculates the next trajectory with less noise. :

[0058]

[0059] Mathematically, this step is equivalent to the network calculating a gradient direction based on the terrain and dynamic constraints it has learned, and guiding all points in the sequence to make a small, coordinated adjustment towards a more reasonable and feasible region.

[0060] Step 3: Convergence and Output

[0061] go through After several iterations of fine-tuning, the randomness in the initial noise sequence is gradually eliminated, and effective structural information gradually emerges. The final trajectory is obtained. The sequence converges into a smooth, coherent sequence of landing points that conforms to both the robot's limb dynamics constraints and perfectly adapts to the terrain geometry. This sequence is then sent to the underlying controller for execution.

[0062] 3. Kinematic Feasibility Verification Module

[0063] A lightweight inverse kinematics feasibility verification layer is embedded inside the conditional denoising network. A feedforward neural network is used to handle any candidate landing point. Perform a rapid kinematic accessibility assessment.

[0064] The input to the feedforward neural network is the three-dimensional position of the candidate landing point relative to the robot's hip joint coordinate system. The robot's current leg configuration and the robot's overall posture The output of the feedforward neural network is a scalar reachability score. .

[0065] The lightweight inverse kinematics feasibility verification layer is specifically as follows:

[0066] (1) Training data construction

[0067] In supervised learning mode, the first step is to build a dataset containing a large number of "state-label" pairs, where the "label" is the true value. The input to the feedforward neural network is obtained using a high-precision physical simulation engine or an analytical inverse kinematics solver.

[0068] Data sampling: Randomly sample a large number of robot state combinations in the simulation environment or geometric space. For each data sample... The corresponding input vector Includes: candidate landing point locations This refers to random sampling points relative to the hip joint coordinate system; leg configuration This refers to the robot's joint angles and body posture. This refers to the robot's pitch, yaw, and other attitude angles.

[0069] Tag generation: (The above) Input into a rigorous physics simulation engine or analytical solver to generate labels. ;Label Represents the feasibility score, where:

[0070] Positive samples ( Simultaneously satisfying all of the following conditions:

[0071] Joint angle constraints: Whether the joint angles required to execute this gait are within the physical limits. The physical limits are determined according to the robot model.

[0072] Singularity avoidance: Will the landing point of the target cause the leg to fall into or approach a kinematic singularity? This is determined by the robot's joint position.

[0073] Foot collision risk: Assess the possibility of interference with one's own limbs during the swing leg's movement along the planned trajectory to the target landing point; specifically, by discretely sampling the planned trajectory, calculate the minimum distance between the swing leg model and other stationary limb models. If this distance is lower than a preset safety threshold, a collision risk is determined to exist.

[0074] negative samples ( (Violation of any of the above conditions)

[0075] (2) Network model construction

[0076] This lightweight inverse motion testing layer is designed as a feedforward neural network, serving as a mapping function from geometric states to feasibility scores. The feedforward neural network includes:

[0077] Input layer, receiving vector It includes the landing point coordinates, joint angles, and fuselage attitude.

[0078] Hidden layers are designed as lightweight fully connected layers (MLPs) using ReLU or Tanh activation functions to ensure real-time inference and differentiability.

[0079] Output layer, outputs a scalar The Sigmoid activation function is typically used to restrict the output to between 0 and 1, representing the probability of feasibility.

[0080] (3) Loss function design

[0081] The constructed feedforward neural network is trained using training data, with the goal of minimizing the difference between the predicted score and the true label. The differences between them.

[0082] The loss function used in the training process is the binary cross-entropy loss function, which is common in probabilistic prediction tasks:

[0083]

[0084] in, This represents the number of samples.

[0085] Candidate landing point Predicted ratings Within the specified range, determine if the landing point is feasible; otherwise, exclude the candidate landing point.

[0086] 4. Implementation method

[0087] The specific implementation steps are as follows: In order to enable the upper network to have the spatiotemporal look-ahead capability in sparse terrain, that is, to generate a sequence of landing points that both meet the current dynamic constraints and create favorable conditions for subsequent motion, this invention adopts a reinforcement learning-based training method.

[0088] The core of this training method lies in designing a reward mechanism that evaluates not only the quality of individual landing points, but also the contribution of the entire sequence of landing points to the success of the long-term task.

[0089] (1) Training environment construction

[0090] Simulation platform: Use a high-fidelity physics simulation platform (such as NVIDIA Isaac Gym or MuJoCo) that supports massively parallel sampling to accelerate training.

[0091] Procedural terrain generation: Construct a procedural terrain generator capable of dynamically generating sparse terrain of varying difficulty. Terrain parameters include: average spacing between stepping stones. Surface area of ​​stepping stones and the height difference between adjacent stepping stones By adjusting these parameters, various scenarios can be simulated, ranging from flat ground to extremely fragmented terrain.

[0092] (2) Training paradigm and network architecture

[0093] Algorithm selection: The Proximal Policy Optimization (PPO) algorithm was used for training. PPO is an online policy reinforcement learning algorithm suitable for decision-making problems with continuous action spaces (landing point coordinates).

[0094] Autoregressive generative training: During training, the network at each time step... A length of [length] needs to be generated. sequence The key to training is to enhance the network's understanding. right The effect of conditional constraints.

[0095] (3) Design of composite reward function

[0096] To guide the network in learning forward-looking strategies, this invention designs a composite reward function. It evaluates the entire generated sequence at each decision step:

[0097]

[0098] in, The weighting of each reward.

[0099] Progress Rewards This encourages the robot to move in the direction and at the target speed.

[0100]

[0101] in, It is the velocity vector of the robot's center of mass. It is the unit vector of the target direction. This indicates the robot's maximum linear velocity. This means that the input values ​​will be truncated within a specified range. This reward ensures that the planned sequence of landing points will result in actual displacement.

[0102] Feasibility constraint rewards This is crucial for ensuring physical feasibility. By utilizing the network's built-in inverse kinematics feasibility verification layer, the feasibility of each landing point in the sequence is checked. Conduct an assessment:

[0103]

[0104] in, For the feasibility score of inverse kinematics, To avoid the occurrence of infeasible values ​​in small quantities (10 in this embodiment), -5 If any point in the sequence causes the IK solution to fail (such as a singularity or self-collision), this term will generate a huge negative reward (penalty). This forces the network to learn to reject points that, although they may seem to allow for rapid progress, will lead to subsequent gait stagnation.

[0105] Sequence smoothness reward To ensure gait consistency, abrupt changes between adjacent planned footholds are penalized, guiding the network to generate uniform and natural gait sequences.

[0106]

[0107] Terminal feasibility reward :

[0108]

[0109] in, As a foothold Centered on, preset search radius The total effective feasible trampling area (or volume) within. This is a preset threshold determined based on the robot's specific configuration. This reward specifically evaluates the last planning point in the sequence. Scalability. This reward item will detect... Are there enough other feasible stepping stones within a certain radius: the endpoint Located in an isolated area with no way forward, the system will impose penalties to deter such short-sighted decisions; conversely, if the endpoint... Additional rewards are given to networks located in highly accessible areas with abundant follow-up options. This mechanism explicitly guides networks to learn planning strategies with a long-term perspective.

[0110] III. Lower-level motion control network

[0111] The lower-level motion control network, serving as the system's execution layer, is responsible for translating the abstract execution points of the upper-level planning into concrete joint movement commands. This design employs a reinforcement learning-based whole-body motion controller to ensure accurate and stable execution of the upper-level planning results.

[0112] 1. Network architecture and interfaces

[0113] (1) Observation space design:

[0114] Planning instruction: Current target landing point location The location of the next target landing point ;

[0115] Robot state: Robot linear velocity angular velocity ,attitude ( , ), joint position Joint speed ;

[0116] User command: Desired base speed Expected turning speed ;

[0117] Historical information: Action sequence of the past two time steps .

[0118] (2) Motion space design:

[0119] Output: Target position offsets for 12 joints .

[0120] (3) Network structure:

[0121] Basic architecture: 4-layer fully connected neural network;

[0122] Input layer: Observation vectors (total dimension 58);

[0123] Hidden layer: 512→256→128 neurons, ReLU activation function;

[0124] Output layer: 12 neurons, tanh activation function.

[0125] 2. Reward Function Design

[0126] The reward function balances multiple competing objectives, guiding the network to learn high-quality movement strategies.

[0127] Landing point tracking accuracy bonus: ,in, Location of the landing point The location of the target landing point;

[0128] Speed ​​tracking performance bonus: ;

[0129] Motion smoothness reward: ;

[0130] Energy efficiency bonus: ;

[0131] Attitude stability reward: ;

[0132] Total reward function: .

[0133] in, This represents the difference in action output between two moments. Indicates the output torque. This represents the difference between the target pose and the current pose.

[0134] Through the above design, the lower-level motion control network can effectively execute the upper-level planning instructions, maintaining system stability while ensuring tracking accuracy, thus forming a complete control closed loop. The network's simple architecture ensures real-time performance and is suitable for practical robot deployment.

[0135] The embodiments described above are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A hierarchical reinforcement learning motion planning and control method for legged robots in sparse terrain, characterized in that: Upper-level landing point generation network: Based on a multi-scale perceptual encoder, terrain features rich in semantic information are extracted from the elevation map. The terrain features, robot real-time status and target commands are used as inputs to a conditional denoising network. The conditional denoising network models the landing point planning as a parallel progressive denoising process to generate a robot landing point sequence. During the process of determining the landing point sequence, the kinematic accessibility of candidate landing points is evaluated in real time through the inverse kinematics feasibility verification layer, thereby determining the planned landing point sequence. The lower-level motion control network receives the planned landing point sequence and the robot's real-time status, and outputs robust joint motion commands based on reinforcement learning to control the legged robot's movements.

2. The hierarchical reinforcement learning motion planning and control method for legged robots according to claim 1, characterized in that, The conditional denoising network models landing point planning as a parallel, progressive denoising process to generate a robot landing point sequence. Specifically, this includes sampling a landing point sequence composed of standard Gaussian noise as the initial trajectory, and the conditional denoising network executing... arrive The iterative loop.

3. The hierarchical reinforcement learning motion planning and control method for legged robots according to claim 2, characterized in that, In each iteration In the middle, the conditional denoising network receives the current noisy landing point sequence. Current iteration step and conditional input ,predict Noise components included The predicted noise components are used to perform a "denoising" operation on the noisy landing point sequence to calculate the next trajectory.

4. The hierarchical reinforcement learning motion planning and control method for legged robots according to claim 3, characterized in that, The kinematic reachability of candidate landing points is evaluated in real time through an inverse kinematics feasibility verification layer. Specifically, this is achieved using a feedforward neural network, whose input is the three-dimensional position of the candidate landing point relative to the robot's hip joint coordinate system. The robot's current leg configuration and the robot's body posture The output of the feedforward neural network is a scalar reachability score. The rating is in Within the specified range, determine if the landing point is feasible; otherwise, exclude the candidate landing point.

5. The hierarchical reinforcement learning motion planning and control method for legged robots according to claim 4, characterized in that, The construction of the training data for the feedforward neural network is specifically as follows: In a simulation environment or geometric space, multiple combinations of robot states are randomly sampled. For each sampled data sample... The corresponding input vector Including three-dimensional position Leg structure and fuselage attitude ; The Input into a physics simulation engine for solving and generate labels. ,Label This represents the feasibility score.

6. The hierarchical reinforcement learning motion planning and control method for legged robots according to claim 5, characterized in that, At that time, the label is a positive sample, which simultaneously satisfies joint angle constraints, singular configuration avoidance, and foot collision risk. When the condition is met, the label is negative, meaning that the condition does not meet any of the following conditions: joint angle constraint, singular configuration avoidance, and foot collision risk.

7. The hierarchical reinforcement learning motion planning and control method for legged robots according to claim 1, characterized in that, The composite reward function of the upper-layer landing point generation network Progress Rewards Feasibility constraint rewards Sequence smoothness reward Terminal feasibility reward ,in, , , , Assigning weights to each reward Let the velocity vector be the center of mass of the robot. Let the target direction be the unit vector. This indicates the robot's maximum linear velocity. This indicates that the input value will be truncated within a specified range. To avoid decimals in infeasible values, Therefore, the landing point The total effective feasible trampling area or volume within a preset search radius, centered on the target area. This is a preset threshold.

8. The hierarchical reinforcement learning motion planning and control method for legged robots according to claim 1, characterized in that, The reward function of the lower-level motion control network is Landing point tracking accuracy bonus Speed ​​tracking performance bonus Motion smoothness reward Energy efficiency reward Attitude stability reward ;in, Location of the landing point The location of the target landing point. For the desired linear velocity, For the desired steering speed, This represents the difference in action output between two moments. Indicates the output torque. This represents the difference between the target pose and the current pose.

Images