Four-legged robot parkour navigation method and system based on multi-modal feature fusion

Abstract

The application relates to a kind of four-legged robot parkour navigation method and system based on multimodal feature fusion, method includes: constructing multimodal parkour terrain dataset, including depth image, proprioceptive information and control instruction;The depth image sequence and proprioceptive sequence of historical time step are encoded, to obtain spatiotemporal fusion feature;Control instruction and corresponding target position vector are encoded, to obtain historical decision feature;The depth image collected to current time is encoded, to obtain fine spatial feature;Spatiotemporal fusion feature, historical decision feature and fine spatial feature are fused, to form conditional context;Based on diffusion strategy model, with conditional context as condition, denoising is carried out, and navigation control instruction is generated, to complete parkour navigation task.Compared with prior art, the application has the advantages of high navigation success rate, path planning diversity, effective processing of complex 3D terrain, overcoming the problem of disconnection between high-level planning and low-level motion control, etc.

Description

Technical Field

[0001] This invention relates to the field of robot navigation technology, and in particular to a method and system for parkour navigation of quadruped robots based on multimodal feature fusion. Background Technology

[0002] In recent years, the locomotion capabilities of quadruped robots have made significant progress through learning-based methods, demonstrating remarkable agility and robustness across various terrains. Existing research has enabled quadruped robots to perform parkour skills, including running, jumping, climbing, and vaulting, to traverse crowded streets, challenging terrains, or cluttered warehouses.

[0003] However, existing technologies have significant shortcomings when implementing complex parkour navigation tasks. First, traditional SLAM-based navigation methods rely on map building and predefined rules, leading to a mismatch between environmental perception and control commands, and the planned path is prone to getting trapped in local optima, limiting the robot's ability to navigate flexibly in complex environments. Second, while map-independent visual navigation models (such as diffusion-based navigation models) can effectively solve local optima problems, they suffer from a severe disconnect between high-level planning and low-level motion control, often resulting in the robot's camera avoiding obstacles while its body collides. Most importantly, both SLAM-based methods and visual navigation models rely on tracking local target coordinates for navigation, and cannot provide the fine-grained speed adjustment commands required for parkour navigation tasks like human-controlled intelligent agents.

[0004] Although end-to-end motion navigation frameworks based on vision-language-action (VLA) and reinforcement learning (RL) have been explored, they still face key limitations. VLA methods are limited to two-dimensional planes with pre-defined navigable paths, failing to meet the high dynamic response required for parkour navigation. RL-based methods suffer from poor convergence, high computational cost, and unstable training, weakening their navigation performance in complex environments.

[0005] Therefore, there is an urgent need in this field for an autonomous parkour navigation solution that can output precise speed commands, achieve close coupling between high-level planning and low-level motion, and perform diverse path planning in complex three-dimensional terrain. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a quadruped robot parkour navigation method and system based on multimodal feature fusion that can output fine speed commands, achieve tight coupling between high-level planning and low-level motion, and perform diversified path planning in complex three-dimensional terrain.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] A method for parkour navigation of a quadruped robot based on multimodal feature fusion, comprising:

[0009] Data is collected in advance of the parkour process of the quadruped robot to construct a multimodal parkour terrain dataset, which includes depth images, proprioceptive information and control commands of the quadruped robot collected under various terrains.

[0010] Encode the depth image sequence and proprioceptive sequence of historical time steps formed in the multimodal parkour terrain dataset to obtain spatiotemporal fusion features;

[0011] The control commands and corresponding target position vectors in the multimodal parkour terrain dataset are encoded to obtain historical decision features;

[0012] Encode the depth image acquired by the quadruped robot at the current moment to obtain fine spatial features;

[0013] The spatiotemporal fusion features, historical decision features, and refined spatial features are fused together to form a conditional context;

[0014] Based on the diffusion strategy model, and taking the conditional context as a condition, denoising processing is performed to generate navigation control commands for the current moment, so as to drive the quadruped robot to complete the parkour navigation task.

[0015] Furthermore, the diffusion strategy model is a denoising diffusion probability model. This denoising diffusion probability model uses a noise prediction network to gradually sample and denoise the noisy action vectors obtained from the standard Gaussian distribution in K-step denoising iterations, guided by the conditional context, to generate the final noise-free navigation command.

[0016] Furthermore, in each denoising iteration step, the noisy action vector is... With conditional context Input noise prediction network In the process of sampling and denoising, the corresponding denoising equation is expressed as follows:

[0017]

[0018] In the formula, For the denoising iteration step k Updated noisy action vectors and All are scalar coefficients. Additional Gaussian random noise is added for each step. For variance, It is a unit vector.

[0019] Furthermore, the fusion process of the spatiotemporal fusion features, historical decision features, and refined spatial features includes:

[0020] A multi-layer Transformer network based on a multi-head self-attention mechanism is adopted. First, the spatiotemporal fusion features, historical decision features, and fine spatial features are mapped and normalized respectively. Then, dynamic dependencies are established between different modal features through cross-attention layers, thereby realizing the unified modeling of spatiotemporal fusion features, historical decision features, and fine spatial features.

[0021] Furthermore, the proprioceptive information includes base angular velocity, joint position, joint velocity, information on the most recently executed action, and gravity vector;

[0022] The various terrain features include gaps, hurdles, stairs, gravel, boxes, and flat ground;

[0023] The data acquisition methods include script-controlled acquisition and / or expert remote control acquisition.

[0024] Furthermore, the method employs a multimodal temporal encoder to encode the depth image sequence and the proprioceptive sequence to obtain spatiotemporal fusion features;

[0025] The multimodal temporal encoder adopts a Transformer architecture, which segments the depth image sequence into an image patch sequence and inputs it into the encoder along with the proprioceptive sequence. Cross-modal feature fusion is performed through a cross-attention mechanism to obtain the spatiotemporal fusion feature.

[0026] Furthermore, the method uses the current spatial encoder to encode the depth image acquired at the current moment to obtain fine spatial features;

[0027] The current spatial encoder uses a convolutional neural network architecture to extract detailed spatial features from depth images.

[0028] Furthermore, the method employs a decision memory encoder to encode the control commands and the corresponding target position vectors to obtain historical decision features;

[0029] The decision memory encoder employs a multilayer perceptron architecture to model the geometric relationship between historical actions and the target.

[0030] Furthermore, the method also includes sending the generated navigation control command at the current moment to the underlying motion controller of the quadruped robot to drive the quadruped robot to complete the parkour navigation task.

[0031] The navigation control commands include angle and speed information.

[0032] The present invention also provides a quadruped robot parkour navigation system that implements the quadruped robot parkour navigation method based on multimodal feature fusion as described above, comprising:

[0033] The data acquisition module is used to collect data on the parkour process of the quadruped robot in advance and build a multimodal parkour terrain dataset. The multimodal parkour terrain dataset includes depth images, proprioceptive information and control commands of the quadruped robot collected under various terrains.

[0034] The multimodal feature encoding module encodes the depth image sequence and proprioceptive sequence of historical time steps formed in the multimodal parkour terrain dataset to obtain spatiotemporal fusion features; it encodes the control commands and corresponding target position vectors in the multimodal parkour terrain dataset to obtain historical decision features; and it encodes the depth image acquired by the quadruped robot at the current moment to obtain fine spatial features.

[0035] The diffusion strategy decision module is used to fuse the spatiotemporal fusion features, historical decision features, and fine spatial features to form a conditional context; based on the diffusion strategy model, and using the conditional context as a condition, it performs denoising processing to generate navigation control commands for the current moment.

[0036] The communication interface module is used to send navigation control commands to the underlying motion controller of the quadruped robot, driving the quadruped robot to complete parkour navigation tasks.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] (1) This invention innovatively integrates multimodal perception information (historical visual temporal information, current spatial detail information, proprioceptive information) and decision memory information (historical actions, target information) to obtain conditional context, providing comprehensive information support for the diffusion model including environmental state, motion trend and target orientation, and realizing precise conditional guidance for the speed navigation control command generation process; it can ensure that the intermediate actions generated by the diffusion model in each time step conform to environmental constraints and task intent, and the final output navigation command has continuity, stability and task adaptability.

[0039] The denoising guidance process of this invention enables the diffusion model to have adaptive generation capabilities based on the perception context, allowing the robot to understand complex 3D environments more comprehensively and robustly, and make decisions with long-term continuity and intent. It can generate speed and angular velocity commands that conform to motion planning laws in real time in complex 3D terrain.

[0040] (2) Improved navigation performance: The generative diffusion model is adopted as the core of the strategy, which can effectively model the multimodal distribution of navigation instructions (i.e., there are multiple feasible ways to pass through in the same scene), thereby generating diversified and high-performance navigation paths with a success rate and path efficiency significantly higher than traditional methods.

[0041] (3) Laying the foundation for simulation transfer: The dataset constructed and the proposed method are based on non-privileged sensor information (forward depth camera, proprioception), which shields the global information that can only be obtained in the simulation environment (such as the absolute coordinates of the agent and the global map), making the method have great potential for transfer to real robots.

[0042] (4) Strong generalization ability: After sufficient training, the method of the present invention can achieve accurate, efficient and diverse parkour navigation in long-term complex environments that were not seen during training, combined with a powerful underlying motion controller.

[0043] (5) Solved the disconnect problem: This invention redefines the navigation task as directly outputting control commands with speed information, rather than abstract path points (x, y), so that high-level navigation planning and low-level motion control are naturally coupled, effectively avoiding the collision problem caused by the disconnect between perception and execution. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating a quadruped robot parkour navigation method based on multimodal feature fusion provided in an embodiment of the present invention.

[0045] Figure 2 This is a schematic diagram illustrating the process of constructing a multimodal parkour terrain dataset according to an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of the training process for a quadruped robot parkour navigation method based on multimodal feature fusion provided in an embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0048] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0049] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0050] Example 1

[0051] like Figure 1 and Figure 3 As shown, this embodiment provides a quadruped robot parkour navigation method based on multimodal feature fusion, including:

[0052] S1: Data is collected in advance of the quadruped robot's parkour process to construct a multimodal parkour terrain dataset, which includes depth images, proprioceptive information and control commands of the quadruped robot collected under various terrains.

[0053] Proprioceptive information includes base angular velocity, joint position, joint velocity, information on the most recently executed action, and gravity vector;

[0054] The terrain is preferably complex, including gaps with a width of 0.4m to 0.7m, hurdles with a height of 0.4m to 0.5m, stairs with a step height of 0.15m to 0.2m, as well as gravel, boxes and flat ground.

[0055] The multimodal parkour terrain dataset is preferably constructed by combining automated script-based data collection with expert remote-controlled data collection to ensure data diversity and coverage of feasible solutions.

[0056] like Figure 2 As shown, the process of constructing the multimodal parkour terrain dataset in this embodiment includes:

[0057] S101: Build and select terrain environment;

[0058] S102: Determine whether the current terrain environment has a feasible solution for traversing the terrain. If it does, collect data automatically through a script; otherwise, collect data through remote operation by an expert.

[0059] S103: Collect a multimodal parkour terrain dataset;

[0060] S104: Augmenting the multimodal parkour terrain dataset;

[0061] S105: Complete the construction of a multimodal parkour terrain dataset.

[0062] S2: Encode the depth image sequence and proprioceptive sequence of historical time steps formed in the multimodal parkour terrain dataset to obtain spatiotemporal fusion features;

[0063] Preferably, a multimodal temporal encoder is used to encode the depth image sequence and the proprioceptive sequence to obtain spatiotemporal fusion features;

[0064] The multimodal temporal encoder adopts the Transformer architecture, which segments the depth image sequence into an image patch sequence and inputs it into the encoder along with the proprioceptive sequence. Cross-modal feature fusion is performed through a cross-attention mechanism to obtain spatiotemporal fusion features.

[0065] Specifically, the multimodal temporal encoder segments the depth image sequence of historical K frames into image blocks and embeds them together with the proprioceptive sequence of K frames from the same period (such as joint angles, body orientation, velocity, etc.) into a unified feature space. It distinguishes them through learnable modal embedding and fuses them through a cross-attention mechanism, finally outputting the encoded environmental features.

[0066] S3: Encode the control commands and corresponding target position vectors in the multimodal parkour terrain dataset to obtain historical decision features;

[0067] Preferably, the depth image acquired at the current moment is encoded using the current spatial encoder to obtain fine spatial features;

[0068] Current spatial encoders employ a convolutional neural network architecture to extract detailed spatial features from depth images.

[0069] Specifically, the current spatial encoder uses a shallow convolutional neural network (CNN) to process the depth image of the current frame and extract fine, high-resolution spatial terrain features to deal with scenarios that require immediate response, such as sudden obstacles.

[0070] S4: Encode the depth image acquired by the quadruped robot at the current moment to obtain fine spatial features;

[0071] Preferably, a decision memory encoder is used to encode the control commands and the corresponding target position vectors to obtain historical decision features;

[0072] The decision memory encoder employs a multilayer perceptron architecture to model the geometric relationship between historical actions and the target.

[0073] Specifically, the decision memory encoder uses a multilayer perceptron (MLP) to receive the most recent T-step historical action sequence (i.e., navigation instructions) and the current target's position vector relative to the robot, and outputs the encoded decision and target guidance features.

[0074] S5: Integrate spatiotemporal fusion features, historical decision-making features, and refined spatial features to form a conditional context;

[0075] Preferably, the fusion process includes: employing a multi-layer Transformer network based on a multi-head self-attention mechanism; firstly, mapping and normalizing the spatiotemporal fusion features, historical decision features, and fine spatial features respectively; then, establishing dynamic dependencies between different modal features through cross-attention layers, thereby achieving unified modeling of temporal, spatial, and decision information. This fusion mechanism maintains temporal consistency and modal alignment characteristics while generating a unified conditional context. After being constructed by this network, the conditional context can provide the diffusion model with comprehensive information support including environmental state, motion trend, and target orientation, enabling precise conditional guidance for the subsequent speed navigation control command generation process.

[0076] S6: Based on the diffusion strategy model, and with the conditional context as the condition, perform noise reduction processing to generate the navigation control command for the current moment;

[0077] Preferably, the diffusion strategy model is the Denoising Diffusion Probability Model (DDPM). This model uses a noise prediction network to gradually sample and denoise the noisy action vectors obtained from the standard Gaussian distribution in K-step denoising iterations, guided by the conditional context, to generate the final noise-free navigation command.

[0078] In this embodiment, the denoising diffusion probability model uses a noise prediction network to progressively denoise the initial Gaussian noise samples in K-step (e.g., K=10) denoising iterations to generate the final noise-free navigation command. The denoising process is guided by a conditional context.

[0079] The denoising process of the diffusion strategy model adopts a denoised diffusion probability model structure, the core of which is a conditional noise prediction network. The model first samples noisy action vectors from a standard Gaussian distribution. Then, noise is gradually reduced in K iterations to generate the final noise-free navigation instructions. In each denoising step, the model removes noise from the current action state. With conditional context Input noise prediction network The network predicts noise based on noise scheduling parameters and performs updates according to the following denoising equation:

[0080]

[0081] In the formula, For the denoising iteration step k Updated noisy action vectors and All are scalar coefficients. Additional Gaussian random noise is added for each step. For variance, It is a unit vector.

[0082] conditional context Composed of spatiotemporal features, spatial features, and decision features, it provides semantic and geometric guidance throughout the denoising process, ensuring that the denoising direction remains consistent with the target task. This guidance mechanism ensures that the intermediate actions generated by the diffusion model at each time step conform to environmental constraints and task intent, resulting in navigation commands that are continuous, stable, and task-adaptive.

[0083] This denoising guidance process enables the diffusion model to have adaptive generation capabilities based on the perception context, and can generate velocity and angular velocity commands that conform to motion planning laws in real time in complex 3D terrain.

[0084] S7: Send the generated navigation control command for the current moment to the quadruped robot's underlying motion controller to drive the quadruped robot to complete the parkour navigation task.

[0085] The following describes an implementation process of the above solution:

[0086] Implementation Environment: This embodiment was conducted in the NVIDIA Isaac Sim simulation environment, using the Unitree Go1 quadruped robot model. The robot is equipped with a forward-facing depth camera (simulating an Intel RealSense D435i).

[0087] Dataset construction (S1):

[0088] In Isaac Sim, various terrains were constructed, including gaps, hurdles, staircases, gravels, boxes, and flat surfaces. Data was collected in two ways: A. Scripted Control: For each basic terrain, 50 pairs of start and end points were randomly generated. An automated script controlled the robot's movement, recording depth maps, proprioception (including linear / angular velocities, joint positions / velocities, gravity vectors, etc.), and control commands. B. Expert Remote Control: An expert operator controlled the robot to traverse the terrain via a remote control device, particularly for collecting multiple feasible solutions for the same terrain (e.g., crossing the same obstacle in different ways) and complex obstacle avoidance trajectories. The final dataset (QPTD) contained a total navigation distance of 31.0 km, 6400 trajectories, and approximately 250,000 depth images.

[0089] Model Training: The MulDP model was trained using the AdamW optimizer with a learning rate of 10. -4 The batch size is 512, and the training duration is 1000 epochs. Cosine learning rate decay and warm-up strategies are used to stabilize the training process. The loss function is the mean squared error of noise prediction (MSE Loss).

[0090] Navigation execution (S7):

[0091] 1. Deploy the trained MulDP model in the simulation environment. During each control cycle (10Hz):

[0092] 2. Obtain current and historical depth image sequences, proprioceptive sequences, historical action sequences, and target vectors.

[0093] 3. Input the MulDP model and generate navigation commands. .

[0094] 4. Send the command to the underlying motion controller (RENet motion strategy is used in this embodiment).

[0095] 5. The underlying motion controller translates navigation commands into specific joint torques to drive the robot's movement.

[0096] Example 2

[0097] This embodiment provides a quadruped robot parkour navigation system that implements the quadruped robot parkour navigation method based on multimodal feature fusion as described in Embodiment 1, including:

[0098] The data acquisition module is used to collect data on the parkour process of the quadruped robot in advance and build a multimodal parkour terrain dataset. The multimodal parkour terrain dataset includes depth images, proprioceptive information and control commands of the quadruped robot collected under various terrains.

[0099] The multimodal feature encoding module encodes the depth image sequence and proprioceptive sequence of historical time steps formed in the multimodal parkour terrain dataset to obtain spatiotemporal fusion features; it encodes the control commands and corresponding target position vectors in the multimodal parkour terrain dataset to obtain historical decision features; and it encodes the depth image acquired by the quadruped robot at the current moment to obtain fine spatial features.

[0100] The diffusion strategy decision module is used to fuse spatiotemporal fusion features, historical decision features, and fine spatial features to form a conditional context; based on the diffusion strategy model, and using the conditional context as a condition, it performs denoising processing to generate navigation control commands for the current moment.

[0101] The communication interface module is used to send navigation control commands to the underlying motion controller of the quadruped robot, driving the quadruped robot to complete parkour navigation tasks.

[0102] It should be noted that the specific content and beneficial effects of the system in this application can be found in the above method embodiments, and will not be repeated here.

[0103] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A four-legged robot parkour navigation method based on multi-modal feature fusion, characterized in that, The method comprises the following steps: Data collection is performed in advance on the parkour process of the quadruped robot, and a multi-modal parkour terrain dataset is constructed, which includes depth images, proprioceptive information and control instructions of the quadruped robot collected under various terrains; The depth image sequence and the proprioceptive sequence of the historical time step formed in the multi-modal parkour terrain dataset are encoded to obtain spatio-temporal fusion features; The control instructions and the corresponding target position vectors in the multi-modal parkour terrain dataset are encoded to obtain historical decision features; The depth image collected at the current time of the quadruped robot is encoded to obtain fine spatial features; The spatio-temporal fusion features, the historical decision features and the fine spatial features are fused to form a conditional context; Based on a diffusion strategy model, the conditional context is taken as a condition to perform denoising processing, and a navigation control instruction at the current time is generated to drive the quadruped robot to complete the parkour navigation task.

2. The four-legged robot parkour navigation method based on multi-modal feature fusion according to claim 1, characterized in that, The diffusion strategy model is a denoising diffusion probability model, which is guided by the conditional context in K-step denoising iteration to gradually sample and denoise the noisy action vector sampled from the standard Gaussian distribution to generate the final noise-free navigation instruction through a noise prediction network.

3. The four-legged robot parkour navigation method based on multi-modal feature fusion according to claim 2, characterized in that, In each denoising iteration step, the noisy action vector with conditional context together with the input noise prediction network In the sampling denoising, the expression of the corresponding denoising equation is: wherein is the denoising iteration step k is the updated noisy action vector, and are scalar coefficients, is the additional Gaussian random noise added at each step, is the variance, is the unit vector.

4. The four-legged robot parkour navigation method based on multi-modal feature fusion according to claim 1, characterized in that, The fusion process of the spatio-temporal fusion features, the historical decision features and the fine spatial features comprises: A multi-layer Transformer network based on a multi-head self-attention mechanism is adopted to firstly map and normalize the spatio-temporal fusion features, the historical decision features and the fine spatial features, and then establish a dynamic dependency relationship between different modal features through a cross-attention layer, so as to realize unified modeling of the spatio-temporal fusion features, the historical decision features and the fine spatial features.

5. The four-legged robot parkour navigation method based on multi-modal feature fusion according to claim 1, characterized in that, The proprioceptive information includes base angular velocity, joint position, joint velocity, recently executed action information and gravity vector; The various terrains include gaps, hurdles, stairs, gravel, boxes and flat ground; The data collection mode includes script control collection and expert remote control operation collection.

6. The four-legged robot parkour navigation method based on multi-modal feature fusion according to claim 1, characterized in that, The method adopts a multi-modal time sequence encoder to encode the depth image sequence and the proprioceptive sequence to obtain spatio-temporal fusion features; The multi-modal time sequence encoder adopts a Transformer architecture, divides the depth image sequence into an image block sequence, inputs the image block sequence and the proprioceptive sequence into the encoder, and performs cross-modal feature fusion through a cross-attention mechanism to obtain the spatio-temporal fusion features.

7. The four-legged robot parkour navigation method based on multi-modal feature fusion according to claim 1, characterized in that, The method adopts a current space encoder to encode the depth image collected at the current time to obtain fine spatial features; The current space encoder adopts a convolutional neural network architecture to extract detailed spatial features in the depth image.

8. The four-legged robot parkour navigation method based on multi-modal feature fusion according to claim 1, characterized in that, The method adopts a decision memory encoder to encode the control instructions and the corresponding target position vectors to obtain historical decision features; The decision memory encoder adopts a multi-layer perceptron architecture to model the geometric relationship between the historical actions and the target.

9. The four-legged robot parkour navigation method based on multi-modal feature fusion according to claim 1, characterized in that, The method further comprises sending the generated navigation control instruction at the current time to a lower motion controller of the quadruped robot to drive the quadruped robot to complete the parkour navigation task; The navigation control instruction includes angle and speed information.

10. A quadruped robot parkour navigation system for implementing the quadruped robot parkour navigation method based on multi-modal feature fusion according to any one of claims 1-9, characterized in that, The application relates to a navigation control method and device for a quadruped robot. The method comprises the following steps: a data acquisition module is used for pre-acquiring data of a parkour process of the quadruped robot, and a multi-modal parkour terrain data set is constructed, wherein the multi-modal parkour terrain data set comprises depth images, body sensation information and control instructions of the quadruped robot acquired under multiple terrains; a multi-modal feature coding module is used for coding a depth image sequence and a body sensation sequence of a historical time step formed in the multi-modal parkour terrain data set, so as to obtain spatio-temporal fusion features; control instructions and corresponding target position vectors in the multi-modal parkour terrain data set are coded, so as to obtain historical decision features; and a depth image acquired by the quadruped robot at a current time is coded, so as to obtain fine spatial features; a diffusion strategy decision module is used for fusing the spatio-temporal fusion features, the historical decision features and the fine spatial features, so as to form a conditional context; a diffusion strategy model is used for carrying out denoising processing based on the conditional context, so as to generate a navigation control instruction at the current time; and a communication interface module is used for sending the navigation control instruction to a bottom motion controller of the quadruped robot, so as to drive the quadruped robot to complete a parkour navigation task.

Images