Autonomous navigation method for quadruped robot based on big-small brain collaborative architecture

Abstract

The application provides a kind of four-legged robot autonomous navigation method based on big brain coordination architecture, it is related to four-legged robot autonomous navigation technical field, and the hierarchical coordination is realized by decoupling of large language model Agent and local navigation stack: brain is responsible for semantic understanding and long-range planning, cerebellum independently undertakes path tracking, safety monitoring and emergency obstacle avoidance braking, reduces the risk of decision and control coupling, improves robustness under weak network. The system adopts on-demand image capture and multi-source perception fusion strategy, optimizes communication bandwidth and environment modeling accuracy. At the same time, the decision context diagnosis and TF supplementary chain mechanism are constructed, the navigation fault is reproducible and locatable, the traditional end-to-end system is difficult to debug and difficult to attribute problem is solved, and the safety and maintainability of the system are enhanced.

Description

Technical Field

[0001] This invention relates to the field of autonomous navigation for quadruped robots, and in particular to an autonomous navigation method for quadruped robots based on a cerebellum-cerebrum collaborative architecture. Background Technology

[0002] With the development of artificial intelligence technology, quadruped robots are increasingly being used in complex, unstructured environments such as homes, office buildings, or the wild. To achieve true autonomy, robots not only need basic obstacle avoidance and mobility capabilities, but also advanced abilities to understand human natural language commands and perform logical reasoning. Current mobile robot navigation technologies are mainly divided into two categories: LiDAR-based SLAM global mapping navigation and vision-based navigation.

[0003] Global mapping models based on LiDAR SLAM are currently a mature solution used in industry. This approach typically relies on high-precision LiDAR sensors to acquire point cloud data of the environment, constructs a high-precision global grid map or point cloud map using SLAM algorithms, and then utilizes AI on top of this. Alternatively, algorithms such as Dijkstra's algorithm can be used for path planning. However, this approach has significant limitations. Multi-beam LiDAR is expensive, significantly increasing the manufacturing cost of robots and limiting its widespread adoption in mass-market consumer products. Furthermore, the entire work area must be pre-scanned and mapped before use, and any changes in the environmental layout, such as furniture movement or item stacking, require remapping or complex map updates, making it unsuitable for rapidly changing scenarios. Additionally, LiDAR can only perceive geometric obstacles and cannot understand semantic information such as "chair" or "kitchen," making it difficult to execute commands that conform to human natural language habits, such as "go to the red sofa."

[0004] Pure vision-based navigation models utilize low-cost RGB cameras and depth cameras as primary sensors, significantly reducing hardware costs. Furthermore, the image data contains rich texture and semantic information, enabling robots to understand their environment and navigate like humans, thus offering broader application prospects. With the development of deep learning and large language models, pure vision-based navigation models have gradually evolved into two main branches: end-to-end vision-language-action models and models that combine vision-language models with underlying controllers.

[0005] Vision-language-action models: Representative works include Google's RoboticTransformer2. These models attempt to build an end-to-end "generalist" brain, directly taking visual images and natural language commands as input, passing them through a huge Transformer network, and directly outputting the robot's low-level control signals. Their advantages lie in the high degree of model integration and strong generalization ability.

[0006] Visual language model (VLM) and underlying controller collaborative model: Representative works include SayCan or LM-Nav. These methods employ a layered architecture: the upper layer utilizes visual language models such as GPT-4V and Gemini for semantic understanding and high-level planning, for example, breaking down "getting an apple" into "finding the table," "navigating to the table," and "grabbing"; the lower layer retains traditional control algorithms to execute specific motion control. This architecture combines the reasoning power of large models with the stability of traditional control. Although existing "visual language + underlying collaborative" architectures have solved the semantic navigation problem to some extent, they still face significant challenges in practical engineering implementation.

[0007] The current collaborative architecture's heavy reliance on high-bandwidth networks leads to incompatibility with the environment. Existing collaborative solutions typically assume the robot is in an ideal network environment, requiring real-time uploading of continuous video streams to a large cloud-based model for decision-making. However, in real-world robot operating environments such as basements, outdoor environments, or areas with signal interference, network bandwidth is often limited and unstable. Video stream transmission is highly susceptible to network congestion and high latency, causing decision-making links to break down. Furthermore, there is a coupling risk between high-level decision-making and low-level security. In the current architecture, if the large cloud-based model fails to return instructions in a timely manner due to network latency, computational time consumption, or "illusion," the low-level controller often remains in a "blind waiting" state or executes outdated instructions, lacking an independent low-level security takeover mechanism, which can easily lead to collision incidents.

[0008] When quadruped robots perform natural language navigation tasks in real unstructured environments, they face problems such as coupling between high-level strategies and low-level control, unstable collaboration due to limited network bandwidth and communication jitter, difficulty in reproducing and locating faults in the perception / control link, and lack of clear safety takeover boundaries.

[0009] Therefore, a method for autonomous navigation of quadruped robots based on a brain-brain collaborative architecture and a large language model is proposed to solve the above problems. Summary of the Invention

[0010] The purpose of this invention is to provide an autonomous navigation method for quadruped robots based on a brain-brain collaborative architecture. Through a low-bandwidth strategy of on-demand perception, a clear high-low layer decoupling architecture, and a full-link diagnostic mechanism, robust navigation of the robot in real-world constrained environments is achieved.

[0011] To achieve the above objectives, this invention provides an autonomous navigation method for quadruped robots based on a cerebellum-cerebrum collaborative architecture. This method decouples the navigation task through a hierarchical cerebellum-cerebrum architecture, specifically including the following steps: S1: The large language model Agent acts as the brain to process semantic understanding and long-term planning; the local navigation stack acts as the cerebellum to handle path tracking and safety monitoring. S2: Optimize the perception link through on-demand image capture and multi-source fusion strategies, trigger image acquisition at the decision moment and combine enhancement strategies to obtain key frames; S3: Visual depth information and LiDAR data are fused at the cerebellum level to construct a local environment model covering the entire field of vision; S4: Establish a full-link decision context diagnosis mechanism to record the prompts, images, local cost map snapshots and underlying instructions for each decision, and combine them with a temporary TF link supplementation mechanism to accurately reproduce and locate occasional navigation failures.

[0012] Preferably, the cerebellum-brain hierarchical architecture includes a high-level decision-making module, a low-level execution and security module, and a dual-channel interaction interface; The high-level decision-making module is responsible for semantic understanding and long-term planning. Its input interfaces include natural language task instructions, environmental image data, snapshot summaries of local cost maps, and sector summaries of laser scanning data. The output interface of the high-level decision-making module is configured as an action client to send navigation targets containing discrete path points.

[0013] Preferably, the underlying execution and safety module includes a controller service and a safety monitor. The controller service acts as an action server, receives navigation targets and uses a local planner to calculate chassis speed commands. The safety monitor runs independently and has the highest priority control permissions.

[0014] Preferably, the cerebellum and cerebellum interact via a dual-channel interface using "structured cue words + action primitives," and the decision input variable P is defined as a concatenation of multimodal features: ; in This indicates a feature concatenation operation. and These represent the image encoder and the map summary encoder, respectively. Indicates system preset text, Representing user natural language commands, This indicates the best image selected. This represents the current local map.

[0015] Preferably, in S2, an on-demand image acquisition strategy is set based on the state, and an image acquisition trigger function is defined. When the high-level decision-making module enters the decision-making state, the acquisition time window is opened. The set of image sequences within the acquisition time window is denoted as : ; from Select the freshest frame image Upload to the cloud or large model interface, and keep the camera data stream silent at other times.

[0016] Preferably, in step S3, the depth image data from the camera is obtained through the depth map conversion module. With camera intrinsic parameter matrix By using projection transformation, the 3D visual depth information is projected onto a 2D plane under the robot's base link coordinate system, thus constructing a laser dataset obtained after simulating depth camera data. : ; in, The height of the point cloud. To effectively scan the height range, The coordinate position in a two-dimensional plane; The laser dataset obtained by simulating depth camera data through the QoS bridging module With virtual observation data published using the best strategy, local cost maps At any moment The update function is expressed as: ; in Represents the obstacle expansion function. Spatial union fusion of heterogeneous sensor data Represents the confidence function. This represents the laser dataset obtained after simulating depth camera data. Through the above steps, a full-view environment model was built at the bottom layer.

[0017] Preferably, the end-to-end decision context diagnosis mechanism in S4 includes an independent safety monitoring module based on control laws, which calculates the minimum observation distance of obstacles around the robot in real time. : ; The safety monitoring module controls the underlying speed commands. control law The definition is as follows: ; in The preset physical emergency stop safety threshold, The threshold for emergency stop cancellation. This is the deceleration coefficient.

[0018] Preferably, during the execution of a navigation task, when a new task instruction is received, the high-level decision-making module constructs... Call the inference function of the large language model LLM Generate the next action : ; The original returned data is recorded synchronously for subsequent diagnosis; after obtaining the original returned data, it is verified through the policy validation module. A legality check and amplitude limit are performed. If the forward laser scanning distance is less than the preset "decision safety threshold",... The verification module suppresses forward commands and forces them to switch to a rotation search strategy. Once the navigation target passes the verification, it is sent to the underlying execution and safety module. The underlying controller performs trajectory sampling and scoring on the local cost map based on the DWB algorithm or MPC algorithm, and generates speed commands to drive the robot's movement. When the action execution status returns to success or abort, the high-level decision module enters the next round of decision-making loop to ensure a steady-state timing of "execution completed first, then decision made".

[0019] Therefore, the present invention employs the above-mentioned autonomous navigation method for quadruped robots based on a cerebellum-cerebrum collaborative architecture, and the technical effects are as follows: (1) The present invention physically and logically separates semantic decision-making from real-time execution, which can significantly improve the safety and robustness of the robot in complex environments.

[0020] (2) This invention designs a hierarchical architecture, in which the "brain" is responsible for semantic understanding, image analysis and long-range goal planning through a large language model; while the "cerebellum" runs independently on the robot through a local navigation system, using local lidar and cost map to be responsible for real-time path tracking, dynamic obstacle avoidance and safe emergency stop. Through this collaborative mechanism, even if the brain's decision-making is delayed or the network is disconnected, the cerebellum can still independently maintain the robot's steady-state movement or perform forced braking based on local perception data, thereby effectively blocking the downward propagation of faults and taking into account both the intelligent decision-making ability of the high-level layer and the execution accessibility and safety of the low-level layer.

[0021] (3) The present invention integrates visual depth information with lidar data, which can significantly improve the motion stability and passability of the robot in complex unstructured environments.

[0022] (4) The cerebellar execution layer of this invention adopts a strategy of visual and laser fusion. By converting the three-dimensional depth map acquired by the depth camera into two-dimensional laser simulation data, a local cost map is constructed. Through the local map and the depth camera, the cerebellum can construct a model of the surrounding environment, and use the robot system controller to perform accurate path tracking under the constraints of multi-source information, and cooperate with the safety module to monitor the distance to obstacles in real time. This fusion mechanism effectively balances the safe obstacle avoidance distance and the accessibility of path execution, eliminates blind spots and false detections caused by single-modal perception, reduces erroneous actions and stuttering during navigation, and ensures the continuous steady-state operation of the robot.

[0023] (5) This invention establishes a full-link decision context diagnosis mechanism, recording the prompts, images, local cost map snapshots, and underlying instructions for each decision, and in conjunction with a temporary TF link replenishment mechanism. This enables occasional navigation failures to be accurately reproduced and located, filling the gap in the existing end-to-end system's difficulty in debugging and attribution. Attached Figure Description

[0024] Figure 1 This is a flowchart of an autonomous navigation method for a quadruped robot based on a cerebellum-cerebrum collaborative architecture, according to the present invention. Figure 2 This is a diagram of the autonomous navigation architecture of a quadruped robot based on a brain-brain collaborative architecture and a large language model in an embodiment of the present invention. Detailed Implementation

[0025] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0026] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0027] Example 1 like Figures 1-2 As shown, this invention provides an autonomous navigation method for quadruped robots based on a cerebellum-cerebrum collaborative architecture. The navigation task is decoupled through a hierarchical cerebellum-cerebrum architecture, specifically including the following steps: S1: A large language model agent acts as the brain for semantic understanding and long-term planning. This design allows the execution layer to operate independently of the decision-making layer. Even if the brain fails due to network latency or reasoning errors, the cerebellum can still independently perform obstacle avoidance or safe braking based on the local cost map, thus preventing the failure from propagating to the lower layers and ensuring the robot's physical safety. The local navigation stack acts as the cerebellum for path tracking and safety monitoring. The cerebellum execution layer employs a vision-laser fusion strategy, converting the 3D depth map acquired by the depth camera into 2D laser simulation data to construct a local cost map. Through the local map and depth camera, the cerebellum can construct a model of the surrounding environment. Using the robot system controller, precise path tracking is performed under the constraints of multi-source information, while the safety module monitors obstacle distances in real time. This fusion mechanism effectively balances safe obstacle avoidance distance and path execution accessibility, eliminating blind spots and false detections caused by single-modal perception, reducing erroneous actions and stuttering during navigation, and ensuring the robot's continuous steady-state operation.

[0028] The cerebellum-brain hierarchical architecture includes a high-level decision-making module, a low-level execution and security module, and a dual-channel interaction interface; The high-level decision-making module is responsible for semantic understanding and long-term planning. Its input interfaces include natural language task instructions, environmental image data, snapshot summaries of local cost maps, and sector summaries of laser scanning data. The output interface of the high-level decision-making module is configured as an action client to send navigation targets containing discrete path points.

[0029] The underlying execution and safety module includes a controller service and a safety monitor. The controller service acts as an action server, receiving navigation targets and using the local planner to calculate chassis speed commands. The safety monitor runs independently and has the highest priority control permissions.

[0030] The cerebellum and cerebellum interact through a dual-channel interface using "structured cue words + action primitives," and the decision input variable P is defined as a concatenation of multimodal features: ; in This indicates a feature concatenation operation. and These represent the image encoder and the map summary encoder, respectively. This indicates the system's preset text, which includes a textual description of the robot's physical constraints, task boundaries, behavioral guidelines, and the current system state. It represents the user's natural language instructions. The natural language instructions received from the user interface represent the long-term planning goals that need to be achieved at present. This indicates the best image selected. This represents the current local map.

[0031] S2: By optimizing the perception link through on-demand image capture and multi-source fusion strategies, image acquisition is triggered at the decision moment and key frames are obtained in combination with enhancement strategies. This avoids the full video stream from occupying bandwidth and ensures communication stability in weak network environments.

[0032] S2 sets up an on-demand image acquisition strategy based on state, and defines an image acquisition trigger function. When the high-level decision-making module enters the decision-making state, the data collection time window is opened. The set of image sequences within the acquisition time window is denoted as : ; from Select the freshest frame image Upload to the cloud or large model interface, and keep the camera data stream silent at other times.

[0033] S3: Visual depth information and LiDAR data are fused at the cerebellar level to construct a local environment model covering the entire field of view. This fusion balances accessibility and obstacle avoidance safety, reducing erroneous actions caused by blind spots of a single sensor. In S3, depth image data from the camera is acquired through a depth map conversion module. With camera intrinsic parameter matrix By using projection transformation, the 3D visual depth information is projected onto a 2D plane under the robot's base link coordinate system, thus constructing a laser dataset obtained after simulating depth camera data. : ; in, The height of the point cloud. To effectively scan the height range, The coordinate position in a two-dimensional plane; The laser dataset obtained by simulating depth camera data through the QoS bridging module With virtual observation data published using the best strategy, local cost maps At any moment The update function is expressed as: ; in Represents the obstacle expansion function. Represents the spatial union fusion of heterogeneous sensor data, where This represents a confidence function that reduces the confidence level of older observations over time. This represents the laser dataset obtained after simulating depth camera data.

[0034] Through the above steps, a full-view environment model was built at the bottom layer.

[0035] S4: Establish a full-link decision context diagnostic mechanism, recording prompts, images, local cost map snapshots, and underlying commands for each decision. Combined with a temporary TF (Telematics Transfer) link replenishment mechanism, it accurately reproduces and locates occasional navigation failures. This fills the gap in existing end-to-end systems where debugging and attribution are difficult.

[0036] The end-to-end decision context diagnostic mechanism in S4 includes an independent safety monitoring module based on control laws, which calculates the minimum observation distance of obstacles around the robot in real time. : ; The safety monitoring module controls the underlying speed commands. control law The definition is as follows: ; in The preset physical emergency stop safety threshold, The threshold for emergency stop cancellation. This is the deceleration coefficient.

[0037] During the execution of a navigation task, when a new task instruction is received, the high-level decision-making module is constructed. Call the inference function of the large language model LLM Generate the next action : ; The original returned data is recorded synchronously for subsequent diagnosis; after obtaining the original returned data, it is verified through the policy validation module. A legality check and amplitude limit are performed. If the forward laser scanning distance is less than the preset "decision safety threshold",... The verification module suppresses forward commands and forces them to switch to a rotation search strategy. Once the navigation target passes the verification, it is sent to the underlying execution and safety module. The underlying controller performs trajectory sampling and scoring on the local cost map based on the DWB algorithm or MPC algorithm, and generates speed commands to drive the robot's movement. When the action execution status returns to success or abort, the high-level decision module enters the next round of decision-making loop to ensure a steady-state timing of "execution completed first, then decision made".

[0038] Therefore, this invention adopts the above-mentioned autonomous navigation method for quadruped robots based on a brain-brain collaborative architecture. Through a low-bandwidth strategy of on-demand perception, a clear high-low layer decoupling architecture, and a full-link diagnostic mechanism, it achieves robust navigation of the robot in real-world constrained environments.

[0039] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. An autonomous navigation method for a quadruped robot based on a cerebellum-cerebrum collaborative architecture, characterized in that, Decoupling navigation tasks through a hierarchical architecture involving the cerebellum and cerebellum includes the following steps: S1: The large language model Agent acts as the brain to process semantic understanding and long-term planning; the local navigation stack acts as the cerebellum to handle path tracking and safety monitoring. S2: Optimize the perception link through on-demand image capture and multi-source fusion strategies, trigger image acquisition at the decision moment and combine enhancement strategies to obtain key frames; S3: Visual depth information and LiDAR data are fused at the cerebellum level to construct a local environment model covering the entire field of vision; S4: Establish a full-link decision context diagnosis mechanism to record the prompts, images, local cost map snapshots and underlying instructions for each decision, and combine them with a temporary TF link supplementation mechanism to accurately reproduce and locate occasional navigation failures.

2. The method according to claim 1, wherein, The cerebellum-brain hierarchical architecture includes a high-level decision-making module, a low-level execution and security module, and a dual-channel interaction interface; The high-level decision-making module is responsible for semantic understanding and long-term planning. Its input interfaces include natural language task instructions, environmental image data, snapshot summaries of local cost maps, and sector summaries of laser scanning data. The output interface of the high-level decision-making module is configured as an action client to send navigation targets containing discrete path points.

3. The autonomous navigation method for a quadruped robot based on a cerebellum-cerebrum collaborative architecture according to claim 2, characterized in that, The underlying execution and safety module includes a controller service and a safety monitor. The controller service acts as an action server, receiving navigation targets and using the local planner to calculate chassis speed commands. The safety monitor runs independently and has the highest priority control permissions.

4. The autonomous navigation method for a quadruped robot based on a cerebellum-cerebrum collaborative architecture according to claim 2, characterized in that, The cerebellum and cerebellum interact through a dual-channel interface using "structured cue words + action primitives," and the decision input variable P is defined as a concatenation of multimodal features: ； in This indicates a feature concatenation operation. and These represent the image encoder and the map summary encoder, respectively. Indicates system preset text, Representing user natural language commands, This indicates the best image selected. This represents the current local map.

5. The autonomous navigation method for a quadruped robot based on a cerebellum-cerebrum collaborative architecture according to claim 1, characterized in that, In S2, an on-demand image acquisition strategy is set based on state, and an image acquisition trigger function is defined. When the high-level decision-making module enters the decision-making state, the acquisition time window is opened. The set of image sequences within the acquisition time window is denoted as : ； from Select the freshest frame image Upload to the cloud or large model interface, and keep the camera data stream silent at other times.

6. The autonomous navigation method for a quadruped robot based on a cerebellum-cerebrum collaborative architecture according to claim 1, characterized in that, In S3, depth image data from the camera is obtained through a depth map conversion module. With camera intrinsic parameter matrix By using projection transformation, the 3D visual depth information is projected onto a 2D plane under the robot's base link coordinate system, thus constructing a laser dataset obtained after simulating depth camera data. : ； in, The height of the point cloud. To effectively scan the height range, The coordinate position in a two-dimensional plane; The laser dataset obtained by simulating depth camera data through the QoS bridging module With virtual observation data published using the best strategy, local cost maps At any moment The update function is expressed as: ； in Represents the obstacle expansion function. Spatial union fusion of heterogeneous sensor data Represents the confidence function. This represents the laser dataset obtained after simulating depth camera data. Through the above steps, a full-view environment model was built at the bottom layer.

7. The autonomous navigation method for a quadruped robot based on a cerebellum-cerebrum collaborative architecture according to claim 1, characterized in that, The end-to-end decision context diagnostic mechanism in S4 includes an independent safety monitoring module based on control laws, which calculates the minimum observation distance of obstacles around the robot in real time. : ； The safety monitoring module controls the underlying speed commands. control law The definition is as follows: ； in The preset physical emergency stop safety threshold, The threshold for emergency stop cancellation. This is the deceleration coefficient.

8. The autonomous navigation method for a quadruped robot based on a cerebellum-cerebrum collaborative architecture according to claim 2, characterized in that, During the execution of a navigation task, when a new task instruction is received, the high-level decision-making module is constructed. Call the inference function of the large language model LLM Generate the next action : ； The original returned data is recorded synchronously for subsequent diagnosis; after obtaining the original returned data, it is verified through the policy validation module. A legality check and amplitude limit are performed. If the forward laser scanning distance is less than the preset decision safety threshold, the system will take appropriate action. The verification module suppresses forward commands and forces them to switch to a rotation search strategy. Once the navigation target passes the verification, it is sent to the underlying execution and safety module. The underlying controller performs trajectory sampling and scoring on the local cost map based on the DWB algorithm or MPC algorithm, and generates speed commands to drive the robot's movement. When the action execution status returns to success or abort, the higher-level decision module enters the next round of decision-making loop.

Images