Wheel-legged robot obstacle avoidance method and system based on modal switching space-time reachable tube

Abstract

The application provides a wheel-legged robot obstacle avoidance method and system based on modal switching space-time reachable tube, which comprises the following steps: acquiring the current state and mode of the robot, the target mode and the environment space-time occupation information; predicting the robot state set at continuous time according to the mode switching dynamics; and forming a time-indexed modal switching space-time reachable tube according to the space occupation range of the robot body and the wheel-legged mechanism under each state; determining the switching feasible region and the starting window according to the space-time relationship of the reachable tube, static obstacles, dynamic occupation and terrain constraints; sampling the keep mode trajectory and the mode switching trajectory; respectively calling the corresponding cost channel for segmented evaluation according to before switching, during switching and after switching; generating the target trajectory, the mode switching starting time and the reference control instruction; and performing after safety constraint correction. The application can jointly express the environment passability under different motion modes and the accessibility of the mode switching process, and can determine the safe switching window in combination with the real-time state of the robot.

Description

Technical Field

[0001] This invention belongs to the field of robot autonomous navigation, hybrid power system planning and safety control technology, specifically involving a method and system for obstacle avoidance of wheeled and legged robots based on mode switching spatiotemporal reachability tubes. Background Technology

[0002] Wheeled-legged robots combine the high-speed cruising capabilities of wheels with the obstacle-crossing maneuverability of legs, and can switch movement modes according to terrain conditions within the same task. In unstructured environments, the same spatial location often has different accessibility for wheeled and legged movements: flat areas are suitable for wheeled movement, while heterogeneous terrain may require legged movement; at the same time, the environment also contains dynamic obstacles, static obstacles, and restricted areas that are impassable by both wheels and legs, creating strong coupling constraints between perception, planning, and control.

[0003] Existing local planning methods are typically based on a single kinematic model, making it difficult to explicitly distinguish the differences in drivability under different motion modes. Furthermore, they struggle to simultaneously consider path feasibility and mode switching requirements when restricted areas or heterogeneous terrain exist. Specifically, existing cost map-based navigation methods usually compress environmental information into a single cost function in two-dimensional space, assuming the robot adopts a fixed motion mode. However, for wheeled and legged robots, their mobility depends not only on their spatial location but also on their current motion pattern. For example, the same obstacle area may be impassable for wheeled movement but highly passable for legged movement; conversely, legged movement may incur additional energy consumption in flat areas. Therefore, traditional environment representation methods independent of motion modes cannot fully utilize the multimodal motion advantages of wheeled and legged robots.

[0004] Existing environmental cost mapping or elevation mapping methods often represent accessibility as a scalar field independent of motion modes, failing to uniformly express morphological coupling rules such as "wheeled vehicles are impassable while legged vehicles are accessible" or "both modes are impassable." While existing reinforcement learning methods can be used for robot obstacle avoidance decisions, some methods rely on a large number of training samples and struggle to explicitly integrate environmental accessibility constraints with safety control mechanisms.

[0005] On the other hand, existing methods, when determining motion modes, typically focus on whether the robot can traverse the terrain ahead after switching to the target motion mode, while treating the switch between wheeled and legged modes as instantaneous or only using a fixed time lag. However, mode switching involves continuous processes such as speed adjustment, changes in body height, posture adjustment, and the deployment or retraction of wheel and leg mechanisms. During this process, the robot exhibits dynamic capabilities and spatial shape that change over time.

[0006] When the robot's current speed is high, the target terrain is close, and the switching area is adjacent to static obstacles or dynamic targets, even if the target terrain is traversable in the target motion mode, the robot may not be able to complete the mode switch before reaching the target terrain, or the wheel-leg mechanism may interfere with obstacles during deployment. Selecting a mode solely based on the traversability cost of the target mode cannot determine when the mode switch should be initiated, nor can it verify whether the switching process itself is safe and reachable.

[0007] Therefore, it is necessary to establish a predictive object that can simultaneously describe the evolution of mode switching states, configuration-related space occupancy, environmental dynamic changes, and the timing of switching initiation, and to use this predictive object for both trajectory planning and safety control, thereby solving the problem that the target mode is traversable but the switching process is unattainable or the switching timing is too late.

[0008] Based on the above problems, there is a need for a wheeled and legged robot obstacle avoidance method and system that can jointly express the environmental accessibility and mode switching process accessibility under different motion modes, and can determine the safe switching window based on the robot's real-time status. Summary of the Invention

[0009] This invention provides a method and system for obstacle avoidance of wheeled and legged robots based on a mode-switching spatiotemporal reachability tube, in order to solve the problem that, during mode switching, wheeled and legged robots may be able to pass through the target mode but become unreachable during the switching process due to improper switching timing or unsafe switching process itself.

[0010] To achieve the above objectives, this invention provides an obstacle avoidance method for wheeled-legged robots based on a mode-switching spatiotemporal reachability tube, comprising: The current motion state, current motion mode, and environmental information of the wheeled-legged robot are acquired. The environmental information includes static environment occupancy information, dynamic obstacle state information, and terrain semantic information. Based on the environmental information, a static occupancy representation, a dynamic obstacle spatiotemporal occupancy representation with time index, and a terrain semantic grid are obtained. Then, a modal correlation drivability field dual channel including a wheel cost channel and a leg cost channel is constructed, and the target motion mode is determined based on the cost comparison results of the wheel cost channel and the leg cost channel. When the target motion mode is different from the current motion mode, the robot state during the mode switching phase change is predicted time by time according to the preset mode switching dynamics model and prediction step size, so as to obtain a set of robot states corresponding to multiple prediction time. The mode switching phase is used to characterize the switching progress from the current motion mode to the target motion mode. Based on the robot state set at each prediction time, the corresponding space occupancy set at each prediction time is determined. The space occupancy set is the union of the body occupancy set and the wheel-leg mechanism occupancy set at the corresponding prediction time. Each prediction time and its corresponding space occupancy set are associated in chronological order to form a mode switching spatiotemporal reachability tube. The mode switching spatiotemporal reachability tube is spatiotemporally aligned with the static occupancy representation, the dynamic obstacle spatiotemporal occupancy representation, and the terrain constraints determined based on the terrain semantic grid to obtain the mode switching safety margin and the mode switching feasible time interval. Generate a mode-maintaining candidate trajectory for maintaining the current motion mode and a mode-switching candidate trajectory for switching to the target motion mode within the prediction time domain. Filter the mode-switching candidate trajectories according to the feasible time interval for mode switching. Evaluate the segmented cost of the filtered mode-switching candidate trajectories. Determine the target trajectory, mode-switching initiation time, and reference control command based on the evaluation results and the mode-maintaining candidate trajectories. The reference control command includes recommended linear velocity, recommended angular velocity, and target motion mode. Using the safety margin of the mode switching not being lower than a preset safety threshold and the fact that the spatiotemporal reachability tube of the mode switching does not intersect with the area constrained by the static occupancy representation, the spatiotemporal occupancy representation of the dynamic obstacle, and the terrain semantic grid as hard constraints, the reference control command is modified to maintain the current motion mode, or switch to the target motion mode or perform safety braking at the start of the mode switching.

[0011] In one embodiment, obtaining the static occupancy representation, the time-indexed dynamic obstacle spatiotemporal occupancy representation, and the terrain semantic raster based on the environmental information includes: The occupation grid map is updated based on the static environment occupation information, and a symbolic Euclidean distance field map is generated using Euclidean distance transformation, with the symbolic Euclidean distance field map serving as the static occupation representation. Based on the dynamic obstacle state information, motion trend prediction is performed, the predicted position is mapped to a grid of future time series, and the dynamic occupancy value of the grid is updated according to the time decay weight to generate a spatiotemporal occupancy representation of the dynamic obstacle with time index. The terrain semantic information is obtained, and a terrain semantic grid is labeled according to the terrain semantic information. The terrain semantic grid includes at least three types of terrain semantic labels: flat area, heterogeneous passable area, and bimodal restricted area.

[0012] In one embodiment, the step of further constructing a modal-related drivability field dual-channel including a wheel-walking cost channel and a leg-walking cost channel, and determining the target motion mode based on the cost comparison results of the wheel-walking cost channel and the leg-walking cost channel, includes: The single-point cost is calculated for each grid position within a local window centered on the robot's current position in both wheel-walking and leg-walking modes. The single-point cost is obtained by weighted summation of geometric safety cost, dynamic obstacle spatiotemporal occupancy cost, modal-terrain coupling cost, and state-mode coupling cost. Wherein, the geometric safety cost is determined based on the static occupancy representation, the dynamic obstacle spatiotemporal occupancy cost is determined based on the dynamic obstacle spatiotemporal occupancy representation, the modality-terrain coupling cost is determined based on the correspondence between the terrain semantic grid and the motion pattern, and the state-mode coupling cost is determined based on the matching relationship between the current motion state and the motion pattern; Based on the single-point cost, a wheel-walking mode cost slice and a leg-walking mode cost slice are generated respectively, and stacked into a dual-channel tensor, which serves as the wheel-walking cost channel and the leg-walking cost channel; The cost values ​​of the current position in wheel mode and leg mode are obtained and compared according to the wheel movement cost channel and the leg movement cost channel. If the cost of wheel mode is less than the cost of leg mode, wheel mode is set as the target movement mode; otherwise, leg mode is set as the target movement mode.

[0013] In one embodiment, the step of predicting the robot state during the mode switching phase change according to a preset mode switching dynamics model and prediction step size, to obtain a set of robot states corresponding to multiple prediction times, includes: The uncertainty interval of the current motion state is used as the initial state set; The initial state set is propagated step by step according to the mode switching dynamics model expressed in discrete time form, with the prediction step size as the time interval. By superimposing a set of perturbations determined by model error, perception error and execution delay at each prediction step, a set of robot states corresponding to multiple prediction times is obtained. The robot state set includes the robot's position, heading angle, linear velocity, angular velocity, body height, body posture, wheel and leg mechanism deployment degree, wheel and leg mechanism speed, and mode switching phase.

[0014] In one embodiment, the step of determining the spatial occupancy set corresponding to each prediction time based on the robot state set at each prediction time, wherein the spatial occupancy set is the union of the body occupancy set and the wheel-leg mechanism occupancy set at the corresponding prediction time, and associating each prediction time and its corresponding spatial occupancy set in chronological order to form a mode-switching spatiotemporal reachability tube, includes: For each predicted moment, based on the position, body posture, body height, and wheel-leg mechanism deployment degree in the robot state set at that predicted moment, the robot body geometric model and the wheel-leg mechanism geometric model are transformed to a unified environmental coordinate system, and the body occupancy set and the wheel-leg mechanism occupancy set are determined respectively. The union of the body occupancy set and the wheel-leg mechanism occupancy set is taken as the spatial occupancy set at the predicted moment; The spatial occupancy set changes with the mode switching phase to characterize the continuous change of the robot's external boundary during the mode switching process; By associating each predicted moment and its corresponding spatial occupancy set in chronological order, a spatiotemporal reachability tube for mode switching from the current motion mode to the target motion mode is formed.

[0015] In one embodiment, the step of performing a spatiotemporal alignment determination between the mode switching spatiotemporal reachability tube and the static occupancy representation, the dynamic obstacle spatiotemporal occupancy representation, and the terrain constraints determined based on the terrain semantic grid to obtain the mode switching safety margin and the mode switching feasible time interval includes: For each spatiotemporal slice in the modality switching spatiotemporal reachability tube, the spatial occupancy set corresponding to the prediction time is intersected with the region extracted from the static occupancy representation, the region extracted from the dynamic obstacle spatiotemporal occupancy representation corresponding to the prediction time, and the region extracted from the terrain semantic raster that does not meet the support conditions. The mode switching safety margin is determined based on the minimum spatial distance between each spatiotemporal slice and the above three regions when no intersection occurs. Using candidate start times as parameters, a family of reachable tubes parameterized by start times is generated. The intersection detection and state constraint determination are performed on the spatiotemporal reachable tubes corresponding to each candidate start time. The set of consecutive start times where the mode switching safety margin is not less than a preset safety threshold, all state constraints are satisfied throughout the process, and the state of the switching terminal falls into the feasible state domain of the target motion mode is determined as the feasible time interval for mode switching. The earliest feasible switching time within the feasible time interval for mode switching is determined by the time required for the current motion mode to decelerate to the allowable switching speed, and the latest safe switching time is determined by the remaining time to reach the target terrain boundary or interfere with the obstacle.

[0016] In one embodiment, the process of generating a mode-preserving candidate trajectory that maintains the current motion mode within the prediction time domain and a mode-switching candidate trajectory that switches to the target motion mode involves filtering the mode-switching candidate trajectories based on the feasible time interval for mode switching, evaluating the segmented cost of the filtered mode-switching candidate trajectories, and determining the target trajectory, the mode-switching initiation time, and the reference control command based on the evaluation results and the mode-preserving candidate trajectories, including: Generate mode-preserving candidate trajectories that maintain the current motion mode within the prediction time domain, as well as mode-switching candidate trajectories that switch to the target motion mode; The candidate trajectories for mode switching are filtered according to the feasible time interval for mode switching, and the candidate trajectories for mode switching that do not meet the constraints of the feasible time interval for mode switching are eliminated. The selected candidate trajectories for mode switching are divided into pre-switching, mid-switching, and post-switching segments according to the switching start and completion times. Segmented cost evaluations are performed on each segment, including: accumulating the modal-related traversability cost corresponding to the current motion mode for the pre-switching segment; accumulating the transition cost determined by the mode switching phase, robot state, and environmental spatiotemporal occupancy for the mid-switching segment; and accumulating the modal-related traversability cost corresponding to the target motion mode for the post-switching segment. The total cost of the candidate trajectory for mode switching is then calculated by adding the mode switching time cost, mode switching energy cost, reachability risk cost determined based on the mode switching safety margin, and frequent mode switching suppression cost. Calculate the total cost of the mode-keeping candidate trajectory for maintaining the current motion mode, compare the total cost of each mode-switching candidate trajectory with the total cost of the mode-keeping candidate trajectory, select the trajectory with the minimum total cost as the target trajectory, and determine the mode-switching initiation time and reference control command based on the target trajectory.

[0017] In one embodiment, the hard constraint that the mode switching safety margin is not lower than a preset safety threshold and that the regions constrained by the mode switching spatiotemporal reachability tube and the static occupancy representation, the dynamic obstacle spatiotemporal occupancy representation, and the terrain semantic grid do not intersect is used to modify the reference control command for safety constraints, controlling the wheeled robot to maintain the current motion mode, switch to the target motion mode at the start of the mode switching, or perform safety braking includes: The reference control command is input into the model prediction controller; The model prediction controller reads the mode switching spatiotemporal reachability tube cross section at each prediction step, and uses the fact that the cross section at each prediction time does not intersect with the areas constrained by the static occupancy representation, the dynamic obstacle prediction occupancy representation, and the terrain semantic grid, and that the mode switching safety margin is not lower than a preset safety threshold as a hard constraint to perform safety constraint correction on the reference control command. When the mode switching time requested by the reference control command is earlier than the earliest feasible switching time in the feasible time interval of the mode switching, the current motion mode is maintained and the deceleration waiting trajectory is executed. When the mode switching time requested by the reference control command falls within the feasible time interval for mode switching, the switch to the target motion mode is executed at the mode switching initiation time. If no feasible mode switching time interval exists within the prediction time domain, the mode switching request is rejected and a safety brake is executed.

[0018] In one embodiment, it further includes: The distance required to complete the mode switch is determined based on the current speed on the candidate trajectory, the allowed deceleration, the mode switch duration, and the execution delay. The available distance is determined based on the trajectory distance from the robot's current position to the target terrain boundary; The available distance is subtracted from the distance required to complete the mode switch to obtain the switching distance margin; When determining the feasible time interval for the mode switching, it is also required that the switching distance margin be not less than zero.

[0019] In one embodiment, the method further includes: generating an auxiliary decision vector based on the target trajectory, the auxiliary decision vector including recommended linear velocity, recommended angular velocity, current motion mode, target motion mode, expected switching duration, mode switching safety margin, switching distance margin, earliest feasible switching time, and supervision instruction.

[0020] In one embodiment, the state-mode coupling cost is determined based on at least one of current velocity, current attitude, mode switching phase, target terrain distance, and mode switching safety margin, including: When the robot's linear velocity is high and the terrain in front is flat, an additional penalty is applied to switching to leg-walking mode. When the robot approaches a heterogeneous passable area and the ESDF distance is small, an additional penalty is imposed on maintaining the wheel mode. When the leg-walking modal coefficient is higher than the preset threshold and the candidate mode is the leg-walking mode, and the terrain ahead is a heterogeneous passable area, the cost of reducing the leg-walking mode is reduced.

[0021] Based on the same inventive concept, this invention also proposes an obstacle avoidance system for wheeled-legged robots based on a mode-switching spatiotemporal reachability tube, comprising: The dual-channel mobility field construction module is used to acquire the current motion state, current motion mode, and environmental information of the wheeled and legged robot. The environmental information includes static environment occupancy information, dynamic obstacle state information, and terrain semantic information. Based on the environmental information, a static occupancy representation, a dynamic obstacle spatiotemporal occupancy representation with time index, and a terrain semantic grid are obtained. Then, a modality-related mobility field dual-channel including a wheeled cost channel and a legged cost channel is constructed, and the target motion mode is determined based on the cost comparison results of the wheeled cost channel and the legged cost channel. The robot state set prediction module is used to predict the robot state during the mode switching phase change according to a preset mode switching dynamics model and prediction step size when the target motion mode is different from the current motion mode, and obtain a set of robot states corresponding to multiple prediction times. The mode switching phase is used to characterize the progress of the switch from the current motion mode to the target motion mode. The mode switching reachability prediction module is used to determine the space occupancy set corresponding to each prediction time based on the robot state set at each prediction time. The space occupancy set is the union of the body occupancy set and the wheel and leg mechanism occupancy set at the corresponding prediction time. The prediction time and its corresponding space occupancy set are associated in chronological order to form a mode switching spatiotemporal reachability tube. The mode switching feasibility determination module is used to perform spatiotemporal alignment determination between the mode switching spatiotemporal reachability tube and the static occupancy representation, the dynamic obstacle spatiotemporal occupancy representation, and the terrain constraints determined based on the terrain semantic grid, so as to obtain the mode switching safety margin and the mode switching feasible time interval. The target trajectory planning module is used to generate mode-maintaining candidate trajectories for maintaining the current motion mode and mode-switching candidate trajectories for switching to the target motion mode within the prediction time domain. The mode-switching candidate trajectories are filtered according to the feasible time interval of the mode-switching, and the segmented cost of the filtered mode-switching candidate trajectories is evaluated. Based on the evaluation results and the mode-maintaining candidate trajectories, the target trajectory, the mode-switching start time, and the reference control command are determined. The reference control command includes the recommended linear velocity, the recommended angular velocity, and the target motion mode. The safety control module is used to correct the reference control command based on the hard constraints that the mode switching safety margin is not lower than a preset safety threshold and that the spatiotemporal reachability tube of the mode switching does not intersect with the area constrained by the static occupancy representation, the dynamic obstacle spatiotemporal occupancy representation and the terrain semantic grid. This allows the wheeled robot to maintain its current motion mode, switch to the target motion mode or perform safety braking at the start of the mode switching.

[0022] Based on the same inventive concept, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any of the preceding claims.

[0023] Based on the same inventive concept, embodiments of the present invention also propose a computer storage medium storing at least one executable instruction that causes a processor to perform the method described in any of the preceding claims.

[0024] Compared with existing technologies, the obstacle avoidance method for wheeled and legged robots based on mode switching spatiotemporal reachability tubes of the present invention acquires the robot's current state, current mode, static environmental occupancy, dynamic obstacle prediction, and terrain information to construct MACF cost channels corresponding to wheeled and legged movement. When the target mode differs from the current mode, the state set is propagated time-by-time based on the mode switching dynamics model, and a mode switching spatiotemporal reachability tube is formed according to the configuration footprints of the body and wheeled and legged mechanisms in each state. The reachability tube is time-aligned with static occupancy, dynamic spatiotemporal occupancy, and terrain constraints to determine the mode switching safety margin, switching distance margin, and switching feasible time interval. Candidate trajectories for maintaining the mode and mode switching are jointly generated, and segmented cost evaluation is performed before, during, and after switching. Finally, the mode switching request is executed or corrected by the safety supervisor and the model prediction controller, forming a closed loop of perception-accessibility modeling-switching reachability prediction-joint planning-safe execution.

[0025] Unlike traditional methods that represent environmental accessibility as a single scalar field, this invention establishes independent cost evaluation channels for wheeled and legged modes, allowing the same spatial location to have different accessibility evaluations under different motion modes. This enables joint modeling of the dual-modal motion characteristics of wheels and legs with environmental geometry, dynamic occupancy, and terrain semantics. Furthermore, unlike traditional environmental cost maps, the modally correlated accessibility field dual channels in this invention do not establish fixed costs for the environment itself, but rather establish a correlation evaluation between the robot's motion mode and the environmental state.

[0026] This invention constructs a spatiotemporal reachability tube for mode switching, enabling an explicit description of the continuous changes in speed, attitude, and wheel-leg configuration during mode switching. This allows for accurate identification of dangerous situations where the target mode is passable but the switching process itself is inaccessible. Furthermore, by aligning the reachability tube with static obstacles, dynamic occupancy, and terrain support conditions in time, a feasible time interval containing the earliest feasible switching moment and the latest safe switching moment can be determined. Combined with switching distance margin and piecewise MACF cost, candidate trajectories for holding and switching modes are jointly evaluated, effectively avoiding late switching due to excessively high current speed or insufficient remaining distance. Finally, the same spatiotemporal reachability tube for mode switching serves as both a planning screening basis and a predictive control safety constraint. When switching is infeasible, it triggers deceleration, delay, holding mode, or safety braking, ensuring consistency between the planning and execution layers and achieving safe and reliable mode switching control. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the obstacle avoidance method for a wheeled-legged robot based on a mode-switching spatiotemporal reachability tube in an embodiment of the present invention.

[0028] Figure 2This is the overall architecture of a modally coupled multi-sensory hierarchical obstacle avoidance system for wheeled and legged robots.

[0029] Figure 3 This is a schematic diagram illustrating the construction principle of the Modal Dependent Accessibility Field (MACF) provided in an embodiment of the present invention; wherein, Figure (a) shows the geometric cost. Distribution, used to characterize static geometric barriers and safety gaps independent of motion patterns; Figure (b) shows the wheel-riding modal cost channel. High costs are observed in heterogeneous passable areas and in bimodal restricted areas; Figure (c) shows the cost channel for leg-walking mode. It exhibits low cost in the same heterogeneous traversable area and high cost in the bimodal restricted area, thus explicitly expressing the differentiated traversability of the same location for different motion modes.

[0030] Figure 4 This is a schematic diagram of the perception-planning processing flow provided in an embodiment of the present invention; wherein, Figure (a) shows the original environment and start-end point settings; Figure (b) shows the Euclidean signed distance field (ESDF); and Figure (c) shows the wheel passage. Figure (d) shows the leg passage. Figure (e) shows the dual-channel fusion result; Figure (f) shows the execution trajectory generated based on this.

[0031] Figure 5 This is a schematic diagram of multimodal sensing information provided in an embodiment of the present invention; wherein, Figure (a) is a semantic occupancy grid, labeled with static obstacles, heterogeneous regions, restricted areas and dynamic targets; Figure (b) is the corresponding ESDF range field. Figure (c) shows dynamic occupancy. Figure (d) shows the cost of round-robin MACF. Figure (e) shows the cost of the MACF for leg movement. Figure (f) shows the fused observations. .

[0032] Figure 6 This is a schematic diagram comparing the traditional cost map and the MACF dual-channel method provided in an embodiment of the present invention; wherein, Figure (a) is the traditional scalar cost field. This compresses heterogeneous areas such as roadside curbs into a single, high-cost dimension lacking motion modes; Figure (b) shows the wheel-traveling passage. High cost is observed in the STEP heterogeneous region and also in the TRENCH bimodal forbidden zone; Figure (c) shows the leg passage. It exhibits low cost in the same STEP region and remains high cost in the TRENCH region, with the leg preference given by the channel difference.

[0033] Figure 7 This is a schematic diagram of the modality switching spatiotemporal reachability prediction and security monitoring process provided in an embodiment of the present invention.

[0034] Figure 8 This is a schematic diagram of the feasible startup window and switching distance margin for mode switching provided in an embodiment of the present invention.

[0035] Figure 9 The flowchart of hierarchical obstacle avoidance decision-making and motion control provided in the embodiments of the present invention.

[0036] Figure 10 Figure 1 is a schematic diagram of a typical simulation experimental environment provided in the embodiments of the present invention; wherein, Figure (a) is a static obstacle scene; Figure (b) is a dynamic obstacle scene; Figure (c) is a ravine bypass scene (including a dual-modal restricted area and a lateral bypass diagram); Figure (d) is a hybrid unstructured scene (static obstacles, dynamic obstacles, restricted areas and heterogeneous passable areas coexist).

[0037] Figure 11 This is a schematic diagram of motion mode switching timing provided in an embodiment of the present invention.

[0038] Figure 12 Figure 1 is a schematic diagram of the robot execution trajectory provided in the embodiment of the present invention; wherein, Figure (a) is the execution trajectory of a static obstacle scene; Figure (b) is the execution trajectory of a dynamic obstacle scene; Figure (c) is the execution trajectory of a ditch-bypass scene; and Figure (d) is the execution trajectory of a mixed unstructured scene.

[0039] Figure 13 This is a schematic diagram of a multi-expert reinforcement learning strategy network structure provided in an embodiment of the present invention.

[0040] Figure 14 This is a schematic diagram of the strategy training process provided in an embodiment of the present invention.

[0041] Figure 15 This is a comparison chart of navigation performance in various scenarios provided in the embodiments of the present invention.

[0042] Figure 16 The following is a comprehensive performance evaluation diagram provided for the embodiments of the present invention; wherein, Figure (a) is a radar comparison of the method of the present invention and the dynamic window method in terms of success rate, security, modal consistency and path efficiency; Figure (b) is a comparison of the target arrival time of the method of the present invention in each successful round.

[0043] Figure 17 The diagram shows a comparison of different method trajectories provided in the embodiments of the present invention; wherein, Figures (a), (b), and (c) are the successful trajectories of the method of the present invention in static obstacles, ravine detours, and mixed unstructured scenarios, respectively; Figures (d), (e), and (f) are the failed trajectories of the dynamic window method in the corresponding three types of scenarios, respectively.

[0044] Figure 18This diagram illustrates the module comparison and verification provided in this embodiment of the invention. Full MC-MPNav represents the complete method of this embodiment; w / o MACF represents removing the dual-channel modality-related accessibility field; w / o MoE represents removing the multi-expert structure; w / o Transformer represents removing the multi-modal token fusion network; w / o MPC represents removing the model prediction control security projection; DWA represents the dynamic window method baseline. The vertical axis represents the average success rate across four scenarios.

[0045] Figure 19 This is a schematic diagram of the obstacle avoidance system for a wheeled legged robot based on a mode-switching spatiotemporal reachability tube in an embodiment of the present invention.

[0046] Figure 20 This is a schematic diagram of an electronic device in an embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0048] like Figure 1 As shown, this embodiment provides an obstacle avoidance method for wheeled-legged robots based on a modal-switching spatiotemporal reachability tube, with a mobile platform possessing combined wheeled and legged motion capabilities as the application object. In other embodiments, the method can also be applied to mobile robots with multiple switchable motion modes.

[0049] The obstacle avoidance method for wheeled-legged robots based on mode-switching spatiotemporal reachability tubes in this embodiment includes: S1: Obtain the current motion state, current motion mode, and environmental information of the wheeled and legged robot. The environmental information includes static environment occupancy information, dynamic obstacle state information, and terrain semantic information. Based on the environmental information, obtain the static occupancy representation, the dynamic obstacle spatiotemporal occupancy representation with time index, and the terrain semantic grid. Then, construct a dual-channel modal correlation drivability field including the wheeled cost channel and the legged cost channel, and determine the target motion mode based on the cost comparison results of the wheeled cost channel and the legged cost channel.

[0050] In this embodiment, step S1 aims to perform environmental perception, dual-channel cost construction, and target pattern determination. The obstacle avoidance method in this embodiment can be achieved through methods such as... Figure 2 The modally coupled multi-sensor hierarchical obstacle avoidance system architecture shown includes a perception layer, a representation layer, a switching reachability layer, a decision layer, and an execution layer. The specific steps of this method are explained in detail below with reference to this architecture.

[0051] See also Figure 2 , Figure 4 and Figure 5In practical deployments, wheeled robots are equipped with multiple sensors for environmental perception. Preferably, the multi-source sensor data first undergoes time synchronization and spatial coordinate unification processing, transforming LiDAR point clouds, visual semantic information, and inertial measurement unit (IMU) information into a unified robot coordinate system. Specifically, LiDAR provides environmental geometric structure information, visual sensors provide obstacle category and terrain semantic information, and the IMU provides robot posture change information. Based on the environmental geometric structure information provided by the LiDAR point cloud, incremental mapping algorithms (such as...) can be used... Algorithms) construct static maps and update static environment occupancy information in real time. Based on obstacle categories and terrain semantic information provided by visual sensors, object detection algorithms (such as...) can be used to construct static maps and update static environment occupancy information in real time. A series of target detection algorithms obtain dynamic obstacle status information, that is, where there are dynamic obstacles and how the dynamic obstacles move.

[0052] When the robot starts moving, the system creates an empty local grid map in memory centered on the robot's current position. All grid cells are initially set to "unknown". As the robot moves, the LiDAR continuously acquires environmental point cloud data. The system uses an incremental mapping algorithm to integrate this new data into the grid map in real time and continuously update it: it maps the scanned obstacle points to the corresponding grid and marks them as "occupied", and marks the empty areas that are scanned as "free". At the same time, as the robot moves, old grids that are outside the current local window are discarded, and new unknown grids are created in the area that is newly entered in front of the window. This makes the grid map a dynamically maintained local map that grows and updates in real time as the robot moves.

[0053] In this embodiment, static environment occupancy information is obtained by processing the LiDAR point cloud, and then stored and maintained in the form of an occupancy grid map. The occupancy grid map is used to record whether each grid is occupied by a static obstacle, and belongs to the intermediate layer environment representation. After updating the grid map according to the static environment occupancy information, a Euclidean Signed Distance Field (ESDF) map is further generated using Euclidean distance transformation. In this embodiment, the static occupancy representation used for subsequent geometric safety cost calculation, local window truncation, and spatiotemporal alignment determination with the mode switching spatiotemporal reachability tube is the ESDF. The ESDF is used to characterize the distance from each spatial location to the nearest obstacle. A positive value indicates that there is a safe distance between the location and the nearest obstacle, a negative value indicates that the location is inside the obstacle, and the larger the absolute value, the farther away from the obstacle. It can also be safely expanded according to the robot's external dimensions. A local ESDF window is truncated with the robot's current position as the center, and used as one of the inputs for subsequent layers. The map resolution can be configured according to actual needs based on the platform size and computing resources.

[0054] In this embodiment, dynamic obstacle state information is first obtained by processing obstacle category and terrain semantic information, and then the motion trend of dynamic obstacles is predicted based on the dynamic obstacle state information. Motion trend prediction based on the position and velocity information of each obstacle includes: estimating the position, velocity, and acceleration of each dynamic obstacle based on historical detection results or target tracking results, and extrapolating its predicted position at multiple future times; mapping each predicted position to a grid map in chronological order, and applying a time decay weight to the prediction results to generate a dynamic obstacle spatiotemporal occupancy representation with a time index. , The value represents the time. Location The probability of being occupied by dynamic obstacles. Simultaneously, for each detected dynamic obstacle, parameters such as relative position, relative velocity, and detection confidence are extracted, and dynamic feature vectors are formed for each individual obstacle, used for downstream soft constraints or decision-making. The value range is [0,1], where 0 indicates no risk of dynamic obstacle occupancy and 1 indicates a 100% probability of dynamic obstacle occupancy. The time decay weight means that the further away the prediction time is, the greater the uncertainty, and the occupancy confidence decreases by a coefficient accordingly.

[0055] Specifically, in the prediction method based on historical detection results, the system caches the detection results of dynamic obstacles, such as position and velocity, from several past frames. Based on this, it estimates their motion trend and maps multiple predicted future positions onto corresponding grid cells in a grid map. For each predicted position, an occupancy value is added to that grid cell, with predictions further back in time receiving less weight. Ultimately, each grid cell accumulates an occupancy probability value at different times, forming... Spatiotemporal occupancy representation of dynamic obstacles The system updates dynamically with each control cycle as the perceived information is updated, ensuring that the system always plans obstacle avoidance based on the latest dynamic obstacle prediction information.

[0056] In addition to prediction methods based on historical detection results, implementations that interface with real sensors can also employ target tracking algorithms such as Kalman filtering and particle filtering to continuously track each dynamic obstacle, obtaining a more stable motion state (position, velocity, acceleration), and then performing multi-step predictions based on the tracking results, followed by generation according to the aforementioned mapping and attenuation methods. .

[0057] In this embodiment, the environmental raster is labeled with three types of terrain semantics based on the acquired terrain semantic information, thereby obtaining a terrain semantic raster. The three types of terrain semantics include: (1) Flat area (FLAT, label value 0): refers to a level ground where both wheels and legs can pass, without significant height differences or obstacles. The wheel-riding mode has the advantages of high efficiency and low energy consumption in flat areas.

[0058] (2) Heterogeneous Passable Area (STEP, label value 1): refers to areas with terrain features such as curbs, steps, small height differences, or narrow embankments. Wheeled chassis cannot pass directly due to ground clearance limitations, but legged mechanisms can cross by lifting their legs and taking steps.

[0059] (3) Dual-modal restricted area (TRENCH, label value 2): refers to a ditch, pit, ledge, or obstacle area whose width exceeds the maximum crossing capability of the legged mode. Neither wheeled nor legged modes can pass directly; the robot must detour through it.

[0060] In this embodiment, the methods for obtaining terrain semantic labels include, but are not limited to, lidar point cloud elevation statistics, RGB-D camera depth map segmentation, or pre-built semantic maps. LiDAR point cloud elevation statistics involve performing height histogram analysis on the point cloud within the raster neighborhood and calculating the maximum height difference. The RGB-D camera depth map is segmented into sections, and local slope and step height are calculated from the depth map. A pre-built semantic map is used to offline label terrain types in a known environment. In one specific implementation, the terrain semantic labeling rule is: when... Lower than the ground clearance of the wheeled chassis The time is marked as FLAT; when The time is marked as STEP. The maximum stride height for legged stances; when A continuous, deep indentation with a width exceeding the maximum stride distance of the legged platform is marked as TRENCH. Thresholds for wheeled chassis ground clearance, maximum stride height of the legged platform, and maximum stride distance of the legged platform are configured based on the specific geometric parameters of the wheeled platform.

[0061] Furthermore, terrain constraints can be determined based on terrain semantic information. The terrain semantic information is stored in the form of terrain semantic rasters, each raster labeled with FLAT, STEP, or TRENCH. Based on these terrain semantic labels, the system determines the following terrain constraints: (1) wheel mode is prohibited from traversing the STEP region; (2) bimodal mode is prohibited from traversing the TRENCH region; (3) terrain areas that do not meet the support conditions (such as areas with slopes exceeding the threshold or areas with abrupt elevation changes) are not allowed to be stood on or traversed. This terrain constraint is used in the subsequent step S4 for spatiotemporal alignment determination with the reachability tube, and as a hard constraint for model prediction control in step S6.

[0062] In this embodiment, see Figure 3 , Figure 4 and Figure 6The modal correlation drivability field dual channel, including wheel-walking cost channel and leg-walking cost channel, is constructed, and the target motion mode is determined based on the cost comparison results of the wheel-walking cost channel and the leg-walking cost channel, including the following process.

[0063] First, construct the Mode-Aware Cost Field. MACF Dual-channel. MACF No longer targeting environmental spatial location Establishing a fixed scalar cost independent of motion patterns Instead, it establishes robot movement patterns. Evaluation of the correlation between environmental conditions and spatial location ,in Indicates the robot's movement patterns, including at least the wheeled mode. With leg walking mode , Indicates the robot's motion state. Indicates time. By establishing independent cost evaluation channels for wheel-movement and leg-movement respectively, the same grid position... Different accessibility evaluations are obtained under different motion modes, thereby jointly modeling the wheel-leg dual-modal motion characteristics with environmental geometry, dynamic occupancy, and terrain semantics. Specifically, the system first receives static occupancy representation (ESDF) and dynamic obstacle spatiotemporal occupancy representation obtained based on environmental information. and topographic semantic raster; then, for each raster position and sports mode Evaluation based on association calculate MACF Single-point cost.

[0064] Unlike traditional two-dimensional cost maps that rely solely on spatial location, this embodiment... MACF Incorporating robot motion patterns, environmental states, and time factors into the mobility evaluation process is essentially a multidimensional cost mapping function defined on the joint space of spatial location and motion patterns.

[0065] in, Indicates the grid position, i.e., the location in the ambient space. Indicates the robot's movement pattern. Indicates the robot's motion state. Represents environmental time variables. MACF It represents the comprehensive passage cost under the corresponding spatial location, movement mode, and state conditions.

[0066] Based on this, for each grid position Robot movement modes Robot motion state and time , MACF A single-point cost is defined as the weighted sum of four costs:

[0067] in, , , , These are non-negative weighting coefficients. Represents the geometric safety cost of a static environment, used to describe the distance risk between a spatial location and a static obstacle; It represents the spatiotemporal occupancy cost of dynamic obstacles, used to describe the motion risk caused by dynamic targets within a future time frame; This represents the modal-terrain coupling cost, used to describe the degree of adaptation between different motion modes and the current terrain conditions; This represents the state-mode coupling cost, used to describe the matching relationship between the robot's current motion state and candidate motion modes. Non-negative weighting coefficients. , , , It can be configured according to task requirements or adaptively adjusted during the training process. The meaning and calculation method of the four costs are as follows.

[0068] Regarding the geometrical safety cost in static environments , ESDF distance based on ESDF (symmetric Euclidean distance field map) Reflects static collision risk. A larger positive ESDF value indicates greater distance from static obstacles, while a negative value indicates being inside an obstacle. Let the safe distance threshold be... Then the geometric cost is defined as:

[0069] in, For the robot's equivalent radius, For safety margin, This is the geometric cost gain coefficient. This piecewise function increases the geometric cost as the robot gets closer to a static obstacle, and the geometric cost is zero beyond a safe distance, without affecting the passage evaluation in areas far from obstacles. With sports mode Regardless of the mode, the same geometrical safety constraints are applied to wheel-walking and leg-walking paths because static obstacles pose a collision risk to both modes.

[0070] Regarding the spatiotemporal occupancy cost of dynamic obstacles , Based on the spatiotemporal occupancy representation of dynamic obstacles Determine, characterize the grid position The risk of being occupied by dynamic obstacles in the future time domain:

[0071] in, This is the dynamic cost gain coefficient. The normalized dynamic occupancy value is obtained by mapping the predicted location of dynamic obstacles to the grid using time decay weights. The same dynamic penalty is applied to both wheel-walking and leg-walking pathways because dynamic obstacles pose a collision risk to both modes. (Dynamic obstacle spatiotemporal occupancy cost) Independent of ESDF ,through Channel representation ensures that dynamic obstacle state information does not contaminate static information. ESDF Distance field, static geometric security and dynamic intent risk in MACF Layers are decoupled and then fused.

[0072] Regarding the cost of modal-terrain coupling , The value varies with the motion pattern With terrain semantic tags Simultaneous changes. In this embodiment, The rules are determined according to Table 1, and this rules are stored in... terrain layer: Table 1 Modal-terrain coupling cost rules

[0073] The physical meaning of the above rule table is: in flat areas, the cost of wheel movement is zero, and the cost of leg movement is slightly higher, making... MACF Dual-channel operation in a rotating pattern in flat areas incurs costs. Less than the cost of leg walking mode This guides decision-makers to choose the wheel-based mode of transportation; in heterogeneous traversable areas, the wheel-based mode is extremely costly, while the leg-based mode is relatively inexpensive, making... MACF Dual-channel presentation This guides the decision-making team to switch to leg-walking mode to overcome obstacles; in restricted areas, both modes are extremely costly. MACF Both channels signaled high costs, guiding decision-makers to implement lateral detours.

[0074] Regarding the cost of state-mode coupling , Based on the robot's current state With sports mode The matching relationship is determined to express the pattern preference at the strategy level. The calculation methods include, but are not limited to: (1) High-speed wheel priority: when the robot's linear speed is... When the terrain is high and the area in front is flat, an additional penalty is applied to switching to leg-walking mode; (2) Near-obstacle leg-walking priority: when the robot approaches a heterogeneous passable area and ESDF When the distance is small, an additional penalty is imposed on maintaining the wheeling mode; (3) Body height matching: when the leg walking mode coefficient At higher levels, the leg walking pattern is in STEP The regional cost is further reduced. (Leg walking modal coefficient) A higher elevation indicates that the robot is partially or completely in a legged state. A higher value indicates a higher degree of completion in the legged posture. Preference (3) is used to encourage robots already in a legged posture to continue using legged mode to overcome obstacles. In a simplified implementation, It can be incorporated into the reward function of the policy network during the training phase. MACF In the construction phase, the value is zero or a small constant; in another implementation... It can be explicitly modeled as a function of velocity, modal coefficients, and terrain semantic labels.

[0075] Determine the position of each grid cell Robot movement modes Robot motion state and time of MACF After calculating the single-point cost, each is processed in a round-robin mode. With leg walking mode Based on the robot's current position ( ) as the center The cost is calculated grid-by-grid within a local window. The specific algorithm flow is shown in Table 2: Table 2:

[0076] Among them, step " ← / / Normalization is a numerical normalization operation, which means dividing each arithmetic value in matrix S by a maximum reference value. Then restrict the result to the interval [0, 1]. This indicates that the original value of each grid cell is scaled proportionally. It is a preset constant (5.0 in this embodiment) representing the maximum allowable cost reference upper limit. This is a truncation function. If the scaled value is less than 0, it is set to 0; if it is greater than 1, it is set to 1; if it is between 0 and 1, it remains unchanged.

[0077] Finally, respectively and By executing the above process, a slice of the round-robin channel cost is obtained. With leg passage cost slice Stacked as a two-channel tensor:

[0078] In this context, the first dimension (2) represents two channels, with index 0 corresponding to the cost slice of the round-robin mode and index 1 corresponding to the cost slice of the leg-robin mode.

[0079] In step S1, based on the two-channel tensor The wheel-movement cost channel and the leg-movement cost channel in the system obtain the cost value of the current position in wheel-movement mode and leg-movement mode, respectively. If the cost of wheel-movement mode is less than the cost of leg-movement mode, wheel-movement is set as the target movement mode; otherwise, leg-movement is set as the target movement mode, which serves as the initial reference for subsequent joint planning and switching feasibility determination. MACF The dual-channel architecture executes in each control cycle, following ESDF. The output dual-channel tensor is refreshed in real time according to the robot's state. It was also refreshed accordingly.

[0080] In this embodiment, the number of grid rows and columns of the local square window centered on the robot's current position are... The window size and normalization coefficient can be configured according to the platform size, map resolution, and policy network input dimension. As an important component of policy network observation, it enables policies to directly perceive the differences in access costs between the current location and its neighborhood in the two modes, without having to implicitly learn modality-terrain rules from raw sensor data.

[0081] To explain more clearly MACF The effects of dual-channel technology are illustrated below, showing the same grid position in three typical terrain scenarios. A comparative example of the cost of wheel-walking and leg-walking pathways. Let this location... ESDF distance (Safety), (No dynamic obstacles) .

[0082] Table 3: Typical Scenarios MACF Dual-channel cost example

[0083] As shown in Table 3, the MACF dual-channel exhibits the largest modal cost difference in the curb / step scenario. This is precisely the key criterion for wheeled robots to switch modes; in flat areas Negative but small absolute values ​​allow the strategy to trade off between efficiency and conservatism; in the gully region, both pathways are costly and This explicitly indicates that there is no mode-switching solution at this location, and a lateral detour is necessary. This structured information is a traditional scalar cost field. Unavailable. See also Figure 6 Traditional cost mapping methods are ineffective at any location. Give a single scalar Its processing chain involves generating a single cost field after environmental perception. Used for planning / decision making. Under this link, curb locations can only be labeled as either high-cost or low-cost, and cannot simultaneously express the dual meaning of "not passable by wheeled vehicles" and "passable by foot vehicles"; the decision-making module cannot... Directly reading modality switching criteria in the middle can only rely on implicit rules or additional heuristics.

[0084] In this embodiment, the processing link of the MACF dual-channel is as follows: First, the perception layer acquires and processes multi-source sensor information. Then, the representation layer constructs cost channels for both wheel-walking and leg-walking modes, forming a dual-channel accessibility evaluation. The MACF single-point cost is obtained by weighted summation of geometric safety cost, dynamic obstacle spatiotemporal occupancy cost, modal-terrain coupling cost, and state-mode coupling cost. By constructing the MACF dual-channel, independent cost evaluation channels are established for wheel-walking and leg-walking modes, allowing the same spatial location to have different costs in the two modes. Based on this, the system selects the motion mode according to the cost difference between the two channels, and finally outputs the mode selection result and corresponding cost information to the switching accessibility layer. Under this link, the curb / step location presents a high cost in the wheel-walking channel and a low cost in the leg-walking channel, with a significant difference in modal costs. It can serve as an explicit modality switching signal, solving the problem that traditional single cost maps cannot express modality differentiation rules such as wheeled vehicles being impassable while legged vehicles are traversable. Therefore, the MACF dual-channel is not a simple extension of existing cost maps, but a traversability evaluation framework re-established in the two-dimensional domain of environmental space and motion patterns.

[0085] In other implementations, the motion pattern can be expanded to more than three, such as wheeling, stepping, climbing, and jumping, and a corresponding multi-channel MACF can be established. The modal-terrain coupling cost rule table can be replaced by offline calibration, online adaptive methods, or learning networks instead of fixed lookup tables. MACF can be combined with 3D voxel meshes or layered elevation maps to support more complex 3D terrain representations. Weights ~ It can be manually configured according to task requirements or used as a learnable parameter in end-to-end training. Those skilled in the art can make equivalent substitutions or modifications to the terrain semantic segmentation rules, cost function form, window size, and normalization method without departing from the concept of this invention, and all such substitutions or modifications should be considered to fall within the protection scope of this invention.

[0086] S2: When the target motion mode is different from the current motion mode, the robot state during the mode switching phase change is predicted time by time according to the preset mode switching dynamics model and prediction step size, so as to obtain a set of robot states corresponding to multiple prediction times. The mode switching phase is used to characterize the progress of switching from the current motion mode to the target motion mode.

[0087] like Figure 7 and Figure 8 As shown, step S2 in this embodiment is the mode switching spatiotemporal reachability prediction link set after the MACF dual-channel construction and before the candidate trajectory evaluation and safety control. It aims to predict the state range that the robot may reach at each moment and the corresponding spatial occupancy range for the continuous process of transitioning from the current motion mode to the target motion mode, and determine whether the mode switching can be executed and when it can be executed earliest.

[0088] In this embodiment, the mode switching state is defined. , in, , Indicates the robot's position. Indicates the heading angle. Indicates linear velocity. Indicates angular velocity. Indicates the height of the aircraft. Indicates the aircraft's posture. and These represent the state of the wheel-leg mechanism and its rate of change, respectively. Indicates the mode switching phase. Indicates the current motion mode. This indicates that the target motion mode has been entered. This indicates a transition process. The wheel-to-leg and leg-to-wheel transitions use corresponding state transition parameters to reflect the differences in duration, speed limits, and configuration changes between the unfolding and retracting processes.

[0089] In discrete prediction period The mode switching dynamics model can be expressed as follows:

[0090] in, For the first The set of robot states at each predicted time point. For the current mode Switch to target mode State transition mapping, This is the set of control inputs for speed, angular velocity, and wheel-leg mechanism. For the perturbation set, This represents the summation of sets. The perturbation set is used at least for envelope model error, sensing error, controller tracking error, and communication or execution delay.

[0091] S3: Determine the space occupancy set corresponding to each prediction time based on the robot state set at each prediction time. The space occupancy set is the union of the body occupancy set and the wheel and leg mechanism occupancy set at the corresponding prediction time. Associate each prediction time and its corresponding space occupancy set in chronological order to form a mode switching spatiotemporal reachability tube.

[0092] In this embodiment, for each prediction time, based on the robot's position, posture, height, and wheel-leg deployment degree in the robot's state set, the robot's body geometric model and wheel-leg mechanism geometric model are transformed to a unified environmental coordinate system, and the robot's occupied set is obtained respectively. Together with the wheel-leg mechanism occupy the set and take This serves as the spatial occupancy set at the predicted time. In computationally limited implementations, circles, ellipses, oriented bounding boxes, convex polygons, or unions of multiple basic geometries can be used. A conservative outer envelope is used; in a three-dimensional implementation, a convex polyhedron, a voxel set, or a three-dimensional oriented bounding box can be used.

[0093] Predicting the time With space occupancy set Associating to obtain spacetime slices And combine them in chronological order into the current pattern Switch to target mode Mode switching spatiotemporal reachability tube Unlike methods that use a fixed robot radius to expand the entire trajectory, The cross-section of the phase of mode switching Body posture and wheel mechanism status The changes can characterize the actual space requirements of the increased lateral or longitudinal envelope when the wheel legs are deployed.

[0094] S4: Perform spatiotemporal alignment determination between the mode switching spatiotemporal reachability tube, the static occupancy representation, the dynamic obstacle spatiotemporal occupancy representation, and the terrain constraints determined based on the terrain semantic grid to obtain the mode switching safety margin and the mode switching feasible time interval.

[0095] In this embodiment, for Each spatiotemporal slice in the data will Separately with static occupied areas The predicted area occupied by dynamic obstacles at the corresponding time. and terrain-restricted areas Perform an intersection check. If any... If the region intersects with the aforementioned region, the corresponding candidate switch is deemed infeasible; if they do not intersect, then the calculation is performed. To the aforementioned static occupied area The predicted area occupied by dynamic obstacles at the corresponding time. and terrain-restricted areas The minimum interval is used, and the minimum value among all prediction times is taken as the mode switching safety margin. Mode switching safety margin The closest distance between the robot and the obstacle during the representation mode switching process.

[0096] Among them, static occupied area Extracted directly from the static occupancy representation, any area whose raster status is "occupied" is considered an occupied area. This represents the space occupied by stationary obstacles in the environment; the predicted area occupied by dynamic obstacles at the corresponding moment. Representation of spatiotemporal occupancy from dynamic obstacles Extract from, that is, extract the future. All grid cells whose predicted occupancy probability is higher than a certain threshold at any given time constitute the predicted occupancy area at that time, representing the dynamic obstacle at some future time. Potentially occupied space; terrain-restricted areas Extracted from the terrain semantic raster, the raster with the semantic label "bimodal restricted area (TRENCH)" is marked as a restricted area, which represents an area where robots are prohibited from entering due to the terrain itself being unsuitable for passage, such as deep ditches and steep slopes.

[0097] In addition to spatial collision conditions, the system also checks state variables such as linear velocity, angular velocity, body tilt angle, body height, joint position, joint velocity, driving torque, ground support, and anti-rollover conditions during mode switching to determine whether the state constraints are met throughout the entire process. These parameters are then checked at the prediction terminal. Feasible state domain for entering the target motion mode Only when mode switching safety margin Not less than the preset safety threshold, and all process state constraints are met. belong Only when the time period is specified will the corresponding time period be marked as a feasible time period for mode switching.

[0098] Specifically, determining whether the state constraints are satisfied throughout the entire process involves: at each predicted moment, comparing the obtained robot state set information, such as velocity, attitude, joint position and velocity, driving torque, and ground contact force, with preset velocity limits, attitude stability boundaries, joint mechanical limits, motor peak torque, ground friction cone, and ZMP stability criteria; if all possible values ​​of the robot state set at the current moment are within the aforementioned constraint boundaries, then the state constraints are satisfied at that moment. This applies to the spatiotemporal reachability of modal switching. The above judgment is executed sequentially at all predicted time points. Only when all time points are satisfied is the switching candidate marked as having satisfied the full-process state constraints; otherwise, it is discarded. By incorporating the physical execution capability of mode switching into the feasibility analysis in this way, switching failures due to actuator saturation, mechanism interference, or dynamic instability are avoided.

[0099] It should be noted that the aforementioned modal switching spatiotemporal reachability tube It depicts the space occupied by an initiated switching process at subsequent moments; it is not itself the switching initiation window. The feasible time interval for mode switching. This represents the moment corresponding to the current control cycle. Next, which candidate launch times? This ensures the feasibility of the entire subsequent switching process. Therefore, the start time is used as a parameter for the spatiotemporal reachability of mode switching, forming a family of reachability tubes. , in, , , The duration of the mode switch. To predict the step size; In order to be in After the start-up switch The set of space occupancy for the robot body and wheel-leg mechanisms corresponding to each step. First, select the current exercise mode. Change state from spread to Obtain the robot state set before the switch. ; and then As initial values, dynamics of mode switching The propagation continues until the switch is complete, generating a corresponding set of space occupants. Thus, different... It corresponds to different driving distances before switching, speed and attitude when entering the switching phase, and relative spatiotemporal position with dynamic obstacles and terrain boundaries during the switching period.

[0100] For each candidate start time All of these were assessed based on spatiotemporal collision, terrain support, and terminal feasibility. The feasibility function for the start-up time is obtained by judging all spatiotemporal slices. .

[0101] If and only if the following conditions are met, otherwise 0: (1) No collision condition for slices.

[0102] right All have ,in This is a statically occupied area. The predicted area occupied by dynamic obstacles at the corresponding time. For terrain areas that do not meet the support or prohibition conditions, entry is prohibited.

[0103] (2) Safety margin conditions. (Note:) for The minimum spacing between each slice and the above-mentioned occupied area is required. , This is the preset mode switching safety threshold.

[0104] (3) State constraints. During the switching period, the speed, angular velocity, body tilt angle, body height, wheel-leg mechanism position and speed, ground support and anti-rollover constraints are all satisfied.

[0105] (4) Terminal feasibility conditions. Set of terminal states to be switched. Fall into the target motion pattern Feasible state domain The feasible state domain of the target motion mode refers to the set of safe ranges that the robot's state variables must satisfy when the robot is moving normally in the target motion mode.

[0106] (5) Switching to completion timing conditions. Let the time when the target terrain boundary or modal forbidden boundary is reached along the current candidate trajectory be . Execution delay is ,Require .

[0107] (6) Switch the distance margin condition. Let... The remaining distance along the trajectory to the boundary at any given time is The distance required to complete the switch is It is determined by the sum of the travel distance during the delay phase, the travel distance during the deceleration phase, and the travel distance during the switching phase; requirements .

[0108] Conditions (1) to (4) are given directly by the reachable tube slice and its robot state set; conditions (5) and (6) align the time length and space occupation of the reachable tube with the arrival time and remaining distance of the target terrain to exclude the case where the target mode itself is passable, but the start is too late and the switch cannot be completed before entering the terrain.

[0109] In this embodiment, the feasible time interval is switched by the feasibility function extraction mode.

[0110] In the current prediction time domain Inside, according to step size Discrete sampling of candidate start times yields

[0111] Calculate one by one , will make The set of discrete times is denoted as .like If the value is empty, it is determined that there is no feasible switchover start time in the current cycle, and an empty window is output. And switch to hold current mode or safety braking. If If the data is not empty, then extract the connected subset in chronological order. Connectivity means that the interval between two adjacent feasible start times does not exceed [a certain value]. When multiple connected subsets exist, the earliest connected subset is selected first; in an alternative implementation, the subset with the smallest safety margin can also be selected. The largest. The left endpoint of the selected connected subset is taken as the earliest feasible start time. The right endpoint serves as the latest safe start time. Thus, the feasible time interval for mode switching is obtained: Its physical meaning is: earlier than At startup, reachable pipes will conflict with constraints of the current corridor, dynamic body, or process that has not yet completed deceleration; later than At startup, the remaining distance or time is insufficient to complete the configuration switch before reaching the target terrain. Therefore, It is not the occupied period on the reachable tube timeline, but the set of start-up times that make the entire reachable tube feasible.

[0112] In this embodiment, the latest start time can be determined by reversing a single currently reachable pipe. In implementations with limited computing resources, it is not necessary to process all... Rebuild the reachable pipe family, and immediately activate the corresponding reachable pipe at the current moment. Inversely, the window endpoints are resolved. Specifically, this includes: based on... The initial distance between each slice and the occupied area is less than The moment and the time of arrival of the target terrain. Calculate using the following formula , Then, reduce the current mode speed to the allowed switching speed. The earliest time to complete, i.e.

[0113] in, The current linear velocity, To allow for deceleration. If ,but ;otherwise This simplified implementation uses a single reachable tube to provide the latest start constraint and velocity convergence to provide the earliest start constraint. It can serve as a computationally efficient implementation example for extracting feasible time intervals from modes using a feasibility function. The complete implementation still uses a family of reachable tubes parameterized at the start time. As the standard.

[0114] The above The system is updated on a rolling basis each control cycle based on the latest status, dynamic occupancy predictions, and terrain boundaries. The updated... , , and The candidate trajectory filtering module at the decision-making level and the switching safety supervisor at the execution level are output together: not falling into... Switching candidates within the same layer are directly eliminated; if the switching time requested by the upper layer does not belong to the same layer... If the condition is met, the following actions will be taken: wait, decelerate before switching, refuse to switch, or brake safely.

[0115] Regarding the switching distance margin and the earliest feasible start window, let the distance from the robot's current position to the target terrain boundary along the current candidate trajectory be... According to the current speed Permissible deceleration Mode switching duration and execution delay Determine the distance required to complete the mode switch. In one implementation, The distance is determined by the sum of the travel distance during the delay phase, the travel distance during the deceleration phase, and the travel distance during the switching phase, and the switching distance margin is calculated. .

[0116] Change the candidate switching start time within the rolling prediction time domain , respectively construct Will satisfy , The set of consecutive startup times based on terminal feasibility conditions is determined as the feasible switching time interval. Its left endpoint is the earliest feasible switching time, and its right endpoint is the latest safe switching time. Therefore, the system no longer only outputs the mode label "should switch to leg mode", but outputs a switching start window with timing constraints.

[0117] S5: Generate candidate trajectories for maintaining the current motion mode and candidate trajectories for switching to the target motion mode within the prediction time domain. Filter the candidate trajectories for mode switching based on the feasible time interval for mode switching. Evaluate the segmented cost of the filtered candidate trajectories for mode switching. Determine the target trajectory, mode switching initiation time, and reference control commands based on the evaluation results and the candidate trajectories for maintaining the current motion mode. The reference control commands include recommended linear velocity, recommended angular velocity, and target motion mode.

[0118] In this embodiment, a joint evaluation is performed on candidate trajectories for maintaining the current motion mode and candidate trajectories for switching to the target motion mode. The candidate trajectory for maintaining the current motion mode serves as a benchmark for evaluating the candidate trajectory for switching to the target motion mode. By comparing the total cost of each candidate trajectory for switching modes with the total cost of the candidate trajectory for maintaining the current motion mode, the system selects the trajectory with the lowest total cost as the target trajectory. This allows the system to determine whether the switching behavior brings substantial traffic benefits, thereby avoiding unnecessary mode switching. For example, in flat areas, while switching to legged mode allows passage, its energy consumption and traffic efficiency are lower than maintaining wheeled mode. The lower cost of the candidate trajectory for maintaining the current motion mode will cause the candidate trajectory for switching to the target motion mode to be eliminated due to its relative disadvantage.

[0119] The decision-making layer can generate at least four candidate modes: maintaining wheel-based movement, maintaining leg-based movement, switching from wheel-based to leg-based movement, and switching from leg-based to wheel-based movement. For example, when the current mode is leg-based movement, two candidate modes are generated: maintaining leg-based movement and switching from leg-based to wheel-based movement. For mode switching candidates, the switching trajectory is divided into pre-switching trajectory segment, switching trajectory segment, and post-switching trajectory segment based on the switching initiation and completion times: the pre-switching trajectory segment uses the current mode's MACF channel; the post-switching trajectory segment uses the target mode. MACF Channel; Track segment during switching uses by The transition cost is determined by the robot's state set and its spatiotemporal occupancy.

[0120] The total cost of the candidate trajectory also includes target distance, heading error, safety clearance, handover duration, handover energy consumption, reachability risk, and frequent handover suppression terms. (Not satisfied) Mode switching candidates that meet the constraints are directly eliminated; for candidates that satisfy the constraints, the one with the smaller total cost is selected, and the recommended linear velocity, recommended angular velocity, current mode, and target mode are output. MACFCost difference, expected switching duration, mode switching safety margin, switching distance margin, earliest feasible switching time, and monitoring instructions.

[0121] When the upper-level decision-making module switches the request mode, the security supervisor checks whether the request time belongs to... If it belongs to If the current time is earlier than the target trajectory, then the switch will be performed; if the current time is earlier than the target trajectory, then the switch will be performed. If the current speed causes..., then maintain the current mode and track the waiting trajectory; if the current speed causes... However, if a feasible solution still exists after deceleration, then the constrained deceleration is performed first, followed by the updated solution. The switching is initiated within the prediction time domain; if no feasible switching window exists within the prediction time domain, the switching request is rejected, and a control scheme that meets safety constraints is selected between maintaining the current mode detour and a safe braking trajectory.

[0122] The aforementioned reachability tube is continuously reconstructed in each control cycle based on the latest robot state, dynamic obstacle prediction, and terrain information. Its output is simultaneously input into the candidate trajectory screening module and the model prediction controller of the decision layer. Therefore, the same prediction object runs through the three stages of "mode switching timing determination - trajectory selection - control safety correction", avoiding inconsistencies where the planning layer considers the switching feasible, but the execution layer cannot complete the switching due to insufficient space.

[0123] In this embodiment, a shape-aware prior guidance mechanism is provided. See also Figure 7 , Figure 9 and Figure 10 In this embodiment MACF and ESDF Under constraints, four candidate trajectories are jointly generated: maintaining wheel movement, maintaining leg movement, wheel-to-leg movement, and leg-to-wheel movement. For mode switching candidates, the previously obtained feasible switching time intervals are used for filtering, and the trajectories are evaluated according to the segmented costs before, during, and after the switch. The morphological perception prior is used to provide the intelligent decision-making module with soft priors that have executable temporal constraints, rather than replacing the intelligent decision-making module in directly outputting the final control action.

[0124]

[0125] In one implementation, the auxiliary decision vector is represented as:

[0126] in, and These are the recommended linear velocity and the recommended angular velocity, respectively. and These are the current mode and the target mode, respectively. Accumulation of candidate trajectories for wheeled and legged walking MACF Cost difference For the expected handover duration, For mode switching safety margin, To switch distance margin, The earliest feasible switchover time, To execute, decelerate, delay, maintain mode, or safety braking monitoring commands. When a dual-modal no-entry zone blocks the target direction, valid target points are mapped to lateral channel guide points to generate maintain mode detour candidates.

[0127] like Figure 13 As shown, a multimodal decision-making and multi-expert reinforcement learning strategy is adopted. The policy network includes a gating network and multiple expert subnetworks, which correspond to behavioral preferences such as wheeled cruising, leg-based obstacle avoidance, dynamic obstacle avoidance, and no-entry detour, respectively. The continuous control actions are output through sparse weighted fusion.

[0128] First, multimodal observations are constructed, including local... ESDF , MACF Multimodal observations were performed using dual channels, dynamic obstacle features, auxiliary decision vector H, robot body state, relative target, terrain semantic vector, mode switching phase, mode switching safety margin, switching distance margin, and historical environment sequence. Among these, the mode switching phase was used to distinguish between stable wheeled movement, transitioning, and stable legged movement states; the historical environment sequence was used to characterize the changes in dynamic occupancy and accessibility over time.

[0129] Then, the aforementioned multimodal observations are input into a multimodal feature fusion network to obtain a unified latent feature representation. In a preferred embodiment, a method based on... Transformer A multimodal feature fusion network of structured features is used to process information from each modality. Token Encoding and self-attention fusion can be used; in other implementations, convolutional networks, gated fusion networks or other equivalent fusion structures may also be used.

[0130] Unlike traditional single-policy networks, this embodiment employs a motion-mode-oriented decoupled multi-expert reinforcement learning strategy, enabling different experts to learn control laws under different environmental conditions. Specifically, the high-speed cruise expert focuses on efficiency optimization in flat areas, the obstacle-crossing expert focuses on the ability to traverse complex terrain, the dynamic obstacle avoidance expert focuses on dynamic obstacle avoidance capabilities, and the detour expert focuses on path adjustment in impassable areas.

[0131] The fused features are obtained through a multimodal feature fusion network. The input is a multi-expert reinforcement learning policy network. This policy network comprises multiple expert subnetworks. Learn dynamic strategies for different motion patterns; gating network Output the weights of each expert Reference control commands Generate and output continuous control actions: , in, The leg-walking mode switching coefficient is used. In one implementation, the expert subnetwork can be configured according to task requirements as at least one of the following: high-speed wheeled cruising expert, low-speed leg-walking obstacle-crossing expert, dynamic obstacle avoidance expert, restricted area detour expert, or dynamic environment adaptation expert; and a sparse gating mechanism is used to select some experts to participate in decision-making. The policy network can adopt a soft actor-commentator (... Soft Actor-Critic , SAC The algorithm performs policy optimization and is compared with the double-delay deep deterministic policy gradient (DDE). Twin Delayed Deep Deterministic Policy Gradient , TD3 Training can be performed using other continuous action reinforcement learning algorithms. Figure 11 As shown.

[0132] The reward function includes terms related to target arrival, progress maintenance, collision avoidance, safety margin, control smoothness, pattern-terrain matching, and no-entry zone detour. The training process can employ a learning strategy that gradually transitions from simple to complex scenarios. For example... Figure 14 As shown.

[0133] S6: Using the mode switching safety margin not lower than the preset safety threshold and the non-intersection of the regions constrained by the spatiotemporal reachability tube of mode switching with the static occupancy representation, the spatiotemporal occupancy representation of dynamic obstacles and the terrain semantic grid as hard constraints, the reference control command is modified for safety constraints, and the wheeled robot is controlled to maintain the current motion mode or switch to the target motion mode or perform safety braking at the start of mode switching.

[0134] In this embodiment, step S6 is intended to perform model predictive control and safe execution.

[0135] See Figure 9 The control commands output by the policy network are used as reference inputs to the Model Predictive Controller (MPC). Within a finite prediction time domain, the MPC searches for velocity commands that satisfy the safety margin of the symbolic Euclidean distance field map and the current motion mode—the feasible terrain domain constraint—to achieve safe tracking and constraint correction of the reference control commands. The commands corrected by the predictive control are then input into the wheel-leg hybrid dynamics model, where the lower-level controller handles the switching between wheel-walking and leg-walking modes and velocity tracking.

[0136] The model predictive controller reads the reachability tube cross-section at each prediction step and uses the non-intersection of this cross-section with static occupancy, dynamic predicted occupancy, and terrain constraints, as well as the mode switching safety margin not being lower than a preset safety threshold, as hard constraints. When the mode switching corresponding to the reference command is infeasible, the controller prioritizes solving for a deceleration trajectory that satisfies the deceleration, attitude stability, and minimum safe distance constraints; if no feasible deceleration trajectory exists, a safe braking command is output. Thus, the reachability tube is not merely an observation feature of the policy network, but directly participates in the control feasible region limitation.

[0137] After the decision-making layer selects the target trajectory based on the total cost of candidate trajectories, its output reference control command is not executed directly, but is sent to the model predictive controller (MPC). Within the finite prediction time domain, the MPC uses hard constraints—that the cross-section of the mode-switching spatiotemporal reachability tube at each prediction time does not intersect with the static occupancy area, the dynamic obstacle prediction occupancy area, and terrain constraints, and that the mode-switching safety margin is not lower than a preset safety threshold—to perform safety corrections on the reference command. Only the command corrected by the MPC is input into the wheel-leg hybrid dynamics model for execution by the lower-level controller. Simultaneously, the same mode-switching spatiotemporal reachability tube runs through the three stages of mode-switching timing determination, trajectory selection, and control safety correction, avoiding inconsistencies where the planning layer deems the switch feasible while the execution layer cannot complete the switch due to insufficient space.

[0138] To verify the effectiveness of the method in this embodiment, tests were conducted in a wheel-leg navigation simulation environment. Four typical scenarios were set up: static obstacles, dynamic obstacles, ditch detours, and mixed unstructured obstacles. Each scenario underwent multiple independent evaluations. See [link to relevant documentation]. Figure 12 Table 4 shows the comparison results between the method of the present invention and the single-modal local programming method; Table 5 shows the detailed indicators of the method of the present invention for different scenarios; Figure 18 Give removal MACF , Planning Hint The comparative verification results of multi-expert structures or predictive control modules are presented. In this embodiment, the single-modal local programming method refers to a method based on a single kinematic model without introducing... MACF Local planning implementation for dual-channel navigation and mode switching. Evaluation metrics include navigation success rate, collision rate, arrival time, and mode switching characteristics.

[0139] To ensure the safety of mode switching timing, evaluation metrics may also include mode switching completion rate, late switching rate, infeasible switching request rate, minimum reachability safety margin, switching distance margin, collision rate during switching, and number of safety monitor interventions. These metrics are used to evaluate the correctness of target mode selection and the feasibility of the mode switching process, avoiding the evaluation of switching quality solely based on the mode label after reaching the target.

[0140] Table 4: Comparison of success rates (%) between the method in this embodiment and the single-modal local programming method

[0141] Table 5: Detailed Indicators for Various Scenarios in This Embodiment

[0142] Experimental results show that the obstacle avoidance method in this embodiment has good navigation performance in scenarios containing heterogeneous terrain and restricted areas. It can form a lateral detour trajectory in ravine detour scenarios and complete the coordinated switching between wheel-walking and leg-walking modes.

[0143] Figure 4 Show ESDF — MACF —Trajectory processing flow; Figure 5 It displays multimodal sensing information; Figure 10 This is a schematic diagram of a typical simulation experimental environment; Figure 14 This is a comprehensive performance evaluation chart; Figure 15 A diagram showing the comparison of navigation performance in different scenarios; Figure 17 The trajectory is compared with that of other methods.

[0144] To further verify the effectiveness of each key technical module of this invention, ablation experiments were conducted on the modality-related drivability field, morphology-aware auxiliary decision-making mechanism, multi-expert reinforcement learning strategy, and model predictive control safety execution module. During the experiments, the environmental configuration, robot initial state, target position, and training parameters were kept consistent; only the corresponding functional modules were removed, and the results were compared with the complete method.

[0145] Specifically, the following four ablation models are set up: (1) Remove MACF Module: The dual-channel drivability modeling corresponding to the wheel-walking mode and the leg-walking mode is cancelled, and the environment representation is degenerated into a single cost map that is independent of the motion mode, which is used to verify the impact of modality-dependent environment representation on the adaptability to heterogeneous terrain; (2) Remove Planning Hint Module: No longer uses local trajectory sampling results to generate morphological perception auxiliary information, but relies solely on the reinforcement learning policy network to autonomously complete the environment exploration, which is used to verify the impact of the prior guidance mechanism on the policy convergence speed and local decision stability; (3) Remove the multi-expert reinforcement learning module: replace the multi-expert structure with a single-policy network so that different motion modes can share a unified control strategy, which is used to verify the effect of motion mode decoupling learning on improving the adaptability to complex environments; (4) Remove the model prediction control module: directly execute the output action of the reinforcement learning strategy without safety constraint correction, which is used to verify the role of the prediction control module in collision avoidance, action smoothing and safe execution.

[0146] (5) Remove the mode switching reachability module: retain MACF Dual-channel and target mode selection, but the mode switching is processed as instantaneous or with a fixed delay, without configuration-related space occupancy prediction, switching window determination and switching distance margin calculation, to verify the impact of the reachable tube on late switching, collisions during switching and safety monitoring intervention.

[0147] Experimental results show that the complete method outperforms various ablation models in terms of navigation success rate, collision avoidance capability, and mode switching stability, such as... Figure 18 As shown, remove MACF Dual-channel Planning Hint After removing multi-expert architectures or predictive control modules, system performance declined to varying degrees. Specifically, removing... MACF After the module was removed, the robot exhibited incorrect mode selection in heterogeneous regions with significant differences in wheel-leg motion adaptability, leading to a deterioration in local path quality; Planning Hint Subsequently, the policy network needs to autonomously explore more ineffective actions, which manifests as a decrease in convergence speed and a decrease in local detour efficiency; after removing the multi-expert policy, a single policy is difficult to simultaneously meet the control requirements of high-speed wheel travel and complex terrain leg travel; after removing the model prediction control module, although it can complete basic navigation tasks, the safety margin is reduced in narrow passages and dynamic obstacle environments.

[0148] MACF Dual-Channel Tensor As the core channel for policy network observation, it is input into the multimodal fusion network alongside ESDF, dynamic features, and PlanningHint, eliminating the need for the policy to implicitly learn modal-terrain rules from raw sensor data. Simultaneously, the PlanningHint module generates an auxiliary decision vector H, which includes recommended linear velocity, recommended angular velocity, current mode, target mode, expected switching duration, mode switching safety margin, switching distance margin, earliest feasible switching time, and supervisory instructions, directly informing the policy network of the currently feasible switching window and velocity suggestions in numerical form. Historical MACF sequences further enable the policy to perceive recent changes in drivability. This prior information effectively reduces the policy's ineffective exploration space, improving training efficiency and generalization ability.

[0149] Therefore, it can be seen that the present invention proposes... MACF The dual-channel environment modeling, morphological perception prior guidance, multi-expert reinforcement learning decision-making, and predictive control safety execution modules are not simply superimposed, but rather work synergistically at the three levels of environment representation, intelligent decision-making, and safety execution to jointly constitute an autonomous obstacle avoidance closed-loop system for wheeled and legged robots suitable for complex unstructured environments.

[0150] This invention can be applied to scenarios such as complex site inspection, emergency transportation, factory logistics, and autonomous operation in unstructured environments. Through MACF dual-channel modeling and hierarchical decision-control architecture, autonomous obstacle avoidance and motion mode switching of wheeled robots can be achieved while ensuring operational safety, demonstrating good industrial applicability.

[0151] The present invention provides an obstacle avoidance method for wheeled and legged robots based on a modal switching spatiotemporal reachability tube. This method establishes an environmental accessibility representation related to the motion mode through a dual-channel MACF (Motion-Modal-Focused Functional Array), enabling the system to provide different evaluation results for the same spatial location based on wheeled and legged modes. This solves the fundamental problem that traditional single-cost maps cannot express modal differentiation rules, providing accurate cost basis for subsequent mode selection and switching decisions. Based on the aforementioned environmental representation, the modal switching spatiotemporal reachability tube continuously predicts speed changes, attitude adjustments, and wheel-leg configuration changes during mode switching. It also determines the feasible time interval for mode switching, including the earliest feasible switching time and the latest safe switching time, through spatiotemporal alignment determination. Combined with switching distance margin determination, this fundamentally solves the problem of timing the switching, effectively avoiding situations where switching too early leads to efficiency degradation or switching too late prevents completion of the switch before the target terrain. The outputs of the aforementioned dual-channel MACF and spatiotemporal reachability tubes simultaneously serve as prior environmental information inputs to the reinforcement learning policy network. The dual-channel MACF tensor allows the policy to directly perceive the difference in passage costs between the two modes. The auxiliary decision vector injects structured information such as feasible time intervals, safety margins, and switching distance margins into the policy in numerical form, significantly reducing the policy's ineffective exploration space and improving training efficiency and generalization ability. Based on this, the model predictive controller uses hard constraints such as the cross-section of the spatiotemporal reachability tube during mode switching not intersecting with obstacles and the mode switching safety margin not falling below a preset safety threshold to safely correct reference control commands. This ensures that mode switching requests and control commands from the decision layer are safely implemented at the execution layer, guaranteeing consistency between upper-layer decisions and lower-layer execution. When switching is infeasible, the system automatically triggers degradation strategies such as deceleration and waiting, delayed switching, maintaining the current mode, or safe braking, further improving the system's safety and reliability in extreme scenarios.

[0152] The present invention has been described in detail with reference to specific embodiments. In some cases, the actions or steps described in the embodiments of the present invention may be performed in a different order than that shown in the embodiments and the desired results may still be achieved. In addition, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0153] Based on the same concept, this invention also provides an obstacle avoidance system for wheeled and legged robots based on modal switching spatiotemporal reachability tubes. This is applied to servers. For example... Figure 19 As shown, the obstacle avoidance system for the wheeled legged robot includes: a dual-channel accessibility field construction module 100, a robot state set prediction module 200, a mode switching reachability prediction module 300, a switching feasibility determination module 400, a target trajectory planning module 500, and a safety control module 600. Among these, The dual-channel mobility field construction module 100 is used to acquire the current motion state, current motion mode and environmental information of the wheel-legged robot. The environmental information includes static environment occupancy information, dynamic obstacle state information and terrain semantic information. Based on the environmental information, the module obtains static occupancy representation, dynamic obstacle spatiotemporal occupancy representation with time index and terrain semantic grid, and then constructs a modality-related dual-channel mobility field including wheel cost channel and leg cost channel. The module also determines the target motion mode based on the cost comparison results of wheel cost channel and leg cost channel. The robot state set prediction module 200 is used to predict the robot state during the mode switching phase change according to the preset mode switching dynamics model and prediction step size when the target motion mode is different from the current motion mode, and obtain the robot state set corresponding to multiple prediction time. The mode switching phase is used to characterize the progress of the switch from the current motion mode to the target motion mode. The mode switching reachability prediction module 300 is used to determine the space occupancy set corresponding to each prediction time based on the robot state set at each prediction time. The space occupancy set is the union of the body occupancy set and the wheel and leg mechanism occupancy set at the corresponding prediction time. The prediction time and its corresponding space occupancy set are associated in time order to form a mode switching spatiotemporal reachability tube. The switching feasibility determination module 400 is used to perform spatiotemporal alignment determination between the mode switching spatiotemporal reachability tube and the static occupancy representation, the dynamic obstacle spatiotemporal occupancy representation, and the terrain constraints determined based on the terrain semantic grid, so as to obtain the mode switching safety margin and the mode switching feasible time interval. The target trajectory planning module 500 is used to generate mode-maintaining candidate trajectories that maintain the current motion mode and mode-switching candidate trajectories that switch to the target motion mode within the prediction time domain. It filters the mode-switching candidate trajectories according to the feasible time interval of mode switching, evaluates the segmented cost of the filtered mode-switching candidate trajectories, and determines the target trajectory, mode-switching start time and reference control command based on the evaluation results and the mode-maintaining candidate trajectories. The reference control command includes recommended linear velocity, recommended angular velocity and target motion mode. The safety control module 600 is used to correct the reference control command for safety constraints, based on the hard constraints that the mode switching safety margin is not lower than the preset safety threshold and that the regions constrained by the spatiotemporal reachability tube of mode switching and the static occupancy representation, the spatiotemporal occupancy representation of dynamic obstacles and the terrain semantic grid do not intersect. This allows the wheeled robot to maintain the current motion mode, switch to the target motion mode at the start of the mode switching, or perform safety braking.

[0154] For ease of description, the above system is described by dividing it into various modules based on their functions. Of course, in implementing the embodiments of the present invention, the functions of each module can be implemented in one or more software and / or hardware.

[0155] The system described in the above embodiments is applied to the corresponding method in the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0156] Based on the same inventive concept, embodiments of the present invention also provide an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method described in any of the above embodiments.

[0157] This invention provides a non-volatile computer storage medium storing at least one executable instruction that can execute the method described in any of the above embodiments.

[0158] Figure 20 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 501, a memory 502, an input / output interface 503, a communication interface 504, and a bus 505. The processor 501, memory 502, input / output interface 503, and communication interface 504 are interconnected internally via the bus 505.

[0159] The processor 501 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0160] The memory 502 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 502 can store the operating system and other application programs. When the technical solution provided by the method embodiment of the present invention is implemented by software or firmware, the relevant program code is stored in the memory 502 and is called and executed by the processor 501.

[0161] Input / output interface 503 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0162] Communication interface 504 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0163] Bus 505 includes a pathway for transmitting information between various components of the device (e.g., processor 501, memory 502, input / output interface 503, and communication interface 504).

[0164] It should be noted that although the above-described device only shows the processor 501, memory 502, input / output interface 503, communication interface 504, and bus 505, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of the present invention, and does not necessarily include all the components shown in the figures.

[0165] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of the different aspects of this application as described above, which are not provided in detail for the sake of brevity. This application is intended to cover all such substitutions, modifications, and variations falling within the broad scope of the embodiments of this invention. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this invention should be included within the protection scope of this application.

Claims

1. A method for obstacle avoidance in a wheeled legged robot based on a mode-switching spatiotemporal reachability tube, characterized in that, include: The current motion state, current motion mode, and environmental information of the wheeled-legged robot are acquired. The environmental information includes static environment occupancy information, dynamic obstacle state information, and terrain semantic information. Based on the environmental information, a static occupancy representation, a dynamic obstacle spatiotemporal occupancy representation with time index, and a terrain semantic grid are obtained. Then, a modal correlation drivability field dual channel including a wheel cost channel and a leg cost channel is constructed, and the target motion mode is determined based on the cost comparison results of the wheel cost channel and the leg cost channel. When the target motion mode is different from the current motion mode, the robot state during the mode switching phase change is predicted time by time according to the preset mode switching dynamics model and prediction step size, so as to obtain a set of robot states corresponding to multiple prediction time. The mode switching phase is used to characterize the switching progress from the current motion mode to the target motion mode. Based on the robot state set at each prediction time, the corresponding space occupancy set at each prediction time is determined. The space occupancy set is the union of the body occupancy set and the wheel-leg mechanism occupancy set at the corresponding prediction time. Each prediction time and its corresponding space occupancy set are associated in chronological order to form a mode switching spatiotemporal reachability tube. The mode switching spatiotemporal reachability tube is spatiotemporally aligned with the static occupancy representation, the dynamic obstacle spatiotemporal occupancy representation, and the terrain constraints determined based on the terrain semantic grid to obtain the mode switching safety margin and the mode switching feasible time interval. Generate a mode-maintaining candidate trajectory for maintaining the current motion mode and a mode-switching candidate trajectory for switching to the target motion mode within the prediction time domain. Filter the mode-switching candidate trajectories according to the feasible time interval for mode switching. Evaluate the segmented cost of the filtered mode-switching candidate trajectories. Determine the target trajectory, mode-switching initiation time, and reference control command based on the evaluation results and the mode-maintaining candidate trajectories. The reference control command includes recommended linear velocity, recommended angular velocity, and target motion mode. Using the safety margin of the mode switching not being lower than a preset safety threshold and the fact that the spatiotemporal reachability tube of the mode switching does not intersect with the area constrained by the static occupancy representation, the spatiotemporal occupancy representation of the dynamic obstacle, and the terrain semantic grid as hard constraints, the reference control command is modified to maintain the current motion mode, or switch to the target motion mode or perform safety braking at the start of the mode switching.

2. The obstacle avoidance method for wheeled-legged robots based on mode-switching spatiotemporal reachability tubes as described in claim 1, characterized in that, The process of obtaining a static occupancy representation, a dynamic obstacle spatiotemporal occupancy representation with a time index, and a terrain semantic raster based on the environmental information includes: The occupation grid map is updated based on the static environment occupation information, and a symbolic Euclidean distance field map is generated using Euclidean distance transformation, with the symbolic Euclidean distance field map serving as the static occupation representation. Based on the dynamic obstacle state information, motion trend prediction is performed, the predicted position is mapped to a grid of future time series, and the dynamic occupancy value of the grid is updated according to the time decay weight to generate a spatiotemporal occupancy representation of the dynamic obstacle with time index. The terrain semantic information is obtained, and a terrain semantic grid is labeled according to the terrain semantic information. The terrain semantic grid includes at least three types of terrain semantic labels: flat area, heterogeneous passable area, and bimodal restricted area.

3. The obstacle avoidance method for wheeled-legged robots based on mode-switching spatiotemporal reachability tubes as described in claim 1, characterized in that, The process further constructs a dual-channel modal correlation drivability field, including a wheel-walking cost channel and a leg-walking cost channel, and determines the target motion mode based on the cost comparison results of the wheel-walking cost channel and the leg-walking cost channel, including: The single-point cost is calculated for each grid position within a local window centered on the robot's current position in both wheel-walking and leg-walking modes. The single-point cost is obtained by weighted summation of geometric safety cost, dynamic obstacle spatiotemporal occupancy cost, modal-terrain coupling cost, and state-mode coupling cost. Wherein, the geometric safety cost is determined based on the static occupancy representation, the dynamic obstacle spatiotemporal occupancy cost is determined based on the dynamic obstacle spatiotemporal occupancy representation, the modality-terrain coupling cost is determined based on the correspondence between the terrain semantic grid and the motion pattern, and the state-mode coupling cost is determined based on the matching relationship between the current motion state and the motion pattern; Based on the single-point cost, a wheel-walking mode cost slice and a leg-walking mode cost slice are generated respectively, and stacked into a dual-channel tensor, which serves as the wheel-walking cost channel and the leg-walking cost channel; The cost values ​​of the current position in wheel mode and leg mode are obtained and compared according to the wheel movement cost channel and the leg movement cost channel. If the cost of wheel mode is less than the cost of leg mode, wheel mode is set as the target movement mode; otherwise, leg mode is set as the target movement mode.

4. The obstacle avoidance method for wheeled-legged robots based on mode-switching spatiotemporal reachability tubes as described in claim 1, characterized in that, The robot state during the mode switching phase change is predicted time-by-time based on a preset mode switching dynamics model and prediction step size, resulting in a set of robot states corresponding to multiple prediction times, including: The uncertainty interval of the current motion state is used as the initial state set; The initial state set is propagated step by step according to the mode switching dynamics model expressed in discrete time form, with the prediction step size as the time interval. By superimposing a set of perturbations determined by model error, perception error and execution delay at each prediction step, a set of robot states corresponding to multiple prediction times is obtained. The robot state set includes the robot's position, heading angle, linear velocity, angular velocity, body height, body posture, wheel and leg mechanism deployment degree, wheel and leg mechanism speed, and mode switching phase.

5. The obstacle avoidance method for a wheeled legged robot based on a mode-switching spatiotemporal reachability tube as described in claim 4, characterized in that, The step involves determining the spatial occupancy set corresponding to each predicted time based on the robot state set at each predicted time. This spatial occupancy set is the union of the body occupancy set and the wheel-leg mechanism occupancy set at the corresponding predicted time. Each predicted time and its corresponding spatial occupancy set are then associated in chronological order to form a mode-switching spatiotemporal reachability tube, including: For each predicted moment, based on the position, body posture, body height, and wheel-leg mechanism deployment degree in the robot state set at that predicted moment, the robot body geometric model and the wheel-leg mechanism geometric model are transformed to a unified environmental coordinate system, and the body occupancy set and the wheel-leg mechanism occupancy set are determined respectively. The union of the body occupancy set and the wheel-leg mechanism occupancy set is taken as the spatial occupancy set at the predicted moment; The spatial occupancy set changes with the mode switching phase to characterize the continuous change of the robot's external boundary during the mode switching process; By associating each predicted moment and its corresponding spatial occupancy set in chronological order, a spatiotemporal reachability tube for mode switching from the current motion mode to the target motion mode is formed.

6. The obstacle avoidance method for a wheeled legged robot based on a mode-switching spatiotemporal reachability tube as described in claim 1, characterized in that, The step of performing spatiotemporal alignment determination between the mode switching spatiotemporal reachability tube and the static occupancy representation, the dynamic obstacle spatiotemporal occupancy representation, and the terrain constraints determined based on the terrain semantic raster to obtain the mode switching safety margin and the mode switching feasible time interval includes: For each spatiotemporal slice in the modality switching spatiotemporal reachability tube, the spatial occupancy set corresponding to the prediction time is intersected with the region extracted from the static occupancy representation, the region extracted from the dynamic obstacle spatiotemporal occupancy representation corresponding to the prediction time, and the region extracted from the terrain semantic raster that does not meet the support conditions. The mode switching safety margin is determined based on the minimum spatial distance between each spatiotemporal slice and the above three regions when no intersection occurs. Using candidate start times as parameters, a family of reachable tubes parameterized by start times is generated. The intersection detection and state constraint determination are performed on the spatiotemporal reachable tubes corresponding to each candidate start time. The set of consecutive start times where the mode switching safety margin is not less than a preset safety threshold, all state constraints are satisfied throughout the process, and the state of the switching terminal falls into the feasible state domain of the target motion mode is determined as the feasible time interval for mode switching. The earliest feasible switching time within the feasible time interval for mode switching is determined by the time required for the current motion mode to decelerate to the allowable switching speed, and the latest safe switching time is determined by the remaining time to reach the target terrain boundary or interfere with the obstacle.

7. The obstacle avoidance method for wheeled-legged robots based on mode-switching spatiotemporal reachability tubes as described in claim 1, characterized in that, The process involves generating candidate trajectories for maintaining the current motion mode within the predicted time domain and candidate trajectories for switching to the target motion mode. The candidate trajectories are then filtered based on the feasible time interval for mode switching. Segmented cost evaluations are performed on the filtered candidate trajectories. Based on the evaluation results and the candidate trajectories for maintaining the current motion mode, the target trajectory, the mode switching initiation time, and the reference control command are determined, including: Generate mode-preserving candidate trajectories that maintain the current motion mode within the prediction time domain, as well as mode-switching candidate trajectories that switch to the target motion mode; The candidate trajectories for mode switching are filtered according to the feasible time interval for mode switching, and the candidate trajectories for mode switching that do not meet the constraints of the feasible time interval for mode switching are eliminated. The selected candidate trajectories for mode switching are divided into pre-switching, mid-switching, and post-switching segments according to the switching start and completion times. Segmented cost evaluations are performed on each segment, including: accumulating the modal-related traversability cost corresponding to the current motion mode for the pre-switching segment; accumulating the transition cost determined by the mode switching phase, robot state, and environmental spatiotemporal occupancy for the mid-switching segment; and accumulating the modal-related traversability cost corresponding to the target motion mode for the post-switching segment. The total cost of the candidate trajectory for mode switching is then calculated by adding the mode switching time cost, mode switching energy cost, reachability risk cost determined based on the mode switching safety margin, and frequent mode switching suppression cost. Calculate the total cost of the mode-keeping candidate trajectory for maintaining the current motion mode, compare the total cost of each mode-switching candidate trajectory with the total cost of the mode-keeping candidate trajectory, select the trajectory with the minimum total cost as the target trajectory, and determine the mode-switching initiation time and reference control command based on the target trajectory.

8. An obstacle avoidance system for a wheeled legged robot based on a mode-switching spatiotemporal reachability tube, characterized in that, include: The dual-channel mobility field construction module is used to acquire the current motion state, current motion mode, and environmental information of the wheeled and legged robot. The environmental information includes static environment occupancy information, dynamic obstacle state information, and terrain semantic information. Based on the environmental information, a static occupancy representation, a dynamic obstacle spatiotemporal occupancy representation with time index, and a terrain semantic grid are obtained. Then, a modality-related mobility field dual-channel including a wheeled cost channel and a legged cost channel is constructed, and the target motion mode is determined based on the cost comparison results of the wheeled cost channel and the legged cost channel. The robot state set prediction module is used to predict the robot state during the mode switching phase change according to a preset mode switching dynamics model and prediction step size when the target motion mode is different from the current motion mode, and obtain a set of robot states corresponding to multiple prediction times. The mode switching phase is used to characterize the progress of the switch from the current motion mode to the target motion mode. The mode switching reachability prediction module is used to determine the space occupancy set corresponding to each prediction time based on the robot state set at each prediction time. The space occupancy set is the union of the body occupancy set and the wheel and leg mechanism occupancy set at the corresponding prediction time. The prediction time and its corresponding space occupancy set are associated in chronological order to form a mode switching spatiotemporal reachability tube. The mode switching feasibility determination module is used to perform spatiotemporal alignment determination between the mode switching spatiotemporal reachability tube and the static occupancy representation, the dynamic obstacle spatiotemporal occupancy representation, and the terrain constraints determined based on the terrain semantic grid, so as to obtain the mode switching safety margin and the mode switching feasible time interval. The target trajectory planning module is used to generate mode-maintaining candidate trajectories for maintaining the current motion mode and mode-switching candidate trajectories for switching to the target motion mode within the prediction time domain. The mode-switching candidate trajectories are filtered according to the feasible time interval of the mode-switching, and the segmented cost of the filtered mode-switching candidate trajectories is evaluated. Based on the evaluation results and the mode-maintaining candidate trajectories, the target trajectory, the mode-switching start time, and the reference control command are determined. The reference control command includes the recommended linear velocity, the recommended angular velocity, and the target motion mode. The safety control module is used to correct the reference control command based on the hard constraints that the mode switching safety margin is not lower than a preset safety threshold and that the spatiotemporal reachability tube of the mode switching does not intersect with the area constrained by the static occupancy representation, the dynamic obstacle spatiotemporal occupancy representation and the terrain semantic grid. This allows the wheeled robot to maintain its current motion mode, switch to the target motion mode or perform safety braking at the start of the mode switching.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the obstacle avoidance method as described in any one of claims 1-7.

10. A computer storage medium, characterized in that, The storage medium stores at least one executable instruction that causes the processor to perform the obstacle avoidance method as described in any one of claims 1-7.

Images