A Large Model-Driven Method for Building Outdoor Hybrid Navigation Maps for Robots

Abstract

This invention relates to the field of robot outdoor navigation technology and discloses a large-model-driven method for constructing a hybrid outdoor navigation map for robots, comprising the following steps: S1, collecting point cloud and image data through airborne sensors in an outdoor non-flat environment; S2, constructing a large-scale textual visual model based on terrain text and image data, and outputting terrain segmentation results; S3, iteratively fusing terrain image labels to generate a local point cloud map with terrain semantic labels based on point cloud data; S4, extracting terrain geometric information and constructing a local grid map representation model; S5, storing terrain geometric and semantic features in grid cells, constructing a comprehensive cost function, and designing an adaptive map resolution to update the local grid map; S6, establishing a global topological map, fusing a hierarchical mapping mechanism, and constructing a grid-topology hybrid navigation map. This invention has strong adaptability and scalability, providing a robust map foundation for subsequent motion planning.

Description

Technical Field

[0001] This invention relates to the field of robot outdoor navigation technology, specifically to a method for constructing a large-scale model-driven robot outdoor hybrid navigation map. Background Technology

[0002] Navigation map construction has long been a focus of attention in the robotics community, with widespread applications in autonomous outdoor operations of mobile robots. In outdoor environments, the advantage of using mobile robots for navigation map construction lies in its ability to gradually build environmental representations suitable for complex terrains through online perception and mapping, without relying on prior information. This enhances the robot's autonomy and operational efficiency in unknown scenarios. To ensure that the constructed navigation map is applicable to various types of mobile robots, it needs to meet three basic requirements: reliability, adaptability, and scalability.

[0003] To address the navigation map construction problem for mobile robots, the robot needs to perceive its surrounding environment through sensors (such as LiDAR, cameras, and IMUs), acquiring multimodal data such as the environment's geometric structure and terrain semantic information, and constructing a corresponding navigation map to provide basic input for subsequent path planning. Currently, several types of navigation maps are used for environmental modeling and task execution, which can be categorized as: metric maps, topological maps, semantic maps, and hybrid maps. Metric maps offer high localization and navigation accuracy, and are easy to build and maintain for small areas; however, it is difficult to build metric maps for large outdoor areas, resulting in high path planning costs, reliance on sensor reliability, and the need for accurate position estimation. Topological maps represent the connectivity of various locations in the environment; they only describe the structure of the environment and are easy to build for large outdoor areas. Semantic maps introduce the categories and attributes of environmental objects on top of geometric information, supporting higher-level understanding and decision-making, but their construction often relies on high-quality scene annotation and model inference. Hybrid maps typically combine two or more map types, aiming to balance local accuracy and global connectivity, achieving complementary advantages between different map types. The above map construction methods, whether singular or simple hybrid maps, are difficult to manage and maintain, and it is also difficult to simultaneously possess reliability, adaptability, and scalability.

[0004] Patent 117804420A discloses a robot navigation method and system based on multi-layer maps, constructing a hybrid map based on semantic topology and semantic grids; Patent 111782828A discloses an efficient method for constructing a composite map of terrain in front of a vehicle, realizing rapid updating, storage, and retrieval of multi-layer grid composite map data. However, the above solutions do not fully consider the synergy between terrain geometry and semantic feature information, and lack robustness in extracting terrain semantic features, failing to meet the complex needs of outdoor navigation map construction.

[0005] The journal article "Hybrid Map-Based Path Planning for Robot Navigation in Unstructured Environments" proposes a hybrid map construction method that integrates two-dimensional grids and 2.5-dimensional elevation maps, achieving the integration of spatial information of terrain features, but it does not consider the semantic information of terrain features. The paper "Research on Robot Construction and Application of Multi-Attribute Map of Field Terrain" (Harbin Institute of Technology, 2024, doi:10.27061 / d.cnki.ghgdu.2024.000626) proposes a map construction method based on multimodal large models and multi-attribute perception of terrain, but its ray casting method for updating grid occupancy probabilities consumes a lot of computational resources.

[0006] Based on this, this invention proposes a large-model-driven method for constructing a robot outdoor hybrid navigation map. Compared with the above technologies, the innovation of this invention lies in parsing the semantic features of the terrain through a designed text-visual large-scale model. This model incorporates a loss function based on terrain semantic association constraints, combined with map geometric features, into a lightweight metric map. Furthermore, a mapping mechanism between topological nodes and grid cells is constructed. During obstacle avoidance, the metric grid layer in the hybrid navigation map can be used to achieve highly safe obstacle avoidance, while navigation tasks such as path planning are efficiently completed using the topological layer. Summary of the Invention

[0007] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a large-model-driven method for constructing outdoor hybrid navigation maps for robots. This method offers advantages such as high reliability and scalability in constructing outdoor navigation maps, stronger adaptability to outdoor environments, and wider application scenarios.

[0008] (II) Technical Solution To achieve the aforementioned goals of constructing highly reliable and scalable outdoor navigation maps, with greater adaptability to outdoor environments and wider application scenarios, this invention provides the following technical solution: a large-model-driven method for constructing hybrid outdoor navigation maps for robots, comprising the following steps: S1. In an outdoor environment, point cloud data and image data of the current environment are collected by the lidar and camera carried by the mobile robot. S2. Construct a large-scale textual visual model of cross-modal terrain semantic information based on terrain text and image data, and verify and optimize the terrain semantic mask through the mask verification module to output the terrain segmentation result; S3. Based on point cloud data, a local point cloud map is constructed using the SLAM method. The terrain labels of multiple frames of images are iteratively fused using recursive Bayesian rules to update the visible and effective point cloud voxels within the camera's view frustum, thus obtaining a local point cloud map with terrain semantic labels. S4. Extract terrain geometry information from the local point cloud map and construct a symbolic distance-volume mapping model to represent the local raster map; The terrain geometry features include at least terrain slope and terrain roughness; S5. Store the terrain geometric features and terrain semantic features of each grid cell in the local grid map, and construct the grid cell comprehensive cost function based on the terrain geometric features and terrain semantic features. At the same time, construct the adaptive grid map resolution based on the importance of terrain semantics to update the local grid map. S6. Construct a global topology map based on the distortion-corrected LiDAR frame and the robot's current odometry pose, establish a mapping mechanism between topology nodes and grid cells, and fuse the hierarchical information of the grid layer and the topology layer to construct a grid-topology hybrid navigation map.

[0009] In this invention, further, in step S2, a large-scale textual visual model of cross-modal terrain semantic information is constructed based on the terrain text and the image data, and the terrain segmentation result is output through a mask verification module. Specifically, this includes the following steps: S201. Perform simple linear iterative clustering on the input original visual image to segment the original visual image into multiple homogeneous terrain blocks; S202. Based on the large text-image model, establish the text-image matching relationship between terrain text prompts and image patches, convert terrain text prompts into embedded features, and extract the visual features of image patches; S203. Generate a heat map based on the embedded features and visual features, and locate terrain semantic cue points based on the heat map to construct a cue point set; S204. Input the set of cue points into the semantic segmentation model to generate an initial semantic mask, and establish an effective mask filtering mechanism to filter the initial semantic mask. S205. The initial semantic mask is reconstructed and semantically optimized through the mask verification module, and the final terrain segmentation result is output.

[0010] In this invention, the set of prompt points is further represented as:

[0011] in, Indicates the first i The location of each terrain semantic cue point in the image. Indicates the first iSemantic labels corresponding to each terrain semantic cue point; The mask verification module is a mask verification module with a Transformer encoder-decoder structure, and it fuses visual features and mask semantics through a multi-head attention mechanism.

[0012] Furthermore, in this invention, the large-scale text visual model is trained using a joint loss function, which is:

[0013] in, The cross-entropy loss is obtained by weighting the semantic similarity matrix of terrain categories. The Dice loss is obtained by weighting the semantic importance of terrain categories. Loss due to spatial continuity constraints; , , These are the weighting coefficients for the corresponding loss terms.

[0014] In this invention, further, in step S3, the terrain labels of multiple frames of images are iteratively fused using recursive Bayesian rules to update the visible and effective point cloud voxels within the camera's view frustum, specifically including the following steps: S301. Perform dynamic voxel grid segmentation on the local point cloud map to reduce the size of the original point cloud data and filter out redundant information. S302. Filter out overhanging structures from the local point cloud map and extract the effective surface point cloud; S303. A multi-resolution point cloud voxel fitting strategy is adopted so that each point cloud voxel stores the corresponding discrete probability distribution and semantic label set. S304. Project the point cloud voxels onto the current image to obtain the pixel coordinates of the point cloud voxels in the current image; S305. Based on the semantic labels observed at the corresponding pixel coordinates in the current image, recursively update the semantic label probabilities of the point cloud voxels. The voxel dimensions used in dynamic voxel mesh segmentation include voxel length. voxel width and voxel height And satisfy:

[0015] in, Indicates the length of the robot body. Indicates the width of the robot body. This indicates the maximum slope the robot can traverse.

[0016] In this invention, further, in step S3, the recursive Bayes rule is expressed as:

[0017] in, For image sequences, This is a local point cloud map, where Z is a normalization constant. For posterior probability, For prior probability, To make voxels Projected onto the current image The pixel coordinates obtained above, In the image Corresponding pixel position Observed semantic tags.

[0018] In this invention, further, step S4, extracting terrain geometric information from the local point cloud map, specifically includes the following steps: S401. Fit the local plane of point cloud voxels using principal component analysis; S402. Calculate the terrain slope of each point cloud voxel based on the angle between the normal vector of the local plane and the horizontal plane. S403. Calculate the terrain roughness of each point cloud voxel based on the distance between each surface point within the point cloud voxel and the geometric center of the point cloud voxel. The slope of the terrain is expressed as:

[0019] Terrain roughness is expressed as:

[0020] in, This represents the angle between the normal vector of the local plane and the horizontal plane. For voxels k One ground point, Centered at the geometric center of the voxel n This indicates the number of ground points within a point cloud voxel.

[0021] In this invention, further, in step S5, the grid cell comprehensive cost function is constructed based on terrain slope, terrain roughness, terrain semantic confidence, and the probability of dynamic obstacle presence. The grid cell comprehensive cost value is expressed as:

[0022] in, ; This is a highly accessible unit area. This is a non-passable unit area; , , , These represent terrain slope, terrain roughness, terrain semantic confidence, and dynamic obstacle weight coefficients, respectively. and These represent setting the maximum terrain slope and roughness thresholds, respectively. This represents the cost of semantic uncertainty in terrain representation; This represents the probability of the presence of a dynamic obstacle.

[0023] In this invention, further, in step S5, an adaptive raster map resolution based on terrain semantic importance is constructed, and the local raster map is dynamically updated, specifically including the following steps: S501. Classify terrain importance levels based on semantic information; S502: Maintain high resolution for critical navigation areas such as roads and grasslands, and reduce resolution for non-critical areas; S503. Combining the robot's current moving speed, the robot's maximum speed threshold, and semantic importance factors, a speed-semantic joint grid map resolution mapping model is constructed. The velocity-semantic joint raster map resolution mapping model is represented as follows:

[0024] in, Scaling factor d res The current raster map resolution, d max , d min These represent the maximum and minimum resolutions of the raster map, respectively. v max The maximum speed threshold for the robot. As a semantic importance factor, This indicates the robot's current moving speed. Indicates the influence of speed; S504. Use a local raster map size fixing strategy to determine the fixed size of the local raster map; The fixed size of the local raster map satisfies:

[0025] in, This indicates a fixed size for a local raster map. This indicates the robot's displacement within the map update interval. Indicates the robot's braking distance. Indicates the error redundancy distance; The robot displacement within the map update interval satisfies: ; The robot's braking distance meets the following requirements: ; in, Indicates the robot map update interval. a max This represents the maximum braking line acceleration on the ground. Indicates the obstacle avoidance response time; S505. A sliding window update strategy is adopted to dynamically calculate the number of sliding grids; The number of sliding grids satisfies: ; S506. During the local grid map update process, insert new observation grids along the robot's movement direction and simultaneously cover the old grids in the opposite direction to maintain a constant total number of grids.

[0026] In this invention, further, in step S6, a mapping mechanism is established between topology nodes and grid cells, and the hierarchical information of the grid layer and the topology layer is fused, specifically including the following steps: S601. The denoised LiDAR point cloud is fused with the robot odometry pose to generate a global point cloud map with spatiotemporal consistency. S602. Establish topological nodes using key raster cells with significant geometric changes and / or semantic boundaries in the raster map as anchor points; S603. Associate each topology node with a set of grid cells and store the pose of the topology node and the connectivity relationship with adjacent topology nodes; S604. By using spatial distance constraints and semantic consistency constraints, the grid cells are assigned to the corresponding topology nodes; The mapping relationship between topology nodes and grid cells is expressed as follows: ; in, For grid cells The center coordinates, For topology nodes Location, For the semantic tags of the grid cells, Represents topology nodes The set of terrain semantic labels for the currently associated raster cells. The semantic entropy function of a set; S605. Map the total cost of the grid cells in the grid layer to the topological edge weights in the topology layer. The edge weights between topological nodes are represented as follows: ; in, The edge weights between topological nodes. This is the set of grid cell paths that connect two nodes. Indicates the current grid cell The corresponding comprehensive value, This represents the step size of the grid cell along the path.

[0027] (III) Beneficial Effects Compared with existing technologies, this invention provides a large-model-driven method for constructing outdoor hybrid navigation maps for robots, which has the following advantages: 1. The large model-driven robot outdoor hybrid navigation map construction method constructs a text-visual large model of cross-modal terrain semantic information based on terrain text and image data, and combines a mask verification module to verify and optimize the terrain semantic mask, which can improve the accuracy and robustness of semantic recognition of complex outdoor terrains such as roads, grass, gravel, and puddles, thus providing a more reliable terrain classification foundation for subsequent map construction.

[0028] 2. The large model-driven method for constructing a robot's outdoor hybrid navigation map recursively fuses terrain labels from multiple frames of images with point cloud voxels from a local point cloud map. This enables the point cloud map to not only express the spatial structure of the environment but also carry terrain semantic information, thereby improving the robot's comprehensive perception of the outdoor environment.

[0029] 3. The large model-driven robot outdoor hybrid navigation map construction method extracts geometric information such as terrain slope and terrain roughness from local point cloud maps and stores them together with terrain semantic features in the grid cells of the local grid map. This enables the grid map to simultaneously reflect terrain type, terrain undulation and flatness, thereby improving the accuracy of terrain accessibility judgment.

[0030] 4. The large model-driven robot outdoor hybrid navigation map construction method constructs a grid cell comprehensive cost function based on terrain slope, terrain roughness, terrain semantic confidence and the probability of dynamic obstacle existence. This enables a comprehensive evaluation of the passage risk of different grid areas, thereby achieving accurate division of highly passable and impassable areas.

[0031] 5. The large model-driven method for constructing outdoor hybrid navigation maps for robots builds an adaptive grid map resolution based on the semantic importance of terrain, and dynamically updates the local grid map in combination with the robot's motion state. This method can maintain high map accuracy in critical navigation areas and reduce computational and storage burden in non-critical areas, thus balancing map representation accuracy and real-time update efficiency. Attached Figure Description

[0032] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This invention provides a framework for constructing local raster maps that couple terrain semantic geometric information. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Example: Please see Figure 1 and Figure 2 This invention discloses a large-model-driven method for constructing hybrid outdoor navigation maps for robots. This method is primarily applied to navigation map construction scenarios for mobile robots in complex outdoor environments, such as park roads, grasslands, gravel areas, muddy areas, puddles, and sloping terrain. The mobile robot includes a robot body, LiDAR, camera, odometry, IMU, and computing unit. The LiDAR is used to collect 3D point cloud data of the current environment; the camera is used to collect visual image data of the current environment; the odometry and IMU provide robot motion state and pose information; and the computing unit performs calculations such as terrain semantic segmentation, point cloud map construction, grid map updating, and topology map construction. The LiDAR, camera, odometry, IMU, computing unit, and SLAM method itself can all be implemented using existing equipment or algorithms. The focus of this invention is to collaboratively fuse terrain semantic information, point cloud geometric information, local grid maps, and global topology maps obtained from a large-scale textual visual model, thereby improving the reliability, adaptability, and scalability of the outdoor navigation map.

[0035] The method of the present invention specifically includes the following steps: S1. In an outdoor environment, point cloud data and image data of the current environment are collected by the lidar and camera carried by the mobile robot. Specifically, after the mobile robot enters the outdoor environment to be navigated or mapped, it turns on the LiDAR, camera, computing unit, and power supply, putting the LiDAR and camera into data acquisition mode. The LiDAR collects 3D point cloud data of the robot's surrounding environment, while the camera simultaneously collects visual image data of the current environment. In order to facilitate the subsequent fusion of image semantic results into the point cloud map, the LiDAR and camera can be calibrated with external parameters, and the collected point cloud data and image data can be synchronized in time, so that point cloud frames and image frames collected at the same time or adjacent times can establish a correspondence.

[0036] In this embodiment, point cloud data is used to represent the spatial structure and terrain undulations in the outdoor environment, while image data is used to extract semantic information of terrain categories such as roads, grasslands, gravel, puddles, and swamps. By simultaneously collecting point cloud data and image data, the subsequent map construction process can utilize not only the geometric representation capabilities of point clouds but also the terrain semantic recognition capabilities of images.

[0037] S2. Construct a large-scale textual visual model of cross-modal terrain semantic information based on terrain text and image data, and verify and optimize the terrain semantic mask through the mask verification module to output the terrain segmentation result; Specifically, for the input raw visual image I Simple linear iterative clustering is performed to segment the original visual image into multiple homogeneous terrain patches. The original visual image can be represented as: ; in, H Indicates the image height. W 3 represents the image width, and 3 represents the number of color channels in the image.

[0038] In this embodiment, simple linear iterative clustering can group pixels in an image that are similar in color, texture, and spatial location into homogeneous terrain blocks, thereby reducing the amount of data required for subsequent semantic matching and segmentation processing, and making the generated terrain cue points more stable. Homogeneous terrain blocks can correspond to local image regions such as road areas, grassland areas, gravel areas, and puddles.

[0039] Furthermore, based on the text-image big model, a text-image matching relationship is established between terrain text prompts and image patches. The terrain text prompts can include texts related to outdoor terrain categories such as "road", "grass", "gravel", "puddles", "swamp", "muddy ground", and "slope". The text-image big model converts the terrain text prompts into text embedding features, extracts the visual features of each image patch, and generates a terrain semantic heatmap based on the matching relationship between the text embedding features and the visual features. By using terrain semantic heatmaps, terrain semantic cue points corresponding to various terrain types can be located. This leads to the construction of a set of cue points:

[0040] in, Indicates the first i The location of each terrain semantic cue point in the image. This represents the semantic label corresponding to the i-th terrain semantic cue point; Furthermore, the set of cue points P Inputting a large semantic segmentation model generates an initial semantic mask, which is used to label the regions of different terrain categories in the image. To reduce the impact of mismatched cue points or noisy masks on subsequent mapping, an effective mask filtering mechanism can be established to filter the initial semantic masks. For example, obviously abnormal masks can be filtered based on indicators such as mask area, mask confidence, consistency between the mask and the heatmap response region, and mask boundary continuity, retaining only effective semantic masks.

[0041] Then, a mask verification module with a Transformer encoder-decoder structure is designed. This module fuses visual features and mask semantics through a multi-head attention mechanism to achieve feature reconstruction and semantic optimization. Specifically, the encoder can jointly encode the visual features, semantic mask features, and textual semantic features of image patches. The decoder reconstructs or corrects the semantic mask based on the encoding results, making the final terrain segmentation result more stable in terms of category boundaries, regional continuity, and differentiation of similar terrain.

[0042] To enhance the ability of the large-scale text vision model to distinguish between multiple terrain types, a joint loss function is constructed:

[0043] in, The cross-entropy loss is obtained by weighting the semantic similarity matrix of terrain categories. The Dice loss is obtained by weighting the semantic importance of terrain categories. Loss due to spatial continuity constraints; , , These are the weighting coefficients for the corresponding loss terms.

[0044] Through the above processing, the large text vision model can not only identify the terrain category in the image based on the terrain text prompts, but also improve the reliability of the terrain semantic segmentation results by combining the mask verification module, thereby providing accurate terrain labels for subsequent point cloud semantic fusion and raster map construction.

[0045] S3. Based on point cloud data, a local point cloud map is constructed using the SLAM method. The terrain labels of multiple frames of images are iteratively fused using recursive Bayesian rules to update the visible and effective point cloud voxels within the camera's view frustum, resulting in a local point cloud map with terrain semantic labels.

[0046] Specifically, the computing unit constructs a local point cloud map based on the point cloud data collected by the LiDAR using the SLAM method. The local point cloud map is used to represent the environmental spatial structure within a certain range around the robot's current position. During the SLAM mapping process, the robot's odometry pose and the inter-frame matching results of the LiDAR can be combined to register and fuse multiple frames of point cloud data to obtain a point cloud map with local consistency. Furthermore, to reduce the data size of the original point cloud and filter out redundant information, dynamic voxel mesh segmentation is performed on the local point cloud map. Discretizing the local point cloud map by voxel mesh can compress a large number of point cloud points into voxel units, thereby reducing the amount of subsequent computation. At the same time, overhanging structures, such as tree branches, suspended obstacles, and trellises, can be filtered out based on features such as voxel height distribution, local normal vectors, and ground continuity, and effective surface point clouds can be extracted.

[0047] To accommodate the robot's mobility, in this embodiment, the voxel size is determined by the robot's body size and the maximum passable slope: Voxel dimensions include voxel length voxel width and voxel height And satisfy:

[0048] in, Indicates the length of the robot body. Indicates the width of the robot body. This indicates the maximum slope the robot can traverse; Determining the voxel size using the above method allows the voxel scale to be adapted to the robot's mobility, avoiding excessive computation due to excessively small voxels and loss of terrain details due to excessively large voxels.

[0049] Furthermore, based on the ground point cloud and the visual terrain semantic segmentation results obtained in step S2, the point cloud semantic labels are updated using Bayesian probability. Specifically, a multi-resolution point cloud voxel fitting strategy is adopted, so that each point cloud voxel stores the corresponding discrete probability distribution and semantic label set. For a valid point cloud voxel that is located within the camera's view frustum and is visible, the voxel is projected into the current image to obtain the pixel coordinates of the voxel in the current image; then, based on the semantic labels observed at the corresponding pixel coordinates in the current image, the semantic label probability of the voxel is recursively updated. Specifically, the recursive Bayes rule is expressed as:

[0050] in, For image sequences, This is a local point cloud map, where Z is a normalization constant. For posterior probability, For prior probability, To make voxels Projected onto the current image The pixel coordinates obtained above, In the image Corresponding pixel position Observed semantic tags.

[0051] By using a recursive Bayesian update method, terrain labels from multiple frames of images can be gradually fused into point cloud voxels, so that the local point cloud map not only has spatial geometric information, but also has stable terrain semantic labels. Compared with the method of assigning semantic labels to point clouds based on only a single frame image, this method can reduce the impact of single-frame misidentification on point cloud semantic results and improve the stability of the point cloud semantic layer.

[0052] S4. Extract terrain geometry information from the local point cloud map and construct a symbolic distance-volume mapping model to represent the local raster map; the terrain geometry features include at least terrain slope and terrain roughness; Specifically, after obtaining a local point cloud map with terrain semantic labels, terrain geometric information such as terrain slope and terrain roughness is extracted from the local point cloud map. Terrain slope is used to reflect the degree of inclination of the ground, and terrain roughness is used to reflect the flatness and undulation of the ground. Both can be used to evaluate the robot's passage risk in the corresponding area. In one implementation, the local plane of a point cloud voxel is fitted using principal component analysis. Specifically, for each point cloud voxel, a set of ground points within that voxel is selected, and the local plane normal vector of that set of points is calculated using principal component analysis; then, based on the angle between the local plane normal vector and the horizontal plane, the terrain slope of that point cloud voxel is calculated. The slope of the terrain is expressed as:

[0053] Terrain roughness is expressed as:

[0054] in, This represents the angle between the normal vector of the local plane and the horizontal plane. For voxels k One ground point, Centered at the geometric center of the voxeln This indicates the number of ground points within a point cloud voxel.

[0055] Furthermore, a symbolic distance-volume mapping model is constructed based on the extracted terrain geometry information to characterize the local raster map. Specifically, voxel information in the local point cloud map can be projected or mapped onto a 2D or 2.5D local raster map, enabling each raster cell in the local raster map to be associated with point cloud voxels within its corresponding spatial range. The symbolic distance-volume mapping model is used to express the distance relationships between raster cells and surrounding terrain surfaces, obstacles, or traversable areas, thus providing a basis for local path planning and obstacle avoidance.

[0056] In this embodiment, a GPU-based hash table voxel storage mechanism can also be used to store the correspondence between point cloud voxels and raster units. Through the hash table index, the voxel, semantic label, slope, roughness and comprehensive value corresponding to a certain spatial location can be quickly queried, thereby improving the efficiency of local raster map construction and updating.

[0057] S5. Store the terrain geometric features and terrain semantic features of each grid cell in the local grid map, and construct the grid cell comprehensive cost function based on the terrain geometric features and terrain semantic features. At the same time, construct the adaptive grid map resolution based on the importance of terrain semantics to update the local grid map. Furthermore, step S5 stores the geometric and semantic features of each raster cell, designs a raster cell comprehensive cost function, and constructs an adaptive raster map resolution based on terrain semantic importance, specifically including the following: First, store the geometric and semantic features of each raster cell. The raster cell synthesis cost is defined as follows:

[0058] in ( This is a highly accessible unit area. (This is an inaccessible unit area) , , , These represent terrain slope, terrain roughness, terrain semantic confidence, and dynamic obstacle weight coefficients, respectively. and These represent setting the maximum terrain slope and roughness thresholds, respectively. This represents the cost of semantic uncertainty in terrain representation. This represents the probability of the presence of a dynamic obstacle.

[0059] Secondly, an adaptive raster map resolution adjustment mechanism based on terrain semantic importance is constructed. Terrain importance is classified according to semantic information, maintaining high resolution for critical navigation areas such as roads and grasslands, while appropriately reducing resolution for non-critical areas such as gravel. Simultaneously, the map size should be positively correlated with the robot's movement speed. Based on this, a speed-semantic joint raster map resolution mapping model is constructed, as follows:

[0060] in, Scaling factor d res The current raster map resolution, d max , d min These represent the maximum and minimum resolutions of the raster map, respectively. v max These are the robot's maximum speed thresholds. For semantic importance factors (when in key regions, When in non-critical areas, ), This indicates the robot's current moving speed. This represents the velocity effect index, used to adjust for the nonlinear effect of velocity on resolution.

[0061] Then, a local grid map size fixing strategy is adopted, and the fixed size of the local grid map is determined based on the ground motion characteristics and obstacle avoidance mechanism of the mobile robot. The update interval displacement needs to be covered. Ground braking distance ( ) and error redundancy ( ),Right now To ensure that obstacle avoidance decisions have a sufficient safety buffer; Among them, the robot displacement within the map update interval satisfies

[0062] in, Set the robot map update interval; The robot's braking distance meets the following requirements:

[0063] in, a max This is the maximum braking line acceleration on the ground. To avoid obstacle response time, It is used to compensate for uncertainties caused by environmental interference such as wheel slippage and positioning errors.

[0064] Finally, a local grid map dynamic update strategy based on a sliding window is designed, and the number of sliding grids is dynamically calculated by combining the robot's ground motion directionality. ; During the local map update process, by parsing the association mechanism between the local map and the circular buffer, new observation grids are inserted along the robot's movement direction, and the oldest grid in the opposite direction is simultaneously overwritten to maintain a constant total number of grids. This achieves efficient memory management and effectively balances real-time updates of the grid map with operational safety.

[0065] S6. Construct a global topology map based on the distortion-corrected LiDAR frame and the robot's current odometry pose, establish a mapping mechanism between topology nodes and grid cells, and fuse the hierarchical information of the grid layer and the topology layer to construct a grid-topology hybrid navigation map.

[0066] Furthermore, the mechanism for establishing the mapping between topological nodes and grid cells described in step S6, and for fusing hierarchical information to construct a grid-topology hybrid navigation map, specifically includes the following: In the global topology map construction process, the denoised LiDAR point cloud is first fused with the robot's odometry pose to generate a global point cloud map with spatiotemporal consistency. Based on this, a node sampling strategy is employed, using key grid cells with significant geometric changes or semantic boundaries in the raster map as anchor points to establish topology nodes. Each topology node... Associate a set of grid cells And store the pose of the node. And its connectivity with neighboring nodes. The core of the mapping mechanism lies in assigning grid cells to the nearest and semantically compatible topological node through dual constraints of spatial distance and semantic consistency. The mapping function is defined as follows:

[0067] in, For grid cells The center coordinates, For topology nodes Location, For the semantic tags of the grid cells, Represents topology nodes The set of terrain semantic labels for the currently associated raster cells. is the semantic entropy function for a set.

[0068] After establishing the mapping relationship between topological nodes and grid cells, the hierarchical information of the grid layer and the topological layer is further integrated to construct a unified hybrid navigation map structure. The grid layer retains high-resolution terrain geometric features (slope, roughness) and semantic features (terrain category, passage cost) for local path planning and environmental interaction; the topological layer abstracts global connectivity through nodes and edges, compresses the map size, and supports global path search and task replanning.

[0069] To achieve collaborative optimization between the two layers, a hierarchical fusion cost function is defined, which maps the fine-grained passage cost of the raster layer to the edge weights of the topology layer, ensuring that the global path planning can reflect the real terrain constraints, as follows:

[0070] in, The edge weights between topological nodes. This is the set of grid cell paths that connect two nodes. Indicates the current grid cell The corresponding comprehensive value, This represents the step size of the grid cell along the path.

[0071] Through this fusion mechanism, the grid-topology hybrid navigation map retains the accurate representation of terrain details from the grid map while inheriting the compact structure and efficient planning capabilities of the topology map. In actual navigation, the robot can quickly complete global path search based on the topology layer and optimize local trajectory and avoid obstacles with the help of the grid layer, achieving a balance between global optimization and local real-time performance.

[0072] In summary, this embodiment extracts outdoor terrain semantic features using a large textual visual model and recursively fuses the terrain semantic results with a local point cloud map, giving the point cloud map both spatial structure and semantic labels. Furthermore, it constructs a local raster map by combining terrain geometric information such as slope and roughness with terrain semantic information, and utilizes a comprehensive cost function and adaptive resolution mechanism to improve the accuracy and update efficiency of the local map's navigability evaluation. Finally, through a mapping mechanism between topological nodes and raster cells, the navigability cost of the raster layer is mapped to the edge weights of the topological layer, thereby constructing a raster-topology hybrid navigation map that combines local fine-grained obstacle avoidance capabilities with global efficient planning capabilities.

[0073] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for constructing a large-model-driven outdoor hybrid navigation map for robots, characterized in that, Includes the following steps: S1. In an outdoor environment, point cloud data and image data of the current environment are collected by the lidar and camera carried by the mobile robot. S2. Construct a large-scale textual visual model of cross-modal terrain semantic information based on terrain text and image data, and verify and optimize the terrain semantic mask through the mask verification module to output the terrain segmentation result; S3. Based on point cloud data, a local point cloud map is constructed using the SLAM method. The terrain labels of multiple frames of images are iteratively fused using recursive Bayesian rules to update the visible and effective point cloud voxels within the camera's view frustum, thus obtaining a local point cloud map with terrain semantic labels. S4. Extract terrain geometry information from the local point cloud map and construct a symbolic distance-volume mapping model to represent the local raster map; The terrain geometry features include at least terrain slope and terrain roughness; S5. Store the terrain geometric features and terrain semantic features of each grid cell in the local grid map, and construct the grid cell comprehensive cost function based on the terrain geometric features and terrain semantic features. At the same time, construct the adaptive grid map resolution based on the importance of terrain semantics to update the local grid map. S6. Construct a global topology map based on the distortion-corrected LiDAR frame and the robot's current odometry pose, establish a mapping mechanism between topology nodes and grid cells, and fuse the hierarchical information of the grid layer and the topology layer to construct a grid-topology hybrid navigation map.

2. The method for constructing a large-model-driven outdoor hybrid navigation map for robots according to claim 1, characterized in that: In step S2, a large-scale textual visual model of cross-modal terrain semantic information is constructed based on the terrain text and the image data, and the terrain segmentation result is output through the mask verification module. Specifically, this includes the following steps: S201. Perform simple linear iterative clustering on the input original visual image to segment the original visual image into multiple homogeneous terrain blocks; S202. Based on the large text-image model, establish the text-image matching relationship between terrain text prompts and image patches, convert terrain text prompts into embedded features, and extract the visual features of image patches; S203. Generate a heat map based on the embedded features and visual features, and locate terrain semantic cue points based on the heat map to construct a cue point set; S204. Input the set of cue points into the semantic segmentation model to generate an initial semantic mask, and establish an effective mask filtering mechanism to filter the initial semantic mask. S205. The initial semantic mask is reconstructed and semantically optimized through the mask verification module, and the final terrain segmentation result is output.

3. The method for constructing a large-model-driven outdoor hybrid navigation map for robots according to claim 2, characterized in that: The set of prompt points is represented as follows: ； in, Indicates the first i The location of a terrain semantic cue point in the image. Indicates the first i Semantic labels corresponding to each terrain semantic cue point; The mask verification module is a mask verification module with a Transformer encoder-decoder structure, and it fuses visual features and mask semantics through a multi-head attention mechanism.

4. A method for constructing a large-model-driven outdoor hybrid navigation map for robots according to claim 2 or 3, characterized in that: The large-scale text vision model is trained using a joint loss function, which is: ； in, The cross-entropy loss is obtained by weighting the semantic similarity matrix of terrain categories. The Dice loss is obtained by weighting the semantic importance of terrain categories. Loss due to spatial continuity constraints; , , These are the weighting coefficients for the corresponding loss terms.

5. The method for constructing a large-model-driven outdoor hybrid navigation map for robots according to claim 1, characterized in that: In step S3, the terrain labels of multiple frames of images are iteratively fused using recursive Bayesian rules to update the visible and effective point cloud voxels within the camera's view frustum. This specifically includes the following steps: S301. Perform dynamic voxel grid segmentation on the local point cloud map to reduce the size of the original point cloud data and filter out redundant information. S302. Filter out overhanging structures from the local point cloud map and extract the effective surface point cloud; S303. A multi-resolution point cloud voxel fitting strategy is adopted so that each point cloud voxel stores the corresponding discrete probability distribution and semantic label set. S304. Project the point cloud voxels onto the current image to obtain the pixel coordinates of the point cloud voxels in the current image; S305. Based on the semantic labels observed at the corresponding pixel coordinates in the current image, recursively update the semantic label probabilities of the point cloud voxels. The voxel dimensions used in dynamic voxel mesh segmentation include voxel length. voxel width and voxel height And satisfy: ； in, Indicates the length of the robot body. Indicates the width of the robot body. This indicates the maximum slope the robot can traverse.

6. The method for constructing a large-model-driven outdoor hybrid navigation map for robots according to claim 5, characterized in that: In step S3, the recursive Bayes rule is expressed as: ； in, For image sequences, This is a local point cloud map, where Z is a normalization constant. For posterior probability, For prior probability, To make voxels Projected onto the current image The pixel coordinates obtained above, In the image Corresponding pixel position Observed semantic tags.

7. The method for constructing a large-model-driven outdoor hybrid navigation map for robots according to claim 1, characterized in that: In step S4, extracting terrain geometry information from the local point cloud map specifically includes the following steps: S401. Fit the local plane of point cloud voxels using principal component analysis; S402. Calculate the terrain slope of each point cloud voxel based on the angle between the normal vector of the local plane and the horizontal plane. S403. Calculate the terrain roughness of each point cloud voxel based on the distance between each surface point within the point cloud voxel and the geometric center of the point cloud voxel. The slope of the terrain is expressed as: ； Terrain roughness is expressed as: ； in, This represents the angle between the normal vector of the local plane and the horizontal plane. For voxels k One ground point, Centered at the geometric center of the voxel n This indicates the number of ground points within a point cloud voxel.

8. The method for constructing a large-model-driven outdoor hybrid navigation map for robots according to claim 1, characterized in that: In step S5, the grid cell comprehensive cost function is constructed based on terrain slope, terrain roughness, terrain semantic confidence, and the probability of dynamic obstacle presence. The comprehensive cost value of the grid cell is expressed as: ； in, ; This is a highly accessible unit area. This is a non-passable unit area; , , , These represent terrain slope, terrain roughness, terrain semantic confidence, and dynamic obstacle weight coefficients, respectively. and These represent setting the maximum terrain slope and roughness thresholds, respectively. This represents the cost of semantic uncertainty in terrain representation; This represents the probability of the presence of a dynamic obstacle.

9. The method for constructing a large-model-driven outdoor hybrid navigation map for robots according to claim 1, characterized in that: In step S5, an adaptive raster map resolution based on terrain semantic importance is constructed, and the local raster map is dynamically updated. This specifically includes the following steps: S501. Classify terrain importance levels based on semantic information; S502: Maintain high resolution for critical navigation areas such as roads and grasslands, and reduce resolution for non-critical areas; S503. Combining the robot's current moving speed, the robot's maximum speed threshold, and semantic importance factors, a speed-semantic joint grid map resolution mapping model is constructed. The velocity-semantic joint raster map resolution mapping model is represented as follows: ； in, Scaling factor d res The current raster map resolution, d max , d min These represent the maximum and minimum resolutions of the raster map, respectively. v max The maximum speed threshold for the robot. As a semantic importance factor, This indicates the robot's current moving speed. Indicates the influence of speed; S504. Use a local raster map size fixing strategy to determine the fixed size of the local raster map; The fixed size of the local raster map satisfies: ； in, This indicates a fixed size for a local raster map. This indicates the robot's displacement within the map update interval. Indicates the robot's braking distance. Indicates the error redundancy distance; The robot displacement within the map update interval satisfies: ； The robot's braking distance meets the following requirements: ； in, Indicates the robot map update interval. a max This represents the maximum braking line acceleration on the ground. Indicates the obstacle avoidance response time; S505. A sliding window update strategy is adopted to dynamically calculate the number of sliding grid cells; The number of sliding grids satisfies: ； S506. During the local grid map update process, insert new observation grids along the robot's movement direction and simultaneously cover the old grids in the opposite direction to maintain a constant total number of grids.

10. The method for constructing a large-model-driven outdoor hybrid navigation map for robots according to claim 1, characterized in that: In step S6, a mapping mechanism between topology nodes and grid cells is established, and the hierarchical information of the grid layer and the topology layer is fused. This specifically includes the following steps: S601. The denoised LiDAR point cloud is fused with the robot odometry pose to generate a global point cloud map with spatiotemporal consistency. S602. Establish topological nodes using key raster cells with significant geometric changes and / or semantic boundaries in the raster map as anchor points; S603. Associate each topology node with a set of grid cells and store the pose of the topology node and the connectivity relationship with adjacent topology nodes. S604. By using spatial distance constraints and semantic consistency constraints, the grid cells are assigned to the corresponding topology nodes; The mapping relationship between topology nodes and grid cells is expressed as follows: ； in, For grid cells The center coordinates, For topology nodes Location, For the semantic tags of the grid cells, Represents topology nodes The set of terrain semantic labels for the currently associated raster cells. The semantic entropy function of a set; S605. Map the total cost of the grid cells in the grid layer to the topological edge weights in the topology layer. The edge weights between topological nodes are represented as follows: ； in, The edge weights between topological nodes. This is the set of grid cell paths that connect two nodes. Indicates the current grid cell The corresponding comprehensive value, This represents the step size of the grid cell along the path.

Images