Robot navigation methods and systems with enhanced semantic maps

Abstract

This application belongs to the field of navigation and positioning, specifically disclosing a robot navigation method and system with enhanced semantic maps. This application extracts geometric and semantic information from pre-stored GIS map files to generate GIS data, providing a structured foundation with both global geographic information and semantic annotations for subsequent processing. Then, through hierarchical mapping and feature matching, the GIS data is aligned with the SLAM map, achieving a unified association between local geometric information and global geographic information, overcoming the limitation of separation between the two in traditional navigation maps. Based on this, a fused semantic map is generated by dynamically associating semantic information during navigation, enabling the robot to use environmental semantics for perception and reasoning in real time. Finally, path planning and decision control are performed based on this fused semantic map, significantly reducing the burden on the robot to repeatedly build maps using a single SLAM, and improving navigation efficiency and environmental adaptability.

Description

Technical Field

[0001] This application belongs to the field of navigation and positioning technology, specifically relating to a robot navigation method and system with enhanced semantic maps. Background Technology

[0002] Currently, robot navigation systems based on Simultaneous Localization and Mapping (SLAM) face two core challenges in outdoor environments. First, there is a lack of global reference. Traditional SLAM-generated semantic maps only contain local geometric features such as point clouds and grids, lacking connections to global geographic information such as road names and building functions. This makes it difficult for robots to achieve efficient path planning and semantic interaction in large-scale scenes. Second, outdoor mapping is inefficient. Outdoor environments are dynamic and large-scale; relying on SLAM for real-time mapping requires processing massive amounts of sensor data, resulting in high computational costs and susceptibility to weather and lighting conditions, leading to insufficient map accuracy and robustness. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the purpose of this application is to provide a robot navigation method and system with enhanced semantic maps, aiming to solve the problem that existing robot navigation maps cannot take into account both local geometry and global geographic information, resulting in a heavy burden on robot navigation mapping.

[0004] The first aspect of this application relates to a robot navigation method with enhanced semantic maps, comprising: extracting geometric and semantic information based on a pre-stored GIS map file, and generating GIS data based on the geometric and semantic information; generating a GIS map structure aligned with a SLAM map based on the GIS data through hierarchical mapping and feature matching; generating a fused semantic map by dynamically associating semantic information with the current SLAM map based on the GIS map structure when the robot performs a navigation task; and performing path planning and decision control based on the fused semantic map.

[0005] In one embodiment, the GIS data includes: a geometric topology and an attribute table; extracting geometric and semantic information based on a pre-stored GIS map file, and generating GIS data based on the geometric and semantic information, includes: parsing vector and raster data of the pre-stored GIS map file using an open-source library; extracting geometric features from the pre-stored GIS map file based on the vector data and performing topology optimization to generate a geometric topology; extracting semantic information of navigable areas based on the raster data using a semantic segmentation model; and generating an attribute table corresponding to the geometric features and semantic information based on the semantic information of the navigable areas and the geometric topology.

[0006] In one embodiment, a GIS map structure aligned with a SLAM map is generated based on GIS data through hierarchical mapping and feature matching. This includes: converting the geometric topology into a topological map through hierarchical mapping based on GIS data, and forming a geometric layer from the topological map; mapping the original labels in the attribute table to standard semantic labels for robot navigation through hierarchical mapping based on GIS data, and forming a semantic layer from the standard semantic labels; generating a GIS map structure by associating the semantic layer and the geometric layer based on the attribute table; and aligning the GIS map structure with the SLAM map in the coordinate system using a feature matching algorithm.

[0007] In one embodiment, when the robot performs a navigation task, semantic information is dynamically associated with the current SLAM map based on the GIS map structure to generate a fused semantic map. This includes: continuously updating the SLAM local map while the robot performs the navigation task; dynamically associating semantic information with the current SLAM map based on the alignment between the GIS map structure and the current SLAM map to generate a current semantic map; retaining the non-overlapping SLAM local map when there are non-overlapping parts between the GIS map structure and the current SLAM map; and generating a fused semantic map based on the current semantic map and the non-overlapping SLAM local map.

[0008] In one embodiment, when the robot performs a navigation task, based on the GIS map structure, semantic information is dynamically associated with the current SLAM map to generate a fused semantic map. The method further includes: verifying the fused semantic map using a random sampling consensus algorithm to remove abnormal data points; detecting in real time whether the GIS map structure is updated; and after the GIS map structure is updated, dynamically associating semantic information with the current SLAM map based on the current GIS map structure to generate a fused semantic map, thereby updating the fused semantic map.

[0009] In one embodiment, path planning and decision control based on a fused semantic map includes: parsing traffic constraints in semantic tags according to the semantic rules of the fused semantic map; calculating the comprehensive cost of multiple path segments based on the traffic constraints, the cost including distance weight and semantic weight; sorting and filtering the multiple path segments according to the comprehensive cost to generate a candidate path set; selecting the optimal path from the candidate path set by combining real-time environmental perception data; executing motion control according to the optimal path, and continuously updating semantic information during the operation.

[0010] In one embodiment, geometric and semantic information is extracted from a pre-stored GIS map file, and GIS data is generated based on the geometric and semantic information. Prior to this, the method further includes: determining the robot's current working area; loading the corresponding pre-stored GIS map file from the geographic information system database on demand based on the current working area, and performing format verification and integrity checks on the pre-stored GIS map file; and performing SLAM-based navigation when the format verification or integrity check of the pre-stored GIS map file fails.

[0011] The second aspect of this application relates to a robot navigation system with enhanced semantic maps, comprising:

[0012] The data acquisition module is used to extract geometric and semantic information from pre-stored GIS map files, and generate GIS data based on the geometric and semantic information.

[0013] The data processing module is used to generate a GIS map structure aligned with the SLAM map based on GIS data through hierarchical mapping and feature matching.

[0014] The data fusion module is used to dynamically associate semantic information with the current SLAM map based on the GIS map structure when the robot is performing navigation tasks, and generate a fused semantic map.

[0015] The navigation control module is used for path planning and decision control based on the fused semantic map.

[0016] Thirdly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0017] Fourthly, this application provides a computer program product that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0018] It is understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect, and will not be repeated here.

[0019] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art:

[0020] This application extracts geometric and semantic information from pre-stored GIS map files to generate GIS data, providing a structured foundation with both global geographic information and semantic annotation for subsequent processing. Then, through hierarchical mapping and feature matching, the GIS data is aligned with the SLAM map, achieving a unified association between local geometric information and global geographic information, overcoming the limitation of separation between the two in traditional navigation maps. Based on this, a fused semantic map is generated by dynamically associating semantic information during navigation, enabling the robot to use environmental semantics for perception and reasoning in real time. Finally, path planning and decision control are performed based on this fused semantic map.

[0021] Compared with existing technologies, this significantly reduces the burden on robots that rely on repeated mapping using a single SLAM, and improves navigation efficiency and environmental adaptability. Attached Figure Description

[0022] Figure 1 This is one of the flowcharts illustrating the robot navigation method with enhanced semantic maps provided in this application embodiment;

[0023] Figure 2 This is a second schematic flowchart of the robot navigation method with enhanced semantic map provided in the embodiments of this application;

[0024] Figure 3 This is the third flowchart illustrating the robot navigation method with enhanced semantic maps provided in this application embodiment;

[0025] Figure 4 This is the fourth flowchart illustrating the robot navigation method with enhanced semantic maps provided in this application embodiment;

[0026] Figure 5 This is a schematic diagram of the structure of the robot navigation system with enhanced semantic map provided in the embodiments of this application.

[0027] In all the accompanying drawings, the same reference numerals are used to denote the same elements or structures, wherein: 01 is the data acquisition module; 02 is the data processing module; 03 is the data fusion module; and 04 is the navigation control module. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0029] In this application, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone. In this application, the symbol " / " indicates that the related objects are in an "or" relationship, for example, A / B means A or B.

[0030] In this application, the terms "first" and "second," etc., are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0031] In this application, the term "electrical connection" can refer to a direct circuit connection or a signal transmission via a communication protocol.

[0032] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as superior or more advantageous than other embodiments or designs. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0033] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0034] Currently, in the field of robot navigation, although pure SLAM solutions (such as ORB-SLAM3) can build geometric maps, semantic information needs to be manually labeled or extracted by deep learning models in the later stage, which is costly and has poor scalability.

[0035] Based on this, this application proposes a robot navigation method with enhanced semantic maps. Please refer to [link / reference]. Figure 1 , Figure 1 This is one of the flowcharts illustrating the robot navigation method with enhanced semantic maps provided in this application embodiment. In this embodiment, the robot navigation method with enhanced semantic maps includes steps S10 to S40.

[0036] Step S10: Extract geometric and semantic information from the pre-stored GIS map file, and generate GIS data based on the geometric and semantic information.

[0037] It should be noted that pre-stored GIS map files refer to geographic information system map files that are stored in advance on a computer system or other storage devices, such as the geographic information system map files in the robot device of this application. Specifically, this pre-storage can be achieved by the user actively transferring the GIS map file to the robot device, such as through physical means like a data cable or USB flash drive, or by remote import through specific software or platforms; alternatively, the robot device can automatically download the required GIS map file from a designated server or data source according to a preset program or instruction when connected to the network.

[0038] It should be noted that geometric information refers to the spatial feature data of various geographic elements on a map, such as the shape, size, and location of roads, buildings, and bodies of water. Extracting geometric information means identifying and obtaining this spatial feature data from pre-stored GIS map files. Semantic information is descriptive or interpretive information about geographic elements, such as the names of roads and the uses of buildings. Extracting semantic information means obtaining this descriptive content from map files.

[0039] Understandably, generating GIS data based on geometric and semantic information means integrating and processing the previously extracted geometric and semantic information to generate a complete dataset that can be used for analysis, visualization, and application in geographic information systems. GIS data generated from geometric and semantic information may have different forms or contents depending on different emphases.

[0040] It should be noted that, for navigation purposes, the GIS data in this application must include at least a geometric topology and an attribute table. The geometric topology primarily describes the spatial relationships between geographic features, such as the connection methods between roads and the location relationships of intersections. It helps the navigation system accurately determine the accessibility and route direction between different locations. The attribute table is a structured storage of the semantic information of geographic features, presenting the relevant semantic information of each geographic feature in tabular form, such as road names, lengths, and speed limits. The navigation system can obtain more detailed geographic feature information by querying the attribute table, thereby providing users with more accurate and comprehensive navigation services.

[0041] In one feasible implementation, the specific steps of step S10 are given. Please refer to... Figure 2 Step S10 includes steps S11 to S14.

[0042] Step S11: Parse the vector and raster data of the pre-stored GIS map file using an open-source library.

[0043] Understandably, pre-stored GIS map files contain a large amount of vector and raster data. Vector data represents the location and shape of geographic features using geometric shapes such as points, lines, and polygons. Vector data has advantages such as high accuracy and small data volume, making it suitable for representing discrete geographic features, such as roads and buildings. Raster data represents geospatial information in a regular grid format. Each grid cell (pixel) contains a value representing a certain attribute of that location, such as elevation or temperature. Raster data is suitable for representing continuously changing geographic phenomena, such as topography and climate.

[0044] Understandably, parsing the vector and raster data of pre-stored GIS map files using open-source libraries means utilizing the tools and functions provided by these libraries to read, convert, and process the vector and raster data stored in the GIS map files for subsequent analysis, visualization, and application. In the GIS field, open-source libraries such as the Geospatial Data Abstraction Library (GDAL) and ORG Simple Feature Library provide rich interfaces for processing various geospatial data.

[0045] Step S12: Based on vector data, extract geometric features from the pre-stored GIS map file and perform topology optimization to generate a geometric topology structure.

[0046] Understandably, extracting geometric features from a pre-stored GIS map file involves using specific algorithms or tools to separate and identify geometric shapes—that is, geometric features, such as points, lines, and polygons—from the vector data contained in the pre-stored GIS map file, emanating from the complex data structure. For example, extracting all line features representing roads from a map file containing various geographic information about a city.

[0047] Specifically, GIS files can be accessed programmatically, and specific geographic feature classes can be identified based on attribute filtering or layer names. For example, all line features representing roads or polygon features representing land parcels can be separated from complex map files, and their geometry and attribute information can be read into memory. However, these raw features often have topological defects due to digitization errors or inconsistent data sources, such as inaccurate road lines at intersections, overlapping boundaries of adjacent polygons, or gaps. Therefore, rigorous topology optimization is necessary.

[0048] Understandably, topology optimization is an important process performed on extracted geometric features. In geographic information, topological relationships describe the spatial connectivity and adjacency relationships between geographic features. Topology optimization analyzes and adjusts these topological relationships between geometric features to ensure they conform to the actual situation and logic of geographic space. For example, it ensures that road endpoints are correctly connected and adjacent area boundaries are accurately aligned, avoiding errors or unreasonable topological relationships between geometric features, thereby improving the accuracy and reliability of the data.

[0049] Specifically, topology optimization can be achieved by setting tolerance thresholds and topology rules, such as no hanging nodes, no overlaps, and shared boundaries. Engines like GEOS, JTS, or ArcGIS topology toolsets can be used to perform operations such as snapping, gap filling, and overlap removal on geometric objects, ensuring spatial logical consistency between features. For example, vertex snapping will automatically connect road intersections within the tolerance range, eliminating hanging phenomena. Ultimately, the optimized geometric data is explicitly constructed as a topological structure, where the adjacency, connection, and containment relationships between nodes, edges, and faces are fully recorded. For instance, an edge can simultaneously serve as the shared boundary of two polygons, thereby generating an efficient data model that supports network analysis, overlay analysis, and spatial queries, providing a reliable foundation for subsequent geographic computation and decision analysis.

[0050] Understandably, generating the final geometric topology means integrating and organizing the processed geometric elements and their correct topological relationships after the previous extraction and topology optimization operations, forming a data structure that can clearly express the spatial location and interrelationships of geographic elements.

[0051] Step S13: Based on the raster data, extract the semantic information of the navigable area using a semantic segmentation model.

[0052] It's important to note that semantic segmentation is a crucial task in computer vision and image processing. Its goal is to assign a specific category label to each pixel in an image, enabling a fine-grained understanding and differentiation of image content. Semantic segmentation models are deep learning models trained on large amounts of labeled data. They automatically learn the features and patterns of different objects in an image and classify each pixel based on these features. For example, in a remote sensing image containing various features such as roads, buildings, and trees, a semantic segmentation model can label road pixels as "road," building pixels as "building," and so on. In this step, the semantic segmentation model is a key tool for extracting semantic information from navigable regions.

[0053] As we can understand, a navigable area refers to a region in a geographical environment that robots can safely and smoothly traverse, such as roads and sidewalks. By inputting raster data into a pre-trained semantic segmentation model, the model analyzes and judges each pixel in the raster data. Based on the learned features and patterns, it marks pixels belonging to navigable areas and assigns them corresponding semantic labels, such as "road" or "sidewalk." In this way, we extract semantic information about navigable areas from the original raster data. This semantic information clearly tells us which areas are navigable, providing an important basis for subsequent navigation tasks.

[0054] Step S14: Based on the semantic information and geometric topology of the navigable area, generate an attribute table corresponding to the geometric features and semantic information.

[0055] Understandably, in this step, generating the attribute table involves deeply integrating and organizing the semantic information and geometric topology of the navigable area. Specifically, for each geometric element in the geometric topology, its navigable area category (e.g., whether it's a "main road" or a "side road") is determined by combining the semantic information of the navigable area. Simultaneously, other geometric attributes of the element, such as length, area, and coordinate position, can be obtained from the geometric topology. Organizing this information into a table according to certain rules and formats forms the attribute table. This generated attribute table plays a crucial role. It allows information originally scattered across the semantic information and geometric topology of the navigable area to be stored centrally and systematically, facilitating subsequent querying, analysis, and processing.

[0056] In one feasible implementation, the steps for obtaining the pre-stored GIS map file before step S10 are given: determine the robot's current working area; based on the current working area, load the pre-stored GIS map file of the corresponding area from the geographic information system database as needed, and perform format verification and integrity check on the pre-stored GIS map file; if the pre-stored GIS map file fails the format verification or integrity check, execute SLAM-based map navigation.

[0057] Understandably, the system first determines the robot's current working area, typically by parsing its task instructions, receiving user input, or obtaining real-time pose information from the positioning module. Subsequently, based on this area, the system loads pre-stored GIS map files for the corresponding region from a remote or local geographic information system database as needed. This process aims to efficiently transfer the minimum necessary data to save storage space and improve processing speed.

[0058] Understandably, upon successful loading, the system immediately performs rigorous format validation on the file, checking file type, structural integrity, coordinate system consistency, and content integrity to ensure that key layers, attribute fields, and geometric data are complete. If both validation and checks pass smoothly, the file is confirmed as a valid data source and sent to subsequent step S10 for processing. Conversely, if the pre-stored GIS map file fails either the format validation or integrity check, the system will determine that the traditional geographic data is unreliable or unusable and automatically trigger a fault-tolerance mechanism, switching to a SLAM-based real-time map building and navigation scheme to ensure that the robot can maintain its autonomous operation capability even in the absence of a prior map or when the prior map is invalid.

[0059] Step S20: Based on GIS data, generate a GIS map structure aligned with the SLAM map through hierarchical mapping and feature matching.

[0060] It's important to note that layered mapping refers to processing GIS data into layers based on different features or attributes. Each layer represents a specific type of geographic information, such as a road layer, a building layer, or a vegetation layer. This layering approach facilitates the classification, management, and efficient processing of complex geographic data. Through layered mapping, different elements in GIS data can be categorized into their corresponding layers, making the data structure clearer and facilitating subsequent feature extraction and matching operations. For example, when processing GIS data of urban environments, treating buildings as a separate layer allows for a greater focus on their geometric features and spatial distribution, without interference from other geographic elements.

[0061] It's important to note that feature matching refers to finding geographic features with similar characteristics in map data from different data sources or with different representations, and then matching them. In this step, we need to match features in the GIS data, after hierarchical mapping, with those in the SLAM map. SLAM maps are generated by processing data collected in real time from environmental sensors, such as LiDAR and cameras, reflecting the environmental information perceived by robots or mobile devices during movement. The feature matching process typically involves extracting key features from both the GIS data and the SLAM map, such as corners, edges, and specific shapes, and then using algorithms such as nearest neighbor search and feature descriptor matching to determine the correspondence between these features. Through feature matching, geographic features in the GIS data can be associated with actual environmental features in the SLAM map, thereby aligning the two map data and resolving coordinate system differences.

[0062] Understandably, after completing the hierarchical mapping and feature matching, the GIS data is adjusted and optimized based on the matching results to ensure consistency with the SLAM map in spatial location and geometry. This means that the geographic features in the GIS map will accurately correspond to the actual environment reflected in the SLAM map, generating a map structure that contains rich geographic semantic information (from GIS data) and is precisely aligned with the real-time perceived environment (SLAM map).

[0063] In one feasible implementation, the specific steps of step S20 are given. Please refer to... Figure 3 Step S20 includes steps S21 to S24.

[0064] Step S21: Based on GIS data, the geometric topology is transformed into a topological map through hierarchical mapping, and the topological map is used to form a geometric layer.

[0065] Understandably, layered mapping based on GIS data involves transforming the geometric topology. The goal of this transformation is to convert the geometric topology into a "topological map" format more suitable for subsequent processing and applications. A topological map focuses more on expressing the topological relationships between geographic features than on specific geometric shapes and location coordinates. For example, in a topological map, a road might be simplified to a line segment, while the emphasis is on highlighting the connections between this road and other roads and how they are connected.

[0066] Understandably, after the geometric topology is transformed into a topological map through layered mapping, these topological maps are combined according to pre-defined layering rules to form a geometric layer. The geometric layer is a crucial layer in the overall map structure, providing fundamental topological information for subsequent map analysis, path planning, and other applications. For example, vector data can be converted into a robot-usable topological map, such as a weighted graph structure where edges represent paths and weights include distance and traversal difficulty. For isometric data such as building outlines, simplified polygons are generated and associated with semantic labels; the geometric layer can use a compact raster or vector format.

[0067] Step S22: Based on GIS data, the original labels in the attribute table are mapped to standard semantic labels for robot navigation through hierarchical mapping, and the semantic layer is composed of the standard semantic labels.

[0068] It's important to note that raw labels are the initial tag information in the attribute table used to describe the characteristics of geographic elements. They are the most direct records of geographic elements, but due to differences in sources and standards, they may have inconsistencies in expression or fail to meet specific application requirements. For example, in the attribute table, for an area accessible to pedestrians, there might be multiple raw labels such as "sidewalk," "pedestrian street," and "plaza pedestrian area." While these labels express similar semantics, they lack uniformity in form, hindering consistent processing and understanding by robot navigation systems. Standard semantic labels for robot navigation, on the other hand, are a unified and standardized label system developed based on the needs of robot navigation and industry standards. These labels have clear definitions and semantics, accurately describing the meaning of geographic elements in robot navigation. For example, "accessible area," "obstacle," "charging station," and "target location" are common standard semantic labels for robot navigation. By mapping the raw labels in the attribute table to these standard semantic labels, robots can better understand the geographic environment, thereby achieving smarter and safer navigation.

[0069] Understandably, in this step, the mapping process involves converting the original labels in the attribute table into corresponding standard semantic labels for robot navigation based on pre-established rules. For example, the original label "sidewalk" is mapped to "accessible area (pedestrian)," and "building" is mapped to "obstacle." Through this mapping operation, the data can be made more compatible with the requirements of the robot navigation system, improving the system's compatibility and accuracy.

[0070] Understandably, the semantic layer is a hierarchical structure composed of mapped standard semantic labels for robot navigation. In the semantic layer, geographic features are no longer merely objects with geometric shapes and locations, but are endowed with actual semantic meanings. For example, the robot can learn through the semantic layer which areas are safe to traverse, which areas contain obstacles to avoid, and the locations of important facilities such as charging stations. The semantic layer works in conjunction with the geometric layer; the geometric layer provides the geometric topology information of geographic features, while the semantic layer imbues these geometric features with actual semantics, jointly providing comprehensive and accurate information support for robot navigation. For example, attribute tables can be encoded into lightweight semantic knowledge graphs, such as RDF triples, for instance, A (BuildingA, has_function, hospital). It can also support dynamically loading semantic rules, such as "speed limit 5km / h in hospital area." The semantic layer uses lightweight databases, such as SQLite, or graph databases, such as Neo4j, to store attribute relationships.

[0071] Step S23: Generate a GIS map structure based on the semantic and geometric layers associated with the attribute table.

[0072] Step S24: Align the GIS map structure with the SLAM map in coordinate system using a feature matching algorithm.

[0073] Understandably, based on the geographic feature association information contained in the attribute table, the semantic layer and the geometric layer are closely linked, so that the semantic information corresponds to the geometric information, and finally a complete GIS map structure is generated. This structure contains both the geometric features of geographic features and the semantic understanding that meets the needs of robot navigation.

[0074] Understandably, feature matching algorithms extract key features from the GIS map structure and SLAM map, and then find the correspondences between these features. Based on these correspondences, the coordinate systems of the GIS map structure and SLAM map are adjusted and transformed to make them consistent in spatial location, thus achieving coordinate system alignment.

[0075] Step S30: When the robot is performing a navigation task, based on the GIS map structure, the semantic information is dynamically associated with the current SLAM map to generate a fused semantic map.

[0076] Understandably, when robots perform navigation tasks, the environment they encounter is dynamically changing, requiring a precise understanding of environmental information to plan paths, avoid obstacles, and reach the target location. GIS maps store rich geographic semantic information, such as which areas are passable and where charging stations are located, but they are relatively static and based on pre-built geographic data. SLAM maps, on the other hand, are built in real-time by the robot during its movement, reflecting the actual environmental situation perceived by the robot at that moment, but lack detailed semantic annotations. Therefore, combining the two, dynamically associating semantic information, and generating a fused semantic map is crucial for efficient and safe robot navigation.

[0077] It should be noted that dynamically associated semantic information means associating the attribute information of the GIS map structure with the corresponding entities in the SLAM map. For example, a road detected by SLAM corresponds to the road rule "main road + speed limit 60km / h" in the GIS map structure.

[0078] Understandably, the fused semantic map not only includes the real-time environmental geometric information reflected in the current SLAM map, such as the location and shape of obstacles and the extent of passable areas, but also incorporates rich semantic information from the GIS map structure. This balances local geometry and global geographic information while significantly reducing the computational burden.

[0079] In one feasible implementation, the specific steps of step S30 are given. Please refer to... Figure 4 Step S30 includes steps S31 to S33.

[0080] Step S31: When the robot performs navigation tasks, it continuously updates the SLAM local map. Based on the alignment between the GIS map structure and the current SLAM map, it dynamically associates semantic information with the current SLAM map to generate the current semantic map.

[0081] Understandably, during the dynamic process of a robot performing navigation tasks, the environment may change at any time, such as the appearance of new obstacles or the movement of existing objects. Therefore, continuously updating the SLAM local map is essential. The SLAM local map is a map constructed by the robot within its current local area through its own sensors, perceiving the environment in real time. Continuous updates ensure that it accurately reflects the latest environmental conditions surrounding the robot. Meanwhile, the GIS map structure stores comprehensive and pre-processed geographic semantic and geometric information. Based on the areas where the GIS map structure and the current SLAM map are aligned—that is, the regions where they correspond in spatial location and geographic features—the rich semantic information in the GIS map structure, such as "passable areas," "obstacles," and "specific target locations," is dynamically associated with the corresponding locations in the current SLAM map. Therefore, the current SLAM map not only has real-time geometric information but also semantic understanding capabilities, thus generating a current semantic map. This provides the foundation for more intelligent robot navigation; for example, the robot can determine which areas are safe to pass through and which should be avoided based on semantic information.

[0082] Step S32: When there are parts of the GIS map structure that do not overlap with the current SLAM map, retain the non-overlapping SLAM local map.

[0083] Step S33: Generate a fused semantic map based on the current semantic map and the non-overlapping SLAM local maps.

[0084] Understandably, when faced with situations not covered by the GIS map structure but detected in real-time by the robot using SLAM technology, the non-overlapping SLAM local map is of significant value. It reflects new environmental information actually perceived by the robot but not in the GIS map, such as temporarily appearing obstacles or newly opened paths. Retaining this non-overlapping SLAM local map ensures that the robot doesn't miss crucial environmental information, providing complete data support for subsequent more comprehensive and accurate map construction and path planning, and preventing navigation errors due to missing information.

[0085] It should be noted that fusing these two elements integrates known semantic information with the latest environmental detection data. The resulting fused semantic map possesses both semantic understanding of known areas and reflects the latest dynamic changes in the environment. Based on this comprehensive and accurate fused semantic map, robots can plan navigation paths more rationally, such as avoiding newly appearing obstacles and choosing better routes, effectively improving navigation efficiency and safety, and better adapting to complex and ever-changing real-world navigation environments.

[0086] In one feasible implementation, the fused semantic map can be further optimized after step S30.

[0087] Specifically, a random sampling consensus algorithm can be used to verify the fused semantic map and remove abnormal data points; the GIS map structure can be detected in real time; after the GIS map structure is updated, when the robot performs navigation tasks, the fused semantic map is updated by dynamically associating semantic information with the current SLAM map based on the current GIS map structure.

[0088] It's important to note that during the construction of the fused semantic map, various factors such as sensor errors and environmental interference may introduce anomalous data points. These anomalous data points distort the environmental information presented by the map, thus affecting the accuracy of the robot's navigation decisions. The Random Sampling Consensus (RANSAC) algorithm is a powerful data filtering tool. It selects a subset of data through multiple random samplings, constructs a data model that conforms to the characteristics of the majority of data points, and then counts the number of data points that fit this model. After multiple iterations, the model with the highest support rate is determined as the optimal model, while data points that deviate significantly from the optimal model are identified as anomalous data points and removed. By using the RANSAC algorithm to validate the fused semantic map, anomalous data can be effectively removed, making the environment depicted by the map more realistic and reliable, and providing the robot with more accurate navigation guidance.

[0089] It's important to note that GIS map structures are not static. Over time, the geographical environment may change, such as with new buildings, road modifications, or adjustments to regional functions. To ensure the fused semantic map remains consistent with the actual geographical environment, it's necessary to monitor GIS map structure updates in real time. This can be achieved by establishing a real-time connection with the GIS map data source, periodically querying map data update timestamps, or receiving update notifications proactively pushed by the GIS map management system. Once an update to the GIS map structure is detected, it means that the geographic semantic information or geometric topology within the map may have changed, requiring timely adjustments to the fused semantic map.

[0090] It's important to note that once the GIS map structure is confirmed to be updated, to ensure the fused semantic map reflects the latest geographic information, the process of dynamically associating semantic information with the current SLAM map based on the updated GIS map structure needs to be re-executed during the robot's navigation task. Specifically, the robot will perform feature matching and semantic association again based on the semantic labels and geometric information in the updated GIS map structure, combined with the currently constructed SLAM map. This accurately maps the newly added semantic information and modified geographic feature attributes from the GIS map structure to the corresponding locations on the current SLAM map, while retaining the real-time dynamic environmental change information reflected in the SLAM map. Through this process, an updated fused semantic map is generated, ensuring that the robot always navigates based on the latest and most accurate map information, effectively responding to changes in the geographic environment and improving the reliability and adaptability of navigation.

[0091] Step S40: Perform path planning and decision control based on the fused semantic map.

[0092] Understandably, navigation tasks can be achieved by relying on fused semantic maps. First, the navigation goal must be clearly defined. Then, the environment is comprehensively analyzed using the map, and an optimal path from the starting point to the target point is planned using appropriate algorithms. As the robot moves along the path, it continuously monitors the environment and its own state in real time through sensors. Based on the semantic information of the fused semantic map, it assesses the impact and makes decisions based on preset rules, such as deciding to detour or wait when encountering obstacles. The control system then translates these decisions into specific actions. This process enables the robot to navigate autonomously and intelligently in complex environments, ensuring its efficient operation in scenarios such as industrial production, logistics distribution, and smart homes, and promoting the widespread application and development of robotics technology.

[0093] In one feasible implementation, the specific steps of step S40 are given.

[0094] Specifically, step S40 includes: parsing the traffic constraints in the semantic tags according to the semantic rules of the fused semantic map; calculating the comprehensive cost value of multiple path segments based on the traffic constraints, the cost value including distance weight and semantic weight; sorting and filtering the multiple path segments according to the comprehensive cost value to generate a candidate path set; selecting the optimal path from the candidate path set by combining real-time environmental perception data; executing motion control according to the optimal path, and continuously updating semantic information during the operation.

[0095] Understandably, based on the semantic rules set by the fusion semantic map, the passage constraints contained in the semantic labels can be analyzed in depth. For example, it can be clarified which areas are prohibited from passage due to the semantic label of "dangerous area", and which areas labeled "narrow passage" have specific requirements for robot size, so as to provide basic criteria for subsequent path planning.

[0096] Understandably, based on these traffic constraints, the comprehensive cost of multiple path segments is calculated. This cost is not considered in isolation, but rather integrates distance weight and semantic weight. It takes into account both the actual length of the path and the semantic characteristics of the areas traversed by the path, ensuring that the planned path is both short and meets semantic safety requirements.

[0097] Specifically, the comprehensive cost is a multi-objective optimization function, not just considering physical distance. It ensures the basic efficiency of the path through distance weights, striving for the shortest total distance. Simultaneously, it quantifies the compliance, safety, and efficiency of the path through semantic weights. These weights are directly derived from the aforementioned semantic rules: traveling on a "smooth sidewalk" has the lowest cost; traveling on a "gravel road" incurs a moderate penalty due to bumps and increased energy consumption; attempting to traverse areas like "waterlogged areas" or "sandy areas" is costly but inefficient; and any attempt to enter "no-entry" areas (such as "deep water areas" or "dangerous slopes") has its path cost set to infinity, thus being completely excluded by the algorithm. Ultimately, the planning algorithm searches for the path with the lowest comprehensive cost, ensuring that the output is not only a physically short path but also an optimal path that is semantically safe, compliant, efficient, and adaptable to complex outdoor environments.

[0098] Understandably, multiple path segments are sorted and filtered based on the calculated comprehensive cost value to generate a candidate path set, providing a rich set of options for selecting the optimal path. Then, combined with real-time environmental perception data, such as newly detected temporary obstacles by the robot's sensors and dynamic changes in the environment, the optimal path is accurately selected from the candidate path set, enabling path planning to adapt to the real-time environment. Finally, the robot executes motion control based on the optimal path, while continuously updating semantic information during operation to ensure that the robot's understanding of the environment is always synchronized with reality.

[0099] In this embodiment, geometric and semantic information is extracted from a pre-stored GIS map file to generate GIS data, providing a structured foundation with both global geographic information and semantic annotations for subsequent processing. Then, through hierarchical mapping and feature matching, the GIS data is aligned with the SLAM map, achieving a unified association between local geometric information and global geographic information, overcoming the limitation of separation between the two in traditional navigation maps. Based on this, a fused semantic map is generated by dynamically associating semantic information during navigation, enabling the robot to use environmental semantics for perception and reasoning in real time. Finally, path planning and decision control are performed based on this fused semantic map. Compared with existing technologies, this significantly reduces the burden on the robot to repeatedly build maps using a single SLAM, improving navigation efficiency and environmental adaptability.

[0100] Furthermore, based on the above embodiments, this application also proposes a robot navigation system with enhanced semantic maps. The robot navigation system with enhanced semantic maps provided in this application will be described below, and the robot navigation system with enhanced semantic maps described below can be referred to in correspondence with the robot navigation method with enhanced semantic maps described above.

[0101] In this embodiment, please refer to Figure 5 An enhanced semantic map robot navigation system includes: a data acquisition module 01 for extracting geometric and semantic information from a pre-stored GIS map file and generating GIS data based on the geometric and semantic information; a data processing module 02 for generating a GIS map structure aligned with the SLAM map based on the GIS data through hierarchical mapping and feature matching; a data fusion module 03 for dynamically associating semantic information with the current SLAM map based on the GIS map structure to generate a fused semantic map when the robot performs navigation tasks; and a navigation control module 04 for path planning and decision control based on the fused semantic map.

[0102] It is understood that the detailed functional implementation of each of the above units / modules can be found in the description in the foregoing method embodiments, and will not be repeated here.

[0103] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0104] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0105] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0106] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0107] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0108] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0109] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0110] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method of enhanced semantic map-based robot navigation, characterized in that, The method comprises the following steps: extracting geometric information and semantic information based on a pre-stored GIS map file, and generating GIS data based on the geometric information and the semantic information; based on the GIS data, generating a GIS map structure aligned with a SLAM map through hierarchical mapping and feature matching, specifically: based on the GIS data, converting geometric topology into a topological map through hierarchical mapping, and composing a geometric layer from the topological map; based on the GIS data, mapping original labels in an attribute table into standard semantic labels for robot navigation through hierarchical mapping, and composing a semantic layer from the standard semantic labels; based on the attribute table, associating the semantic layer and the geometric layer to generate a GIS map structure; aligning the GIS map structure with the SLAM map through a feature matching algorithm; when a robot performs a navigation task, dynamically associating semantic information in a current SLAM map based on the GIS map structure to generate a fused semantic map; based on the fused semantic map, performing path planning and decision control.

2. The method of Claim 1, wherein, The GIS data comprises geometric topology and an attribute table; based on a pre-stored GIS map file, geometric information and semantic information are extracted, and GIS data is generated based on the geometric information and the semantic information, which comprises the following steps: parsing vector data and raster data of the pre-stored GIS map file through an open source library; based on the vector data, extracting geometric elements of the pre-stored GIS map file and performing topological optimization to generate geometric topology; based on the raster data, extracting navigable area semantic information through a semantic segmentation model; based on the navigable area semantic information and the geometric topology, generating an attribute table corresponding to the geometric elements and the semantic information.

3. The method for robot navigation enhancing semantic maps of claim 1, wherein, When a robot performs a navigation task, dynamically associating semantic information in a current SLAM map based on the GIS map structure to generate a fused semantic map, which comprises the following steps: continuously updating a SLAM local map when the robot performs the navigation task, dynamically associating semantic information in the current SLAM map based on an aligned part of the GIS map structure and the current SLAM map to generate a current semantic map; when there is a non-overlapping part of the GIS map structure and the current SLAM map, retaining the non-overlapping SLAM local map; based on the current semantic map and the non-overlapping SLAM local map, generating a fused semantic map.

4. The robot navigation method of enhancing a semantic map of claim 1, wherein, When a robot performs a navigation task, dynamically associating semantic information in a current SLAM map based on the GIS map structure to generate a fused semantic map, which further comprises the following steps: using a random sample consensus algorithm to check the fused semantic map and remove abnormal data points; real-time detecting whether the GIS map structure is updated; after the GIS map structure is updated, based on the current GIS map structure, dynamically associating semantic information in the current SLAM map to generate a fused semantic map, and updating the fused semantic map.

5. The method for robot navigation enhancing semantic maps of claim 1, wherein, Based on the fused semantic map, performing path planning and decision control, which comprises the following steps: according to semantic rules of the fused semantic map, analyzing traffic constraint conditions in semantic labels; Based on the passage constraint condition, the comprehensive generation value of a plurality of path segments is calculated, the generation value including a distance weight and a semantic weight; According to the comprehensive generation value, a plurality of path segments are sorted and screened to generate a candidate path set; An optimal path is selected from the candidate path set in combination with real-time environment perception data; According to the optimal path, motion control is performed, and semantic information is continuously updated during operation.

6. The method for robot navigation enhancing semantic maps of claim 1, wherein, Based on the pre-stored GIS map file, geometric information and semantic information are extracted, and GIS data is generated based on the geometric information and the semantic information, and the previous method further includes: Determine the current working area of the robot; Based on the current working area, the pre-stored GIS map file of the corresponding area is loaded on demand from the geographic information system database, and the pre-stored GIS map file is format verified and integrity checked; When the pre-stored GIS map file format verification or integrity check fails, SLAM map-based navigation is performed.

7. A robot navigation system that enhances a semantic map, characterized by, It includes: A data acquisition module is configured to extract geometric information and semantic information based on a pre-stored GIS map file, and generate GIS data based on the geometric information and the semantic information; A data processing module is configured to generate a GIS map structure aligned with a SLAM map based on the GIS data through hierarchical mapping and feature matching, specifically: based on the GIS data, the geometric topology structure is converted into a topological map through hierarchical mapping, and the topological map is used to form a geometric layer; based on the GIS data, the original label in the attribute table is mapped into a robot navigation standard semantic label through hierarchical mapping, and the standard semantic label is used to form a semantic layer; based on the attribute table, the semantic layer and the geometric layer are associated to generate a GIS map structure; Through a feature matching algorithm, the GIS map structure and the SLAM map are aligned in the coordinate system; A data fusion module is configured to dynamically associate semantic information in the current SLAM map based on the GIS map structure when the robot performs a navigation task, and generate a fused semantic map; A navigation control module is configured to perform path planning and decision control based on the fused semantic map.

8. A computer-readable storage medium comprising instructions, characterized in that: When the instructions run on an electronic device, the electronic device performs the method of any one of claims 1 to 6.

9. A computer program product comprising a computer program or instructions, characterized in that: When the computer program or instructions run on an electronic device, the electronic device performs the method of any one of claims 1 to 6.

Images