Navigation agent framework construction method based on voxels

Abstract

The invention relates to a voxel-based navigation agent framework construction method, which comprises the following steps of: 1, constructing a path planning framework based on a live event: capturing multi-dimensional situation information for a live environment, quickly collecting environment change, fusing mass heterogeneous data of a multi-element and multi-dimensional sensor, and constructing a path planning framework based on the live event; constructing a panoramic semantic map and an updating mechanism oriented to maneuvering navigation, and realizing a path planning analysis and decision algorithm; and 2, packaging a path navigation agent: constructing the path navigation agent based on a path planning algorithm framework, and driving to complete a complete navigation link closed loop. The problem that the navigation technology facing large-scale and complex scenes is difficult to express in the three-dimensional space of the scene, especially in the aspects of precision and integrity is solved, the navigation provided by the invention does not depend on fixed road network data, can be used in any complex scene, can identify tiny obstacles, and can be applied to the field of navigation. The optimal global maneuvering route can be calculated according to the maneuvering characteristics of the vehicle, and the on-board guidance can be carried out according to the real-time information.

Description

Technical Field

[0001] This invention relates to information system software technology, and more particularly to a method for constructing a voxel-based navigation intelligent agent framework. Background Technology

[0002] The complex urban environment comprises not only roads, rooftops, and the interior and exterior of buildings, but also basements, sewers, subways, and tunnels. Urban terrain is not only dense but also diverse. Intelligent navigation for urban environments demands precise digitization, visualization, interaction, and navigation of geographical environments, building facilities, and entities. This requires enhancing urban situational awareness capabilities and utilizing ubiquitous sensing data for environmental construction, positioning, navigation, and route planning within the task area.

[0003] Automating the reconstruction of spatial intelligence using machine learning or deep learning techniques is a key and challenging problem in computer vision. By capturing static configurations and dynamic changes over time, spatial intelligence should provide a comprehensive representation and understanding of the spatial environment, combining 3D geometry with its temporal evolution. Beyond applications primarily focused on the fundamental components of spatial intelligence (i.e., low-level cues such as depth, camera pose, point cloud maps, and 3D tracking, as well as scene components and dynamics), particular emphasis is placed on the interactions between scene components and the physical plausibility of the reconstructed environment. A holographic basemap needs to be constructed to transform static models into computable, analyzable, and predictable intelligent spaces, serving as a central hub for decision-making.

[0004] Traditional navigation and route planning technologies can calculate suitable routes based on road network data, user-specified starting point, destination, avoidance points, and mode of transportation. However, road network navigation heavily relies on roads and cannot utilize the complex urban environment. Autonomous driving technology, on the other hand, relies on vehicle sensors (visual cameras, LiDAR, etc.) to solve local dynamic planning, but long-distance global planning still requires road network navigation.

[0005] Navigation technologies for large-scale and complex scenarios face challenges in representing the three-dimensional space of the scene, particularly in terms of accuracy and completeness. There is a lack of map representation models specifically for navigation computation, and spatial data structures are redundant, difficult to update, and struggle to adapt to dynamic environments. This invention addresses the following technical problems: (1) Large-scale scenarios have large spatial scales, diverse environments, and high semantic complexity, which significantly increases the consumption of computing and storage resources when expanding from traditional planar maps to three-dimensional space.

[0006] (2) Multimodal data semantic mapping methods rely on the fusion of multiple sensor inputs (such as images, videos, depth, etc.), and require the construction of a multi-layer heterogeneous map fusion representation method.

[0007] (3) There is a lack of incremental navigation planning methods based on real-time input to adapt to dynamic environments. Summary of the Invention

[0008] To address the challenge of representing three-dimensional scene space, particularly in terms of accuracy and completeness, in navigation technologies operating in large-scale and complex scenarios, this invention proposes a voxel-based method for constructing a navigation agent framework. The proposed navigation system does not rely on fixed road network data, can be used in any complex scenario, can identify minute obstacles, calculate the optimal global maneuver route based on vehicle mobility performance, and provide on-the-spot guidance based on real-time information. It can solve: (1) This method addresses the issues of boundary ambiguity and semantic confusion caused by inconsistencies between appearance and semantics, thereby improving users' spatial understanding and interactive experience. This method combines geometric reconstruction and semantic modeling to achieve multi-level fused 3D map representation.

[0009] (2) This method develops an efficient multimodal fusion strategy for map building that can support reliable and flexible queries, and solves the problem of noise and uncertainty in semantic mapping caused by sensor noise and dynamic environmental changes.

[0010] (3) Construct a voxelization process based on spatial occupancy grid, provide a dynamic tile update management mechanism, and provide 3D interpretability for reality-based navigation behavior.

[0011] The technical solution of this invention is as follows: A method for constructing a voxel-based navigation agent framework, comprising: Step 1: Construction of a path planning framework based on real-world events: Capture multi-dimensional situational information in the context of real-world environments, rapidly collect environmental changes, integrate massive heterogeneous data from multiple multi-dimensional sensors, construct a panoramic semantic map and update mechanism for mobile navigation, and realize path planning analysis and decision-making algorithms. Step 1.1: Importing static maps and dynamic sensing data: Considering the characteristics of complex urban environments in terms of spatial refinement, semantics, relevance, and dynamism, the regional environment, buildings, facilities, and static entities are accurately digitized and the latest environmental changes are dynamically received. Step 1.2: Establish a refined environmental representation model: Construct a structured three-dimensional spatial logic model that integrates static and dynamic elements, geometry, topology, and semantic information, including entity element data; Based on a deep learning-based real-time semantic segmentation model and incremental voxel update algorithm, encode the global map into a multi-level three-dimensional map embedding semantic and geometric information; For the geometric constraint information required for navigation, encode the location and shape of key obstacle elements, and encode the appearance and material of some important entities; Step 1.3, Full-space geometric and semantic accessibility analysis and maneuver route planning: Discretize spatial geographic entities into spatial occupancy grids, including intervals occupied by obstacles and intervals not occupied by obstacles; calculate the maneuverable area using voxel mesh update and editing methods; further trim the maneuverable area according to vehicle maneuver parameters to generate navigation area data; merge the navigation area data using grid computing methods; the navigation area data includes the maneuverable area and the connectivity between maneuverable areas. Step 2: Encapsulation of the Path Navigation Agent: A path navigation agent is constructed based on the path planning algorithm framework to drive the completion of the full navigation loop. Through various environmental perception methods and based on a spatial grid, regional environmental information is collected and key elements are dynamically updated. Active perception, fusion mapping, and incremental updates are achieved through a unified map spatial environment representation, enabling high-precision real-time modeling and adaptive maintenance of complex urban environments. Based on a navigation semantic map that balances geometric accuracy and semantic information, combined with deep learning feature extraction, target semantic classification and multi-view data fusion are achieved. This integrates refined updates of the 3D semantic map, dynamic prediction of global maneuver paths, and incremental reasoning for ad-hoc path guidance, realizing path planning and navigation algorithms. This provides intelligent navigation support from environmental perception, semantic understanding, decision planning to action execution, and can collaborate with other agents. Finally, a general large language model is used to complete intent understanding and multimodal expression output, completing the navigation guidance process.

[0012] Furthermore, in step 1.1, the collected heterogeneous data includes laser point clouds, oblique photography, and vector data. Due to their irregular formats, uneven density, and inconsistent scales, a voxel filter is used to project the heterogeneous data into a three-dimensional voxel map. Specifically, an n-meter voxel filter is used to ensure uniform distribution of the collected data, and the data is discretized in the ground plane and height directions to form a uniform spatial three-dimensional grid. This creates lightweight, stable, and easily computed and stored structural features. Global descriptors are extracted to preserve their semantic information. The voxel map retains basic spatial structural information, accurately extracts geometric features from complex three-dimensional scenes, reduces storage requirements, and facilitates subsequent calculations.

[0013] Furthermore, in step 1.2, regarding the structure of the multi-level 3D map, the 3D map is represented as M, where M consists of one static base S and multiple dynamic entities. Representation; each dynamic entity Includes some 3D voxels A semantic description Establish a multi-layer map representation method that includes static layer, dynamic layer, semantic layer and topological layer; The semantic layer output adopts the JSON structured scene graph format, which supports tasks including visualization and interaction, and path planning loss calculation input. The scene graph description is shown below. { "objects": [ {"id": 1, "name": "building", "bbox": [...], "type", "cost": ...}, {"id": 2, "name": "bicycle", "bbox": [...], "type", "cost": ...} ], "relations": [ {"subject": 1, "predicate": "right of", "object": 2: ...}, {"subject": 1, "predicate": "left of", "object": 3: ...} ] } Here, objects are dynamic entities, type is the category, cost is a comprehensive evaluation of passability and loss, and is calculated for subsequent navigation; relations come from the final set after predicate prediction and consistency constraints; through this design, the map can simultaneously meet the needs of visual, knowledge and structured expression, providing high-quality input for subsequent path planning.

[0014] Furthermore, in step 1.3, the route planning includes: inputting vehicle parameters, comprehensively evaluating dynamic and static factors including mobility, traffic conditions, and weather conditions, and automatically generating the optimal action path using spatial entity labels and the topological relationships of neighboring cells within the navigation area grid, as detailed below: An incremental calculation model combining global and local planning is adopted to adapt to dynamic environmental changes; IF (Initial planning OR Environmental changes) Execute global path planning mode ELSE IF (Local obstacle changes OR Local path optimization) Execute local incremental optimization mode END IF In the global path planning mode, an improved A* algorithm is adopted, which introduces directional constraints to avoid unnecessary turns and evaluates obstacles and passage risks at each node in the environment. In the local incremental optimization mode, local replanning based on RRT is adopted to quickly reconstruct local path segments based on the current path and smoothly process connection points.

[0015] The beneficial effects of this invention are as follows: (1) This method achieves sub-meter level 3D reconstruction capability for complex spaces. In large-scale scenes, the spatial scale is large, the environment is diverse and the complexity is high. The high-quality reconstruction of this method can enhance spatial understanding and interactive experience. It improves the ability to deeply integrate and coordinate the expression of complex environmental spaces under a unified representation framework, and the accuracy reaches the sub-meter level.

[0016] (2) This method constructs an instance-level scene semantic annotation framework. Instance-level scene reconstruction focuses more on the semantic understanding of the scene. Semantic maps are the foundation for agents to perform advanced reasoning and decision-making in complex environments. In addition to capturing the geometric information of the environment, it is also necessary to include high-level semantic information, which is crucial for performing complex tasks such as path navigation, target retrieval and scene understanding.

[0017] (3) This method has the ability to dynamically update the spatial occupancy grid. Under the condition of continuous change of geographical environment, it can construct a dynamic update model of spatial environment information, which can collect complex data information such as environmental history and actual dynamic targets, weather, topography, geological environment, etc., to realize the fusion and update of regional high-precision map data, and has the ability of incremental path planning and guidance. Attached Figure Description

[0018] Figure 1 This is a flowchart of the map updating and path planning framework based on real-time events of the present invention. Figure 2 This is a flowchart of the voxelized map construction process of the present invention; Figure 3 This is a diagram of the multi-level map representation model of the present invention; Figure 4 This is a flowchart of the incremental navigation route planning process of the present invention; Figure 5 This is a flowchart illustrating the execution process of the navigation intelligent agent of this invention. Detailed Implementation

[0019] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0020] This method addresses the challenges of complex spatiotemporal data structures, weak high-precision multidimensional map generation capabilities, and untimely map updates by considering terrain features. It proposes an environment-active perception architecture based on a map representation model, supporting dynamic regional updates and constructing a navigation agent based on a multi-level dynamic map.

[0021] This method consists of two stages: (1) Construction of a path planning framework based on real-world events To capture multi-dimensional situational information in real-world environments, rapidly collect environmental changes, and fuse massive heterogeneous data from multiple multi-dimensional sensors, a panoramic semantic map and update mechanism for maneuver navigation are constructed, enabling path planning analysis and decision-making algorithms. The framework process is as follows: Figure 1 .

[0022] ① Static map and dynamic sensing data import: Considering the characteristics of complex urban environment space refinement, semanticization, correlation and dynamism, the regional environment, building facilities and static entities are accurately digitally constructed, and the latest environmental changes are dynamically received to achieve high-precision alignment and fusion of information such as point cloud and multi-sensor data, and eliminate perception inconsistencies.

[0023] For acquired laser point clouds, oblique photography, and vector data, which have irregular formats, uneven density, and inconsistent scales, a voxel filter is used to project the heterogeneous data into a 3D voxel map, such as... Figure 2 As shown. For example, an n-meter voxel filter is used to distribute the collected data evenly, and the data is discretized in the ground plane and height directions to divide it into a uniform three-dimensional spatial grid. This forms a lightweight, stable, and easily computed and stored structural feature, and global descriptors are extracted to retain its semantic information. The voxel map retains basic spatial structural information, can accurately extract geometric features from complex three-dimensional scenes, and can reduce storage requirements, facilitating subsequent calculations.

[0024] ② Establish a refined environmental representation model: Construct a structured 3D spatial logic model integrating static and dynamic elements, geometry, topology, and semantic information, including entity element data such as terrain, vegetation, buildings, hydrology, roads, bridges, and tunnels. Based on a deep learning-based real-time semantic segmentation model and incremental voxel update algorithm, the global map is encoded into a multi-level 3D map embedding semantic and geometric information. For the geometric constraints required for navigation, the location and shape of key obstacle elements are encoded, as well as the appearance and material of some important entities.

[0025] A schematic diagram of a multi-level map structure is shown below. Figure 3 As shown. In terms of map structure, the 3D map is represented as M, which consists of a static base S and multiple dynamic entities. Representation. Each dynamic entity Includes some 3D voxels A semantic description A multi-layer map representation method is established, which includes a static layer, a dynamic layer, a semantic layer, and a topological layer. The structured description of the multi-layer map is shown in Table 1.

[0026] Table 1. Multi-level map structure description

[0027] The semantic layer output adopts the JSON structured scene graph format, which supports tasks such as visualization and interaction, path planning loss calculation input, etc. The scene graph description is shown below.

[0028] { "objects": [ {"id": 1, "name": "building", "bbox": [...], "type", "cost": ...}, {"id": 2, "name": "bicycle", "bbox": [...], "type", "cost": ...} ], "relations": [ {"subject": 1, "predicate": "right of", "object": 2: ...}, {"subject": 1, "predicate": "left of", "object": 3: ...} ] } In this design, `objects` represents dynamic entities, `type` represents categories, `cost` comprehensively evaluates passability and loss, and is calculated for subsequent navigation; `relations` comes from the final set after predicate prediction and consistency constraints. This design allows the map to simultaneously meet visual, knowledge-based, and structured representation needs, providing high-quality input for subsequent path planning.

[0029] ③ Full-space geometric and semantic accessibility analysis and maneuver route planning: Spatial geographic entities are discretized into spatial occupancy grids, including intervals occupied by obstacles and intervals not occupied by obstacles. A voxel mesh update and editing method is used to calculate the maneuverable region. The maneuverable region is further pruned based on vehicle maneuver parameters to generate navigation region data. A grid computing method is used to merge the navigation region data, which includes the maneuverable region and the connectivity relationships between maneuverable regions.

[0030] Input vehicle parameters, comprehensively evaluate dynamic and static factors such as mobility, traffic conditions, and weather conditions, and automatically generate the optimal travel path using spatial entity labels and the topological relationships of neighboring cells within the navigation area grid. The path planning steps are as follows: Figure 4 As shown.

[0031] An incremental calculation model combining global and local planning is adopted to adapt to dynamic environmental changes.

[0032] IF (Initial planning OR Environmental changes) Execute global path planning mode ELSE IF (Local obstacle changes OR Local path optimization) Execute local incremental optimization mode END IF In the global path planning mode, an improved A* algorithm is adopted, directional constraints are introduced to avoid unnecessary turns, and obstacles and passage risks at each node in the environment are evaluated. In the local incremental optimization mode, local replanning based on RRT is adopted to quickly reconstruct local path segments based on the current path and smoothly process connection points.

[0033] (2) Encapsulation of path navigation agent A path navigation agent is constructed based on a path planning algorithm framework to drive the completion of a full navigation loop. The agent's execution flow is as follows: Figure 5 .

[0034] By employing various environmental sensing methods and based on spatial grids, regional environmental information collection and rapid updates of key dynamic changes are achieved. Through a unified map spatial environment representation, proactive sensing, fusion mapping, and incremental updates are accomplished, enabling high-precision real-time modeling and adaptive maintenance of complex urban environments.

[0035] Based on a navigation semantic map that balances geometric accuracy and semantic information, combined with deep learning feature extraction, it achieves target semantic classification and multi-view data fusion. It integrates fine-grained updates of the 3D semantic map, dynamic prediction of global maneuver paths, and incremental reasoning of ad hoc guidance paths to achieve sub-meter-level path planning and navigation algorithms. It realizes intelligent navigation support from environmental perception, semantic understanding, decision planning to action execution, and can collaborate with other intelligent agents, such as calling points of interest to query important landmark information along the way. Finally, it combines a general large language model to complete intent understanding and multimodal expression output to complete the navigation guidance process.

[0036] The above-described embodiments are merely one implementation of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention should be determined by the appended claims.

Claims

1. A method for constructing a voxel-based navigation intelligent agent framework, characterized in that, include: Step 1: Construction of a path planning framework based on real-world events: Capture multi-dimensional situational information in the context of real-world environments, rapidly collect environmental changes, integrate massive heterogeneous data from multiple multi-dimensional sensors, construct a panoramic semantic map and update mechanism for mobile navigation, and realize path planning analysis and decision-making algorithms. Step 1.1: Importing static maps and dynamic sensing data: Considering the characteristics of complex urban environments in terms of spatial refinement, semantics, relevance, and dynamism, the regional environment, buildings, facilities, and static entities are accurately digitized and the latest environmental changes are dynamically received. Step 1.2: Establish a refined environmental representation model: Construct a structured three-dimensional spatial logic model that integrates static and dynamic elements, geometry, topology, and semantic information, including entity element data; Based on a deep learning-based real-time semantic segmentation model and incremental voxel update algorithm, encode the global map into a multi-level three-dimensional map embedding semantic and geometric information; For the geometric constraint information required for navigation, encode the location and shape of key obstacle elements, and encode the appearance and material of some important entities; Step 1.3, Full-space geometric and semantic accessibility analysis and maneuver route planning: Discretize spatial geographic entities into spatial occupancy grids, including intervals occupied by obstacles and intervals not occupied by obstacles; The maneuverable region is calculated using a voxel mesh update and editing method. The maneuverable region is further trimmed based on vehicle maneuvering parameters to generate navigation region data. The navigation region data is then merged using a mesh calculation method. The navigation region data includes the maneuverable region and the connectivity between maneuverable regions. Step 2: Encapsulation of Path Navigation Agent: Construct a path navigation agent based on the path planning algorithm framework to drive the completion of the complete navigation link closed loop; realize the collection of regional environmental information and rapid updates of key elements through various environmental perception methods based on spatial grid; achieve active perception, fusion mapping and incremental updates through unified map spatial environment representation, and realize high-precision real-time modeling and adaptive maintenance of complex urban environments. Based on a navigation semantic map that balances geometric accuracy and semantic information, combined with deep learning feature extraction, it achieves target semantic classification and multi-view data fusion. It integrates fine-grained updates of the 3D semantic map, dynamic prediction of global maneuver paths, and incremental reasoning of ad hoc guidance paths to realize path planning and navigation algorithms. It enables intelligent navigation support from environmental perception, semantic understanding, decision planning to action execution, and can collaborate with other intelligent agents. Finally, it combines a general large language model to complete intent understanding and multimodal expression output, thus completing the navigation guidance process.

2. The method for constructing a voxel-based navigation agent framework according to claim 1, characterized in that, In step 1.1, the collected heterogeneous data includes laser point clouds, oblique photography, and vector data. Due to irregular formats, uneven density, and inconsistent scales, a voxel filter is used to project the heterogeneous data into a three-dimensional voxel map. Specifically, an n-meter voxel filter is used to ensure uniform distribution of the collected data. The data is discretized in the ground plane and height directions, divided into a uniform three-dimensional spatial grid, forming a lightweight, stable, and easily computed and stored structural feature. Global descriptors are extracted to preserve their semantic information. The voxel map retains basic spatial structural information, can accurately extract geometric features from complex three-dimensional scenes, and can reduce storage requirements, facilitating subsequent calculations.

3. The method for constructing a voxel-based navigation agent framework according to claim 1, characterized in that, In step 1.2, regarding the structure of the multi-level 3D map, the 3D map is represented as M, which consists of one static base S and multiple dynamic entities. Representation; each dynamic entity Includes some 3D voxels A semantic description Establish a multi-layer map representation method that includes static layer, dynamic layer, semantic layer and topological layer; The semantic layer output adopts the JSON structured scene graph format, which supports tasks including visualization and interaction, and path planning loss calculation input. The scene graph description is shown below. { "objects": [ {"id": 1, "name": "building", "bbox": [...], "type", "cost": ...}, {"id": 2, "name": "bicycle", "bbox": [...], "type", "cost": ...} ], "relations": [ {"subject": 1, "predicate": "right of", "object": 2: ...}, {"subject": 1, "predicate": "left of", "object": 3: ...} ] } Here, objects are dynamic entities, type is the category, cost is a comprehensive evaluation of passability and loss, and is calculated for subsequent navigation; relations come from the final set after predicate prediction and consistency constraints; through this design, the map can simultaneously meet the needs of visual, knowledge and structured expression, providing high-quality input for subsequent path planning.

4. The method for constructing a voxel-based navigation agent framework according to claim 1, characterized in that, In step 1.3, the route planning includes: inputting vehicle parameters, comprehensively evaluating dynamic and static factors including mobility, traffic conditions, and weather conditions, and automatically generating the optimal action path using spatial entity labels and the topological relationships of neighboring cells within the navigation area grid, as detailed below: An incremental calculation model combining global and local planning is adopted to adapt to dynamic environmental changes; IF (Initial planning OR Environmental changes) Execute global path planning mode ELSE IF (Local obstacle changes OR Local path optimization) Execute local incremental optimization mode END IF In the global path planning mode, an improved A* algorithm is adopted, which introduces directional constraints to avoid unnecessary turns and evaluates obstacles and passage risks at each node in the environment. In the local incremental optimization mode, local replanning based on RRT is adopted to quickly reconstruct local path segments based on the current path and smoothly process connection points.

Images