Indoor planar graph-guided cognitive navigation method for self-contained agent

Abstract

The invention discloses an indoor planar graph guided cognitive navigation method for an intelligent agent with a body. The method comprises the following steps: identifying and correcting a guide view image; carrying out adaptive semantic segmentation and labeling; constructing a semantic-driven topological map; reasoning and self-positioning based on semantic clues; and hierarchical navigation planning and control. According to the embodiment of the invention, a brand new normal form for semantic topology mapping and navigation in combination with the indoor plane guide view is provided, a multi-scene and multi-scale guide view data set is constructed, an efficient image segmentation and correction model is trained, semantic information is innovatively converted into a lightweight topological graph, and the navigation efficiency is improved. And semantic reasoning and a hierarchical navigation framework based on an evidence theory are fused, so that the success rate and the efficiency of navigation and the adaptability to a dynamic environment are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent navigation technology, and in particular to an embodied intelligent agent cognitive navigation method guided by an indoor floor plan. Background Technology

[0002] As autonomous navigation robots continue to drive the intelligent transformation of various industries, the development of autonomous navigation in large indoor scenarios is also facing both opportunities and challenges. However, indoor autonomous navigation systems still face the following problems: building exterior walls will cause GPS failure, indoor obstacles are uncertain and dynamic, and indoor scenarios are similar and information is dense.

[0003] The objective of this application is to construct an embodied intelligent agent cognitive navigation method guided by an indoor planar navigation map in a real large shopping mall environment. This method aims to solve the inherent challenges of navigation environment in actual map-based zero-shot target navigation (ZSON) and the technical difficulties in the application of current mainstream solutions. It seeks to provide a new paradigm of cognitive autonomous navigation based on semantic topology map construction for indoor navigation.

[0004] However, semantic topology construction and navigation based on planar guide maps face numerous challenges: First, actual captured guide maps suffer from perspective distortion and uneven lighting, affecting image quality and structure extraction. Second, annotation styles vary across different locations, and semantic information is presented in inconsistent forms (such as text, color, and symbols), making unified parsing difficult. Furthermore, text detection and recognition accuracy is low in complex backgrounds, and traditional OCR methods are susceptible to interference. The fusion of semantic and spatial information is also fraught with difficulties due to differences in coordinate systems and inconsistent annotations. Topology construction also needs to overcome issues such as node ambiguity and poor adaptability to dynamic environments, while ensuring real-time performance and robustness during navigation, placing high demands on algorithm design and system integration.

[0005] Therefore, it is necessary to improve one or more of the problems existing in the above-mentioned related technical solutions.

[0006] It should be noted that this section is intended to provide background or context for the technical solutions of this disclosure as set forth in the claims. The description herein does not constitute an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0007] In view of the challenges of current indoor navigation solutions, such as high hardware resource consumption, poor environmental adaptability, and difficulty in effectively integrating and utilizing the rich semantic and spatial information in indoor scenes, the purpose of this invention is to provide an embodied intelligent agent cognitive navigation method guided by indoor floor plan, thereby overcoming at least to some extent one or more problems caused by the limitations and defects of related technologies.

[0008] This invention first provides an embodied intelligent agent cognitive navigation method guided by an indoor floor plan, comprising: S1, Guide View Image Recognition and Correction: The input image is processed using a semantic segmentation model based on the UNET architecture to accurately segment the region where the guide view is located and generate a binary mask; based on the obtained mask, a perspective correction algorithm is executed to obtain the corrected guide view image; S2, Adaptive Semantic Segmentation and Labeling: Superpixel pre-segmentation and semantic classification are performed on the corrected guide view image, and a divide-and-conquer strategy is used for text detection to transform the heterogeneous semantic information in the corrected guide view into structured data with accurate semantic labels. S3, semantic-driven topology map construction: determine topology nodes based on structured data, connect each topology node to the nearest walkable node, and obtain a topology map; S4, Semantic Clue-Based Reasoning and Self-Localization: Process the real-time visual observations of the agent, extract the observed semantic landmarks, for each observed landmark, filter candidate semantic nodes in the pre-constructed semantic topology map and assign basic probability values, fuse the basic probability values ​​of multiple semantic landmarks to generate the confidence distribution of the agent's position, minimize the difference between the vectors of the observed landmarks and the agent and the vectors of the landmarks and their corresponding nodes in the prior map, align the observation configuration with the map structure, and correct perception errors; S5, Hierarchical navigation planning and control: Calculate the shortest path from the starting point to the target point on the topology map. For any two adjacent topology nodes on the path, calculate the connection vector, distance and orientation angle between them, generate a vector sequence, and the agent moves according to the vector sequence.

[0009] In this invention, step S1, which involves performing a perspective correction algorithm based on the obtained mask to obtain the corrected guide view image, includes: S11, Identify the largest quadrilateral region in the mask through connected component analysis; S12, calculate the deviation of each corner point of the quadrilateral from 90 degrees to estimate the degree of distortion; S13, calculate the perspective transformation matrix between the quadrilateral and a standard rectangle; S14, apply the transformation matrix to obtain the corrected guide view image.

[0010] In this invention, in S2, Felzenszwalb's graph theory algorithm is used to perform superpixel pre-segmentation on the corrected guide view image, and a lightweight encoder-decoder CNN is used for semantic classification.

[0011] In this invention, S3 includes: S31, Based on the structured data, find the set of visible semantic nodes within a preset radius for each semantic node; S32, construct a polygon from the set of visible semantic nodes and calculate its centroid as the optimal topological node position; S33, extract the corridor area from the road mask, identify the skeleton intersections, and add midpoints as support nodes between the intersections and corners; S34 connects each topology node to the nearest passable node to obtain a topology map.

[0012] In this invention, S4 includes: S41 processes the real-time visual observations of the agent and extracts the observed semantic landmarks and their spatial locations; S42, for each observed landmark, find its nearest preset number of candidate semantic nodes in the pre-built topological map using Euclidean distance; S43, assign basic probability values ​​to each candidate semantic node using a Gaussian kernel function; S44 uses Dempster's combination rule to fuse the basic probability assignments of multiple semantic landmarks to generate the confidence distribution of the agent's location; S44 minimizes the difference between the observed landmark and agent vectors and the vectors of landmarks and their corresponding nodes in the prior map, aligns the observation configuration with the map structure, and corrects perception errors.

[0013] In this invention, S5 includes: S51. Use Dijkstra's algorithm to calculate the shortest path from the starting point to the target point on the topological graph. This path is a sequence of topological nodes. S52, for any two adjacent topological nodes on the path, calculate their connection vector, distance and orientation angle, and generate a vector sequence; S53, the agent uses a local motion planner to move following a sequence of vectors. Whenever it reaches a topological node, it performs semantic-based relocalization to correct the accumulated odometry drift and realign the estimated pose with the global map.

[0014] The present invention further provides an embodied intelligent agent cognitive navigation system guided by an indoor floor plan, comprising: The wayfinding image recognition and correction unit uses a semantic segmentation model based on the UNET architecture to process the input image, accurately segment the region where the wayfinding view is located, and generate a binary mask; based on the obtained mask, a perspective correction algorithm is executed to obtain the corrected wayfinding image. Adaptive semantic segmentation and annotation unit: performs superpixel pre-segmentation and semantic classification on the corrected guide view image, adopts a divide-and-conquer strategy for text detection, and transforms the heterogeneous semantic information in the corrected guide view into structured data with accurate semantic labels; Semantic-driven topology map building unit: Determine topology nodes based on structured data, connect each topology node to the nearest walkable node, and obtain a topology map; The semantic cue-based reasoning and self-localization unit processes the agent's real-time visual observations, extracts observed semantic landmarks, filters candidate semantic nodes in a pre-constructed semantic topology map for each observed landmark and assigns basic probability values, fuses the basic probability values ​​of multiple semantic landmarks to generate the confidence distribution of the agent's position, minimizes the difference between the vectors of the observed landmarks and the agent and the vectors of the landmarks and their corresponding nodes in the prior map, aligns the observation configuration with the map structure, and corrects perception errors. Hierarchical navigation planning and control unit: Calculates the shortest path from the starting point to the target point on the topology map. For any two adjacent topology nodes on the path, calculates the connectivity vector, distance, and orientation angle between them, and generates a vector sequence. The agent moves according to the vector sequence.

[0015] The technical solution provided by this invention may include the following beneficial effects: This invention presents an embodied intelligent agent cognitive navigation method guided by an indoor floor plan. It proposes a novel paradigm for semantic topology mapping and navigation by combining indoor floor plan navigation maps. It constructs a multi-scene, multi-scale navigation map dataset, trains an efficient image segmentation and correction model, innovatively transforms semantic information into a lightweight topology map, and integrates semantic reasoning based on evidence theory and a hierarchical navigation framework, which significantly improves the success rate, efficiency, and adaptability to dynamic environments of navigation. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0017] Figure 1 A flowchart illustrating an embodied intelligent agent cognitive navigation method guided by an indoor floor plan in an exemplary embodiment of this disclosure is shown. Figure 2 This diagram illustrates the overall architecture of an embodied intelligent agent cognitive navigation system guided by an indoor floor plan, as shown in an exemplary embodiment of this disclosure. Figure 3 This diagram illustrates the environment parsing and semantic construction flowchart in an exemplary embodiment of this disclosure; Figure 4 This diagram illustrates a flowchart of a semantic topology graph construction algorithm in an exemplary embodiment of this disclosure. Figure 5This diagram illustrates a semantic clue reasoning localization and hierarchical navigation framework in an exemplary embodiment of this disclosure. Figure 6 This diagram illustrates the visualization results of semantic clue-based reasoning and self-localization effect verification in an exemplary embodiment of this disclosure. Detailed Implementation

[0018] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0019] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0020] This example implementation first provides an embodied intelligent agent cognitive navigation method guided by an indoor floor plan. Please refer to [reference needed]. Figure 1 This method may include: S1-S5, as follows: S1, Guide View Image Recognition and Correction: The input image is processed using a semantic segmentation model based on the UNET architecture to accurately segment the region where the guide view is located and generate a binary mask; based on the obtained mask, a perspective correction algorithm is executed to obtain the corrected guide view image; S2, Adaptive Semantic Segmentation and Labeling: Superpixel pre-segmentation and semantic classification are performed on the corrected guide view image, and a divide-and-conquer strategy is used for text detection to transform the heterogeneous semantic information in the corrected guide view into structured data with accurate semantic labels. S3, semantic-driven topology map construction: determine topology nodes based on structured data, connect each topology node to the nearest walkable node, and obtain a topology map; S4, Semantic Clue-Based Reasoning and Self-Localization: Process the real-time visual observations of the agent, extract the observed semantic landmarks, for each observed landmark, filter candidate semantic nodes in the pre-constructed semantic topology map and assign basic probability values, fuse the basic probability values ​​of multiple semantic landmarks to generate the confidence distribution of the agent's position, minimize the difference between the vectors of the observed landmarks and the agent and the vectors of the landmarks and their corresponding nodes in the prior map, align the observation configuration with the map structure, and correct perception errors; S5, Hierarchical navigation planning and control: Calculate the shortest path from the starting point to the target point on the topology map. For any two adjacent topology nodes on the path, calculate the connection vector, distance and orientation angle between them, generate a vector sequence, and the agent moves according to the vector sequence.

[0021] In this embodiment, a novel paradigm for semantic topology mapping and navigation by combining indoor planar wayfinding maps is proposed. A multi-scene, multi-scale wayfinding map dataset is constructed, and an efficient image segmentation and correction model is trained. Semantic information is innovatively transformed into a lightweight topology map, and a semantic reasoning and hierarchical navigation framework based on evidence theory are integrated, which significantly improves the success rate, efficiency and adaptability to dynamic environments of navigation.

[0022] The specific process of each step in the above embodiments will be described below.

[0023] S1, Guide View Image Recognition and Correction.

[0024] In S1, the step of performing a perspective correction algorithm based on the obtained mask to obtain the corrected guide view image includes: S11, Identify the largest quadrilateral region in the mask through connected component analysis; S12, calculate the deviation of each corner point of the quadrilateral from 90 degrees to estimate the degree of distortion; S13, calculate the perspective transformation matrix between the quadrilateral and a standard rectangle; S14, apply the transformation matrix to obtain the corrected guide view image.

[0025] Specifically, the original acquired guide view presents challenges due to perspective distortion and uneven lighting. This invention employs a two-stage preprocessing method. First, a semantic segmentation model based on the UNET architecture is used to process the input image, accurately segmenting the region containing the guide view and generating a binary mask. The encoder-decoder structure of this network, combined with skip connections, effectively preserves detailed information, and its loss function uses cross-entropy loss. Subsequently, based on the obtained mask, a perspective correction algorithm is executed: 1) Identifying the largest quadrilateral region in the mask through connected component analysis; 2) Calculating the deviation of each corner point from 90 degrees to estimate the degree of distortion; 3) Calculating the perspective transformation matrix between the quadrilateral and a standard rectangle; 4) Applying this transformation matrix to obtain the corrected image. Experiments show that the segmentation accuracy of this method (using the average intersection-union ratio mIoU) reaches 88%, which is significantly better than the traditional computer vision method (58%).

[0026] S2, Adaptive Semantic Segmentation and Labeling.

[0027] Please refer to Figure 3 , Figure 3This invention presents a flowchart for environmental analysis and semantic construction. Addressing the challenges of diverse annotation styles and varying semantic information presentation formats (color blocks, text) in guide views, this invention proposes an unsupervised segmentation and annotation process. This process comprises three key components: 1) Superpixel pre-segmentation: Initial oversegmentation is performed using Felzenszwalb's graph theory algorithm, considering similarity in both color and spatial distance dimensions, improving boundary recognition accuracy by 27% compared to the traditional SLIC algorithm. 2) CNN feature optimization: A lightweight encoder-decoder (CNN) is designed, taking the RGB and positional encoding of superpixels as input and outputting the probability distribution of their belonging to K semantic categories. A joint loss function combining cross-entropy loss and Dice loss is used, and spatial smoothing constraints are introduced to suppress noise. 3) Text detection based on label regions: To avoid interference from complex backgrounds, a divide-and-conquer strategy is adopted: first, different regions are separated according to functional attributes and grayscale histograms are generated to establish mappings; then, a lightweight PP-OCR model is applied to the separated label regions for text detection; finally, road boundary references are used to correct the spatial relationship between text annotations and actual observation points.

[0028] S3, semantic-driven topology map construction.

[0029] Please refer to Figure 4 S3 includes: S31, Based on the structured data, find the set of visible semantic nodes within a preset radius for each semantic node; S32, construct a polygon from the set of visible semantic nodes and calculate its centroid as the optimal topological node position; S33, extract the corridor area from the road mask, identify the skeleton intersections, and add midpoints as support nodes between the intersections and corners; S34 connects each topology node to the nearest passable node to obtain a topology map.

[0030] Specifically, this step transforms semantic information into a structured topological graph that can be used for path planning. The construction process consists of two sub-steps: node selection and edge construction.

[0031] Node selection: Given a list of parsed semantic nodes The topology nodes are determined using the following strategies: 1) Semantic region construction: For each semantic node, find the radius. 1) Set of visible semantic nodes within the graph; 2) Centroid calculation: Construct a polygon from the above set and calculate its centroid as the optimal topological node position; 3) Support node generation: Extract corridor regions from the road mask, identify skeleton intersections, and add midpoints between intersections and corners as support nodes to ensure graph connectivity. For sparse regions with fewer than 2 visible semantic nodes, a strategy using the farthest visible point is used to balance node density.

[0032] Edge construction: For each topological node, connect it to the nearest walkable node. The edge weights use a weighted distance metric that considers not only Euclidean distance but also introduces a penalty mechanism: counting the number of black (impassable) pixels traversed by the connecting line segment. and number of white pixels The algorithm continuously updates the paths, replacing longer paths with shorter ones to ensure the global optimality of the graph.

[0033] S4, reasoning and self-localization based on semantic clues.

[0034] S4 includes: S41 processes the real-time visual observations of the agent and extracts the observed semantic landmarks and their spatial locations; S42, for each observed landmark, find its nearest preset number of candidate semantic nodes in the pre-built topological map using Euclidean distance; S43, assign basic probability values ​​to each candidate semantic node using a Gaussian kernel function; S44 uses Dempster's combination rule to fuse the basic probability assignments of multiple semantic landmarks to generate the confidence distribution of the agent's location; S44 minimizes the difference between the observed landmark and agent vectors and the vectors of landmarks and their corresponding nodes in the prior map, aligns the observation configuration with the map structure, and corrects perception errors.

[0035] Specifically, in complex indoor environments, an agent, like a human, infers its location by observing salient semantic landmarks (such as "seeing Starbucks") and their spatial relationships. This invention proposes a semantic reasoning localization method based on Dempster-Shafer evidence theory. Given the agent's global observations G, a visual-language model (VLM) extracts a set of observed semantic landmarks and their spatial locations. For each observed landmark, the K nearest candidate semantic nodes are found in a pre-constructed topological map using Euclidean distance. Based on distance, a basic probability assignment (BPA) is assigned to each candidate node using a Gaussian kernel function. Finally, Dempster's combination rule is used to fuse evidence from multiple different landmarks to obtain a confidence distribution across all possible locations, i.e., the agent's probabilistic localization. Subsequently, further fine localization is achieved through geometric reasoning: minimizing the difference between the vectors of the observed landmarks and the agent and the vectors of the landmarks and their corresponding nodes in the prior map, aligning the observation configuration with the map structure, and improving robustness to perceptual errors.

[0036] S5, hierarchical navigation planning and control.

[0037] S5 includes: S51. Use Dijkstra's algorithm to calculate the shortest path from the starting point to the target point on the topological graph. This path is a sequence of topological nodes. S52, for any two adjacent topological nodes on the path, calculate their connection vector, distance and orientation angle, and generate a vector sequence; S53, the agent uses a local motion planner to move following a sequence of vectors. Whenever it reaches a topological node, it performs semantic-based relocalization to correct the accumulated odometry drift and realign the estimated pose with the global map.

[0038] Specifically, the navigation task is performed on a priori topology map. Given a starting point, a target point, and a topology map, the system operates according to the following steps: 1) Global planning: Use Dijkstra's algorithm to calculate the shortest path from the starting point to the target point on the topology graph. This path is represented as a sequence of topology nodes.

[0039] 2) Vector sequence generation: For any two adjacent topological nodes on the path and Calculate the connected vectors between them. ,distance and direction angle This vector sequence will serve as macroscopic guidance for the local planner.

[0040] 3) Local Planning and Relocalization: The agent uses a local motion planner (such as ROS2's move_base) to move following a vector sequence. Whenever a sub-target point (topology node) is reached, the system triggers semantic-based relocalization to correct accumulated odometry drift and realign the estimated pose with the global map.

[0041] This hierarchical loop of "global planning - local execution - semantic relocation" ensures continuous and robust navigation in complex semantic environments.

[0042] The core idea of ​​this technical solution is to mimic the cognitive navigation strategy of humans in unfamiliar indoor environments. Humans typically first seek out a floor plan of the location to quickly obtain overall spatial layout and semantic information (such as store names and functional area divisions), forming a "cognitive sketch" in their minds composed of key landmarks and connecting paths. Subsequently, during movement, by continuously observing and matching with the "cognitive sketch" in their minds, they determine their own location and navigate to their destination.

[0043] This application transforms this biological intelligence process into a computational model. Its core principle is to convert a static indoor planar navigation view rich in semantic information into a lightweight, symbolic semantic topological map using computer vision and artificial intelligence technologies. When navigating, the intelligent agent no longer relies on continuous matching and positioning of a high-precision geometric map. Instead, it uses its onboard visual sensors to identify semantic landmarks in the environment (such as store logos and sign text) in real time, comparing and probabilistically inferring their meaning with the prior semantic topological map, thereby achieving efficient and robust "cognitive navigation." To achieve the above tasks, this invention adopts the following technical solution: The entire navigation method is divided into two main stages: Offline preprocessing and mapping stage: An indoor floor plan is input, and a semantic topology map is automatically constructed through image correction, semantic segmentation, and text recognition. Online navigation and localization stage: The preprocessed semantic topology map is loaded, and the agent performs self-localization through real-time observation and semantic reasoning. A hierarchical strategy is then employed for global and local path planning to ultimately complete the navigation task. To accurately describe the technical solution, key concepts are first formally defined: Guide view: Defined as an RGB image It contains rich semantic and geometric information such as spatial layout, functional areas, and text annotations; Semantic topological map: defined as a weighted directed graph It is a set of vertices. Each vertex ,in This represents the two-dimensional coordinates of the vertex in the map coordinate system. It is the set of semantic labels associated with the vertex (such as {"Apple Store", "Entrance"}). It is a set of edges, each edge This indicates that there is a passable path between the vertex and the vertex.

[0044] Agent state: At time t, the agent state is defined as follows: ,in It is its estimated pose (position and orientation) in the environment. It is sensor observation data.

[0045] Observation: The agent's location Local environmental information acquired through visual sensors is processed by a visual-language model (VLM) to extract a set of semantic landmarks. Each of the landmarks It includes the distance, azimuth angle, and semantic label relative to the agent.

[0046] Detailed Explanation of Technical Solutions 1. Guide View Image Recognition and Correction Guide views captured by mobile devices suffer from perspective distortion and uneven lighting, resulting in poor processing quality when directly processed. This solution employs a method combining a deep learning-based segmentation network with traditional geometric transformations for correction.

[0047] (1) UNET region detection: The UNET network with encoder-decoder structure is used to detect the original image. Through a mapping Predict a binary mask The white pixels represent the guide view area. The network is trained by minimizing the cross-entropy loss between the predicted mask and the ground truth label. (Network parameters)

[0048] (2) Perspective correction: Perform morphological operations and contour detection on the mask, extract its maximum circumscribed convex quadrilateral, define the target rectangle, and calculate the homography matrix between the two. And apply this transformation to obtain the corrected image. :

[0049] 2. Adaptive semantic segmentation and annotation Guide map annotation styles vary, necessitating an unsupervised or weakly supervised segmentation method adaptable to different environments. This solution combines superpixel oversegmentation, CNN feature learning, and regionalized OCR to achieve semantic annotation.

[0050] (1) Superpixel pre-segmentation: The graph-based Felzenszwalb algorithm is used to over-segment the corrected image to generate a set of superpixels. This algorithm can better preserve the integrity of boundary features using color and position characteristics as input; (2) CNN Feature Optimization and Classification: Design a lightweight CNN network that optimizes each superpixel The network takes color and location features as input and outputs the probability distribution of their belonging to K semantic categories (such as shop, corridor, staircase). It uses a joint loss for optimization.

[0051] in The true labels are one-hot encoded (preliminarily generated through color clustering). This represents the corresponding probability distribution. Spatial smoothness constraints are introduced in iterative optimization: if the superpixel... With neighboring superpixels The difference in category predictions is greater than the threshold =0.3, then force update The category is the neighborhood majority category, which suppresses isolated noise points.

[0052] (3) Text detection in label regions: Instead of running OCR on the entire image, it is first performed in each semantic region after segmentation. For text regions (such as stores), they are first converted into grayscale images and binarized to form a clear text-background contrast. Then, a lightweight PP-OCR model is run on them for text recognition, which greatly improves the detection accuracy in complex backgrounds.

[0053] 3. Semantic-driven topology map construction The vertices of a topology map should not be simple geometric points, but rather key locations with rich semantic information that are meaningful for navigation decisions. This application constructs topology nodes based on the identified semantic entities through visibility analysis.

[0054] Node selection and semantic node generation: The center point of all identified semantic entities (such as shops) constitutes the initial set of semantic nodes. For each of these nodes, a given radius is used as the center. Within, identify all visible semantic nodes and form a set. Calculate the set The centroid of the polygon formed Treat the centroid as a topological node This method ensures that topological nodes are located at the "center" of multiple semantic landmarks, serving as natural decision points. Intersections and endpoints are extracted from the skeleton of passageways such as corridors to serve as supplementary support nodes, guaranteeing the connectivity of the topological graph.

[0055] Edge construction: For any two topology nodes and Determine if a passable path exists between them (based on the segmented passable area mask). If it exists, create an edge. Edge weights: weights The calculation formula is:

[0056] in and These are the connecting segments. The number of pixels traversed through both impassable and passable areas. This weight also considers Euclidean distance (reflected in...). ) and the difficulty of passage (penalty for going through walls, reflected in ).

[0057] 4. Semantic Clue-Based Reasoning and Self-Localization When an agent observes a landmark (such as "Starbucks"), it may be located near all "Starbucks" tags on the map. DS evidence theory provides a rigorous mathematical framework to fuse multiple such uncertain observational evidences into a unified confidence distribution.

[0058] Recognition and Projection: The agent observes a set of landmarks. For each observed landmark... On the map Find the semantic label of all vertices. All matched vertices constitute a candidate vertex set. .

[0059] Basic probability assignment (BPA): For each candidate vertex According to the observed landmarks With map vertices Assign a basic confidence level to the spatial distance between them. The closer the distance, the higher the confidence level; Gaussian kernel function is typically used for calculation.

[0060] Evidence fusion: An agent may observe multiple landmarks at time t, each landmark providing a body of evidence. These pieces of evidence are fused into a comprehensive confidence assignment using Dempster's combination rule.

[0061] Confidence score and pose estimation: The fused confidence score assignment *m* gives the probability that the agent is located at each vertex (or location region). Final pose estimation. The position corresponding to the highest probability value can be taken, or the expected value of all positions.

[0062] 5. Hierarchical navigation planning and control Drawing on the human concept of "macro-planning-micro-execution," the navigation task is decomposed into two levels: global topology path planning and local reactive control. Accumulated errors are corrected through periodic semantic relocation.

[0063] Global topology planning: Given a starting point and target point In the topology diagram G The corresponding node in the data is represented using Dijkstra's algorithm or... The algorithm calculates an optimal sequence of vertices. This sequence indicates "which key decision points will be passed next".

[0064] Local vector navigation: for each segment of the global path Calculate its steering vector :

[0065] Characterizes the nodes The local planner of the location agent will take the current pose as the starting point, use the direction of the vector as the guide, and combine it with a real-time obstacle avoidance algorithm to generate a smooth and safe local path and execute it.

[0066] Loop Closure and Relocalization: When the agent anticipates reaching the next topological node based on vector navigation, it actively triggers DS localization based on semantic observation. The localization result is compared with the odometry-based predicted pose. If the deviation is too large, the localization result is used for correction, thereby eliminating accumulated errors and achieving loop closure. After correction, the agent starts from the new pose and executes vector navigation to the next node, repeating this cycle until the destination is reached.

[0067] The visualization effect of this algorithm is provided by Figure 6 The given solution, along with the navigation comparison scheme, is the ROS2 Nav2 native algorithm, only... Macro-planning and stochastic exploration strategies were experimented with in the Gazebo simulation environment. A large shopping mall .world file was generated based on a custom CAD model, containing five floors, main shop areas, and multiple connecting corridors. The comparison results are shown in Table 1. Table 1 Navigation Experiment Results

[0068] This application provides a complete, efficient, and lightweight indoor cognitive navigation solution. Its innovation lies in fully exploring and utilizing the valuable resource of readily available planar navigation maps within the indoor environment. Through a series of formal definitions and computational models (UNET, Superpixel, CNN, DS evidence theory, and topological graph theory), it successfully transforms human cognitive navigation strategies into machine-executable algorithmic flows. This not only solves the problems of high computational cost and reliance on high-precision sensors inherent in traditional geometric methods, but also overcomes the node ambiguity challenge in topological methods through semantic reasoning, ultimately achieving intelligent navigation with high success rate and low resource consumption in large and complex indoor environments.

[0069] In summary, this application has the following beneficial effects: 1. Achieves extremely high resource efficiency and lightweight deployment: Unlike traditional SLAM technology that relies on dense point cloud maps and massive computing resources, this invention utilizes widely existing indoor planar guide views as prior knowledge. By constructing a semantic topological map, the environment is abstracted into symbolic representations of nodes and edges. This map has extremely small data volume, eliminating the need for expensive LiDAR or multi-sensor fusion, significantly reducing the demand for hardware computing power. This allows the solution to run efficiently on embedded platforms or mobile devices with limited computing resources, laying the foundation for the widespread adoption of lightweight indoor intelligent agents.

[0070] 2. Demonstrates superior navigation performance and success rate: Through a hierarchical navigation framework and semantic reasoning localization, the system successfully simulates the human cognitive decision-making process. Global topology planning ensures path optimization, while local semantic anchor-based vector navigation and DS evidence theory fusion localization endow the agent with strong anti-interference and error correction capabilities. Experimental data shows that in a simulated large shopping mall environment, this method achieves a navigation success rate of up to 92%, with an average path planning and execution time reduction of 20%, significantly outperforming traditional methods. Planning, ROS standard navigation, and exploration-based methods have demonstrated its practicality and robustness in complex, dynamic environments.

[0071] 3. Breakthrough in the fusion of semantic and spatial information: The core innovation of this invention lies in fully mining and utilizing the rich semantic information in the wayfinding view that is overlooked by traditional navigation systems. Through innovative unsupervised segmentation, regionalized OCR detection, and semantic-driven topology construction technologies, visual elements (such as text, icons, and color regions) are successfully transformed into structured knowledge that machines can understand and reason about. This enables the intelligent agent not only to "see" the environment but also to "understand" it, achieving true cognitive navigation and providing key technical support for the natural interaction and efficient task execution of embodied intelligence in human living spaces.

[0072] This disclosure also provides an embodied intelligent agent cognitive navigation system guided by an indoor floor plan, including: The guide view image correction unit uses a semantic segmentation model based on the UNET architecture to process the input image, accurately segment the region where the guide view is located, and generate a binary mask; based on the obtained mask, a perspective correction algorithm is executed to obtain the corrected guide view image. Adaptive semantic segmentation and annotation unit: performs superpixel pre-segmentation and semantic classification on the corrected guide view image, adopts a divide-and-conquer strategy for text detection, and transforms the heterogeneous semantic information in the corrected guide view into structured data with accurate semantic labels; Semantic-driven topology map building unit: Determine topology nodes based on structured data, connect each topology node to the nearest walkable node, and obtain a topology map; The semantic cue-based reasoning and self-localization unit processes the agent's real-time visual observations, extracts observed semantic landmarks, filters candidate semantic nodes in a pre-constructed semantic topology map for each observed landmark and assigns basic probability values, fuses the basic probability values ​​of multiple semantic landmarks to generate the confidence distribution of the agent's position, minimizes the difference between the vectors of the observed landmarks and the agent and the vectors of the landmarks and their corresponding nodes in the prior map, aligns the observation configuration with the map structure, and corrects perception errors. Hierarchical navigation planning and control unit: Calculates the shortest path from the starting point to the target point on the topology map. For any two adjacent topology nodes on the path, calculates the connectivity vector, distance, and orientation angle between them, and generates a vector sequence. The agent moves according to the vector sequence.

[0073] Regarding the system in the above embodiments, the specific manner in which each unit performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0074] This disclosure also provides an embodied intelligent agent cognitive navigation system guided by an indoor floor plan. Please refer to [link / reference]. Figure 2 and Figure 5 It mainly includes the following modules: (1) Environmental Perception and Semantic Analysis Module: This module is the foundation of the system and is responsible for processing and understanding the input raw guide view. The Environmental Perception and Semantic Analysis Module includes a perception module and a semantic analysis module. The perception module uses the guide view image correction unit to detect and correct perspective distortion caused by shooting angle and lighting, providing standardized input for subsequent processing. Secondly, the semantic analysis module is responsible for parsing heterogeneous information in the guide view, segmenting the image into semantic regions with different functional meanings (such as shops, corridors, and stairs), and accurately detecting and recognizing text labels (such as shop names) in the image. Finally, the output of the Environmental Perception and Semantic Analysis Module is corrected structured map data with rich semantic labels.

[0075] (2) Cognitive Map Construction and Management Module: The cognitive map construction and management module is the "brain" of the system, responsible for transforming the parsed semantic information into a structured knowledge representation that can be understood and used by the intelligent agent. Its core is to construct a semantic-driven topological map. This map is centered on semantic nodes (such as shop entrances, key intersections), and the nodes are connected by edges. The weight of the edges combines physical distance and passage difficulty (such as the penalty value for crossing walls). This representation method greatly compresses the storage space of the map (lightweight) while retaining key spatial relationships and semantic information, laying the foundation for high-level task planning and reasoning. The cognitive map construction and management module includes a global positioning module and a topology query module. The global positioning module is used to obtain the optimal neighboring topological nodes from the visual language model through scene reasoning and probability heatmaps. The topology query module is used to query the semantic nodes corresponding to the topological points.

[0076] (3) Navigation Planning Module: This module is the "cerebellum" of the system, responsible for executing specific navigation tasks. It adopts a hierarchical decision-making framework to simulate human navigation strategies. The global planner operates on the topological map, calculating an optimal path sequence consisting of a series of semantic landmarks based on the target task (e.g., "go to the Apple Store"). The local planner is responsible for controlling the agent to move from one landmark to the next in real time. It relies on real-time visual perception to identify semantic anchors in the environment and compares them with the global plan. At the same time, a semantic reasoning localization unit based on Dempster-Shafer evidence theory continuously integrates multi-view observation information and prior map knowledge to estimate the agent's confidence pose online, effectively correcting the accumulated errors and drift during navigation.

[0077] It should be noted that although several modules of the system for executing actions are mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into multiple modules for embodiment. Components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0078] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for embodied intelligent agent cognitive navigation guided by an indoor floor plan, characterized in that, include: S1, Guide View Image Recognition and Correction: The input image is processed using a semantic segmentation model based on the UNET architecture to accurately segment the region where the guide view is located and generate a binary mask; Based on the obtained mask, a perspective correction algorithm is executed to obtain the corrected guide view image; S2, Adaptive Semantic Segmentation and Labeling: Superpixel pre-segmentation and semantic classification are performed on the corrected guide view image, and a divide-and-conquer strategy is used for text detection to transform the heterogeneous semantic information in the corrected guide view into structured data with accurate semantic labels. S3, semantic-driven topology map construction: determine topology nodes based on structured data, connect each topology node to the nearest walkable node, and obtain a topology map; S4, Semantic Clue-Based Reasoning and Self-Localization: Process the real-time visual observations of the agent, extract the observed semantic landmarks, for each observed landmark, filter candidate semantic nodes in the pre-constructed semantic topology map and assign basic probability values, fuse the basic probability values ​​of multiple semantic landmarks to generate the confidence distribution of the agent's position, minimize the difference between the vectors of the observed landmarks and the agent and the vectors of the landmarks and their corresponding nodes in the prior map, align the observation configuration with the map structure, and correct perception errors; S5, Hierarchical navigation planning and control: Calculate the shortest path from the starting point to the target point on the topology map. For any two adjacent topology nodes on the path, calculate the connection vector, distance and orientation angle between them, generate a vector sequence, and the agent moves according to the vector sequence.

2. The embodied intelligent agent cognitive navigation method guided by indoor floor plan according to claim 1, characterized in that, In S1, the step of performing a perspective correction algorithm based on the obtained mask to obtain the corrected guide view image includes: S11, Identify the largest quadrilateral region in the mask through connected component analysis; S12, calculate the deviation of each corner point of the quadrilateral from 90 degrees to estimate the degree of distortion; S13, calculate the perspective transformation matrix between the quadrilateral and a standard rectangle; S14, apply the transformation matrix to obtain the corrected guide view image.

3. The embodied intelligent agent cognitive navigation method guided by indoor floor plan according to claim 1, characterized in that, In S2, Felzenszwalb's graph theory algorithm is used to perform superpixel pre-segmentation on the corrected guide view image, and a lightweight encoder-decoder CNN is used for semantic classification.

4. The embodied intelligent agent cognitive navigation method guided by indoor floor plan according to claim 1, characterized in that, S3 include: S31, Based on the structured data, find the set of visible semantic nodes within a preset radius for each semantic node; S32, construct a polygon from the set of visible semantic nodes and calculate its centroid as the optimal topological node position; S33, extract the corridor area from the road mask, identify the skeleton intersections, and add midpoints as support nodes between the intersections and corners; S34 connects each topology node to the nearest passable node to obtain a topology map.

5. The embodied intelligent agent cognitive navigation method guided by indoor floor plan according to claim 1, characterized in that, S4 include: S41 processes the real-time visual observations of the agent and extracts the observed semantic landmarks and their spatial locations; S42, for each observed landmark, find its nearest preset number of candidate semantic nodes in the pre-built topological map using Euclidean distance; S43, assign basic probability values ​​to each candidate semantic node using a Gaussian kernel function; S44 uses Dempster's combination rule to fuse the basic probability assignments of multiple semantic landmarks to generate the confidence distribution of the agent's location; S44 minimizes the difference between the observed landmark and agent vectors and the vectors of landmarks and their corresponding nodes in the prior map, aligns the observation configuration with the map structure, and corrects perception errors.

6. The embodied intelligent agent cognitive navigation method guided by indoor floor plan according to claim 1, characterized in that, S5 include: S51. Use Dijkstra's algorithm to calculate the shortest path from the starting point to the target point on the topological graph. This path is a sequence of topological nodes. S52, for any two adjacent topological nodes on the path, calculate their connection vector, distance and orientation angle, and generate a vector sequence; S53, the agent uses a local motion planner to move following a sequence of vectors. Whenever it reaches a topological node, it performs semantic-based relocalization to correct the accumulated odometry drift and realign the estimated pose with the global map.

7. An embodied intelligent agent cognitive navigation system guided by an indoor floor plan, characterized in that, include: Guide view image recognition and correction unit: The input image is processed using a semantic segmentation model based on the UNET architecture to accurately segment the region where the guide view is located and generate a binary mask; Based on the obtained mask, a perspective correction algorithm is executed to obtain the corrected guide view image; Adaptive semantic segmentation and annotation unit: performs superpixel pre-segmentation and semantic classification on the corrected guide view image, adopts a divide-and-conquer strategy for text detection, and transforms the heterogeneous semantic information in the corrected guide view into structured data with accurate semantic labels; Semantic-driven topology map building unit: Determine topology nodes based on structured data, connect each topology node to the nearest walkable node, and obtain a topology map; The semantic cue-based reasoning and self-localization unit processes the agent's real-time visual observations, extracts observed semantic landmarks, filters candidate semantic nodes in a pre-constructed semantic topology map for each observed landmark and assigns basic probability values, fuses the basic probability values ​​of multiple semantic landmarks to generate the confidence distribution of the agent's position, minimizes the difference between the vectors of the observed landmarks and the agent and the vectors of the landmarks and their corresponding nodes in the prior map, aligns the observation configuration with the map structure, and corrects perception errors. Hierarchical navigation planning and control unit: Calculates the shortest path from the starting point to the target point on the topology map. For any two adjacent topology nodes on the path, calculates the connectivity vector, distance, and orientation angle between them, and generates a vector sequence. The agent moves according to the vector sequence.

Images