Robot exploration method based on heterogeneous topological graph and lightweight global reasoning

Abstract

The invention discloses a robot exploration method based on a heterogeneous topological graph and lightweight global reasoning, and relates to the technical field of robot exploration, and the method comprises the steps: obtaining the point cloud data of an environment; the point cloud data increment is updated to the occupancy probability grid, and then a probability grid map is constructed; constructing a heterogeneous topological graph by taking an open area or a target landmark in the environment as a vertex and taking connectivity occupying probability grids in the probability grid map as edges; inputting the heterogeneous topological graph into a strategy network to obtain a target viewpoint output by the strategy network; calculating a path of the robot from the current position to the target viewpoint according to the heterogeneous topological graph, and further generating a motion instruction corresponding to the path; and driving the robot according to the motion instruction. According to the method, by introducing the strategy network of the heterogeneous topological graph and the lightweight global reasoning architecture, the autonomous exploration performance and practicability of the robot are remarkably improved, and the exploration efficiency of the robot is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot exploration, and in particular to a robot exploration method based on a heterogeneous topological graph and lightweight global reasoning. BACKGROUND

[0002] An autonomous exploration task aims to enable a robot to automatically construct an environment map and achieve efficient navigation and positioning in an unknown or partially known environment by relying on its own perception and decision-making capabilities; it is a key basic technology in various application scenarios such as unmanned aerial vehicle surveying, warehouse logistics, and underground pipe network detection. At present, the mainstream methods can be roughly divided into two categories: one is a frontier-based method, which evaluates existing map information through frontier points, constructs a corresponding optimization objective, and solves the optimal path to visit all frontier points; the other is an information theory-based method, which takes mutual information or entropy reduction as the target, and selects an observation position or path that can maximize the reduction of map uncertainty.

[0003] However, the efficiency of the current robot exploration scheme is still low. SUMMARY

[0004] Therefore, the embodiments of the present application provide a robot exploration method based on a heterogeneous topological graph and lightweight global reasoning to improve the exploration efficiency of the robot.

[0005] An aspect of the embodiments of the present application provides a robot exploration method based on a heterogeneous topological graph and lightweight global reasoning, which comprises the following steps:

[0006] Obtain point cloud data of an environment;

[0007] Incrementally update the point cloud data into an occupancy probability grid, and then construct an occupancy probability grid map;

[0008] Take an open area or a target landmark in the environment as a vertex, and take the connectivity of the occupancy probability grid in the occupancy probability grid map as an edge, to construct a heterogeneous topological graph;

[0009] Input the heterogeneous topological graph into a policy network to obtain a target viewpoint output by the policy network;

[0010] Calculate a path of a robot from a current position to the target viewpoint according to the heterogeneous topological graph, and then generate a motion instruction corresponding to the path;

[0011] Drive the robot according to the motion instruction.

[0012] In some embodiments, the step of constructing a heterogeneous topological graph by taking an open area or a target landmark in the environment as a vertex and taking the connectivity of the occupancy probability grid in the occupancy probability grid map as an edge comprises the following steps:

[0013] selecting different types of vertices from the probability grid map classified into three types of cells, i.e., free, occupied, and unknown;

[0014] constructing edges between the vertices according to connectivity between the vertices;

[0015] performing a triangulation operation on all the vertices to filter out edges that do not meet feasibility or are determined to be redundant, thereby obtaining the heterogeneous topological graph;

[0016] assigning attribute vectors for decision-making to different types of vertices in the heterogeneous topological graph.

[0017] In some embodiments, the selecting different types of vertices from the probability grid map classified into three types of cells, i.e., free, occupied, and unknown, comprises the following steps:

[0018] extracting frontiers from free cells adjacent to unknown cells for marking an exploration frontier, and extracting boundaries from free cells adjacent to occupied cells for representing boundaries of a passable region;

[0019] calculating a Hamilton-Jacobi skeleton based on an Euclidean signed distance field (ESDF) of the probability grid map, and extracting skeleton points from the Hamilton-Jacobi skeleton;

[0020] calculating viewpoints that can maximize observation of unknown regions;

[0021] wherein the frontiers, the boundaries, and the viewpoints are the vertices.

[0022] In some embodiments, the assigning attribute vectors for decision-making to different types of vertices in the heterogeneous topological graph comprises the following steps:

[0023] assigning corresponding two-dimensional spatial coordinates to the frontiers and the boundaries in the heterogeneous topological graph as a first attribute vector;

[0024] assigning a second attribute vector to the skeleton points in the heterogeneous topological graph; wherein the second attribute vector comprises two-dimensional spatial coordinates of the skeleton points and the attribute vector based on ESDF clearance distance information;

[0025] assigning a third attribute vector to the viewpoints in the heterogeneous topological graph; wherein the third attribute vector comprises two-dimensional spatial coordinates of the viewpoints and potential information gain that the viewpoints can bring to exploration.

[0026] In some embodiments, the inputting the heterogeneous topological graph into a policy network to obtain a target viewpoint output by the policy network comprises the following steps:

[0027] performing type-specific multi-layer perceptron mapping on an attribute vector of each of the vertices in the heterogeneous topological graph to obtain an initial embedding;

[0028] An expression of the initial embedding is:

[0029] ;

[0030] wherein, the initial embedding is, the attribute vector is, a vertex belongs to a type, is the multi-layer perceptron of the corresponding type;

[0031] performing linear transformation on the initial embedding respectively to obtain a query matrix , a key matrix , and a value matrix :

[0032] ;

[0033] wherein, a Frobenius norm is used to normalize as a whole;

[0034] constructing a diagonal normalization matrix :

[0035] ;

[0036] wherein, a full one vector is, a vector is put into a diagonal to form a diagonal matrix;

[0037] calculating a first global node-level embedding :

[0038] ;

[0039] wherein, is a hyperparameter used to balance global attention output and residual connection;

[0040] fusing the first global node-level embedding with a local graph convolution result based on an adjacency matrix to obtain a second global node-level embedding;

[0041] ; ​

[0042] wherein, is the second global node-level embedding; is a fusion coefficient, denotes a local graph convolution operation with as input and as the adjacency matrix;

[0043] a global graph-level embedding is obtained according to the current point embedding as query input and the second global node-level embedding;

[0044] ;

[0045] wherein, is the global graph-level embedding; is the current point embedding, ;

[0046] a candidate viewpoint is selected as a viewpoint whose observation value for the unknown region is greater than a preset threshold according to an action mask of the viewpoint;

[0047] a correlation score between the global graph-level embedding and an embedding of each candidate viewpoint is calculated;

[0048] ;

[0049] wherein, is the correlation score, denotes a set of each candidate viewpoint;

[0050] a softmax normalization is applied to each correlation score to obtain an action and a target viewpoint corresponding to the action.

[0051] In some embodiments, the method further comprises the following steps:

[0052] an expected return of the current state of the robot is calculated according to the global graph-level embedding by using a multi-layer perception.

[0053] In some embodiments, the method further comprises a step of training an exploration strategy of the policy network by using a proximal policy optimization algorithm, and the step of training an exploration strategy of the policy network by using a proximal policy optimization algorithm comprises the following steps:

[0054] in phase 1 of training, an action is corresponded to a complete viewpoint traversal to simplify the action space and stabilize the initial value estimation;

[0055] in phases 2 to 3 of training, the decision frequency is gradually increased from 0.125 Hz to 0.5 Hz, so that the policy can smoothly adapt to more fine-grained control;

[0056] In phase 4 of the training, the 2 Hz frequency condition of the steering target is trained so as to be consistent with the operation rhythm of the robot.

[0057] Another aspect of the embodiment of the present application also provides a robot exploration device based on a heterogeneous topological graph and lightweight global reasoning, which comprises:

[0058] A point cloud acquisition unit is configured to acquire point cloud data of an environment.

[0059] A grid map construction unit is configured to incrementally update the point cloud data into an occupancy probability grid, and then construct a probability grid map.

[0060] A topological graph construction unit is configured to take an open area or a target landmark in the environment as a vertex, and take connectivity of the occupancy probability grid in the probability grid map as an edge, and construct a heterogeneous topological graph.

[0061] A target viewpoint determination unit is configured to input the heterogeneous topological graph into a policy network, and obtain a target viewpoint output by the policy network.

[0062] A motion instruction generation unit is configured to calculate a path of a robot from a current position to the target viewpoint according to the heterogeneous topological graph, and then generate a motion instruction corresponding to the path.

[0063] A driving unit is configured to drive the robot according to the motion instruction.

[0064] Another aspect of the embodiment of the present application also provides an electronic device comprising a processor and a memory.

[0065] The memory is configured to store a program.

[0066] The processor executes the program to implement the method of any one of the above aspects.

[0067] Another aspect of the embodiment of the present application also provides a computer readable storage medium, which stores a program, and the program is executed by a processor to implement the method of any one of the above aspects.

[0068] The present application at least has the following beneficial effects:

[0069] The application can obtain point cloud data of an environment, incrementally update the point cloud data into an occupancy probability grid, and then construct a probability grid map. An open area or a target landmark in the environment is taken as a vertex, and the connectivity of the occupancy probability grid in the probability grid map is taken as an edge, to construct a heterogeneous topology graph. The heterogeneous topology graph is input into a policy network to obtain a target viewpoint output by the policy network. A path from a current position of the robot to the target viewpoint is calculated according to the heterogeneous topology graph, and then a motion instruction corresponding to the path is generated. The robot is driven according to the motion instruction. The application introduces the heterogeneous topology graph and the policy network of the lightweight global reasoning architecture, significantly improves the performance and practicability of the autonomous exploration of the robot, and improves the efficiency of the robot exploration. BRIEF DESCRIPTION OF DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0071] Figure 1 A flowchart of a robot exploration method based on a heterogeneous topology graph and a lightweight global reasoning provided by an embodiment of the application is shown in the figure.

[0072] Figure 2 An example flowchart of a robot exploration method based on a heterogeneous topology graph and a lightweight global reasoning provided by an embodiment of the application is shown in the figure.

[0073] Figure 3 An example graph of a heterogeneous topology graph provided by an embodiment of the application is shown in the figure.

[0074] Figure 4 A flowchart of a scalable lightweight global reasoning scheme based on a heterogeneous topology graph provided by an embodiment of the application is shown in the figure.

[0075] Figure 5 An example graph of a simulation environment provided by an embodiment of the application is shown in the figure.

[0076] Figure 6 An example graph of a finally generated map and exploration path provided by an embodiment of the application is shown in the figure.

[0077] Figure 7 A structural block diagram of a robot exploration device based on a heterogeneous topology graph and a lightweight global reasoning provided by an embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0078] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application and not to limit the present application.

[0079] Before the embodiments of the present application are described in detail, first, some related technologies involved in the embodiments of the present application are described as follows:

[0080] Robot autonomous exploration task: In an unknown or partially known environment, the robot constantly perceives the surrounding information by relying on its own sensors, and automatically decides the next movement or observation strategy on this basis, so as to realize the process of efficiently and comprehensively covering the environment, constructing the environment map and maintaining the positioning accuracy of the robot.

[0081] Deep reinforcement learning (DRL): A machine learning method that combines deep neural networks with reinforcement learning algorithms. The network automatically extracts features from high-dimensional perception inputs, and continuously optimizes the strategy or value function through interaction with the environment, to realize end-to-end decision learning.

[0082] Course learning: By dividing the training process of the model into several stages from easy to difficult, the model is first allowed to quickly learn the basic strategy in low-frequency and simple tasks, and then gradually increases the decision frequency and complexity, and finally transitions to the real deployment scene.

[0083] Heterogeneous topology graph: A graph structure composed of multiple node and edge types, used to compactly express the semantic and connectivity of the environment space. In the exploration system, the nodes include the front points extracted from the intersection of known free space and unknown space, the boundary points of the edge contour that describes the walkable area between known free space and obstacles (occupancy space), the sparse skeleton points obtained after skeletonization of the occupancy grid map, and the key observation points further selected based on the front points and skeleton points.

[0084] Global linear attention mechanism: A lightweight attention mechanism that efficiently captures the global semantic dependency of any node in the graph structure. It aggregates information in a linear manner according to global relevance, thereby reducing the computational complexity from the original quadratic level to linear level while maintaining the long-range information capturing capability.

[0085] Graph convolution: A message passing operation based on graph, which extracts local topological structure information by weighting and aggregating the features of each node and its neighbor nodes according to the connection relationship of the graph.

[0086] Frontier-based exploration planners have been proven to be highly efficient in small-scale environments. Related Art 1 proposes to utilize frontier-driven exploration by sorting frontiers according to the path length from the current position to each frontier point and selecting the shortest one as the next goal. Related Art 2 further introduces "frontier size" as a utility metric to balance the amount of observable information when choosing a goal. Although these methods tend to focus more on short-term gains, possibly sacrificing long-range exploration considerations, they are still close to optimal in simple scenarios with a small number of frontiers. However, when the environment becomes large and the number of frontiers increases, such greedy strategies can suffer from "short-sightedness" and performance degradation.

[0087] In large-scale environments, it is challenging to optimize long trajectories of hundreds of meters online while maintaining fine-grained steps (less than 1 meter) to obtain high-resolution maps. For this reason, some advanced sampling strategies have been proposed to address large-scale exploration. Related Art 3 found that frontiers are sparse globally but dense locally, and thus constructed a hierarchical framework: the upper layer uses global frontier methods for planning, and the lower layer uses Related Art 4 for local optimization. Subsequent work developed a multi-resolution path planning strategy, further improving exploration efficiency and significantly outperforming simple frontier and sampling methods. In particular, Related Art 5 proposed a global planner combined with the Traveling Salesman Problem and a local sampling planner with coverage constraints, which performed best in balancing the accuracy of near and far exploration, and its efficiency was further improved.

[0088] Although some studies have attempted to use deep learning to predict the structure of unknown regions to assist traditional algorithms, most learning-based methods directly decide movements to achieve long-term exploration goals through deep reinforcement learning. Current DRL planners (Related Art 6) generally use convolutional neural networks (CNNs) to select path points from visual beliefs such as maps, but their performance in small-scale environments is only comparable to the simplest frontier method. Notably, the Ariadne framework proposed by Related Art 7 has surpassed traditional frontier methods and CNN-DRL methods in performance by learning attention mechanisms on graphs; however, like other DRL methods, it is difficult to extend to large-scale scenarios.

[0089] Although many existing methods have been validated in real-world environments and have shown great application potential, there are still certain technical limitations, mainly in the following aspects:

[0090] Suboptimality problem: Traditional greedy and heuristic methods do not consider the time dimension when constructing optimization problems, so the paths solved are suboptimal, which can cause serious path "backtracking" in complex large-scale scenarios; similarly, for existing DRL methods, the lack of proper feature extraction and modeling of the environment can also lead to suboptimal learned strategies.

[0091] Scalability problem: traditional methods often design and test algorithms on a few maps without considering the distribution of various terrains in real environments, and when migrating to new maps, they fail due to various boundary conditions; on the other hand, existing DRL methods fail to migrate to large-scale environments due to the excessive redundancy of model design, resulting in quadratic growth of computational complexity.

[0092] Real-time performance challenge: existing methods face challenges in real-time deployment in large-scale environments due to the inability to efficiently represent sparse environments and the high complexity of decision-making algorithms.

[0093] By solving the core problems of existing methods, the present application aims to propose and implement a more efficient and optimal robot autonomous exploration system, bringing innovative solutions to the field of intelligent automation. The present application has the following characteristics:

[0094] Optimality: the present application models the autonomous exploration task as a Markov process, uses path cost as the only daily reward, and accurately models observations through a heterogeneous topology graph. The optimal coverage path is solved through reinforcement learning, and the learned strategy has optimality.

[0095] Scalability and generalization: the present application introduces a dynamically constructed topology graph representation and a lightweight global reasoning model, allowing the strategy to be extended to different distributions and larger-scale maps. Experiments show that the method can handle complex and diverse environments without relying on specific environments.

[0096] Lightweight: the present application constructs a minimal global reasoning model for exploration tasks, replacing the multi-step analysis, modeling, and solving process in traditional methods, allowing a decision-making frequency of 2Hz on low-power platforms in large-scale scenarios.

[0097] Reference Figure 1 The embodiment of the present application provides a robot exploration method based on a heterogeneous topology graph and a lightweight global reasoning, which specifically includes the following steps S100-S150:

[0098] S100: obtaining point cloud data of an environment;

[0099] S110: incrementally updating the point cloud data into an occupancy probability grid, and then constructing a probability grid graph;

[0100] S120: taking an open area or a target landmark in the environment as a vertex, and the connectivity of the occupancy probability grid in the probability grid graph as an edge, to construct a heterogeneous topology graph;

[0101] S130: inputting the heterogeneous topology graph into a policy network to obtain a target viewpoint output by the policy network;

[0102] S140: calculating a path of the robot from a current position to the target viewpoint according to the heterogeneous topological graph, and generating motion instructions corresponding to the path;

[0103] S150: driving the robot according to the motion instructions.

[0104] Optionally, the heterogeneous topological graph is constructed by taking open areas or target landmarks in the environment as vertices and connectivity of the occupancy probability grid in the probability grid map as edges, and includes the following steps:

[0105] selecting different types of the vertices from the probability grid map classified into three types of cells, i.e., free, occupied and unknown;

[0106] constructing the edges corresponding to the vertices according to the connectivity between the vertices;

[0107] performing a triangulation operation on all the vertices, filtering the edges that do not meet the feasibility or are determined to be redundant, and obtaining the heterogeneous topological graph;

[0108] assigning attribute vectors for decision-making to different types of the vertices in the heterogeneous topological graph.

[0109] Optionally, the selecting different types of the vertices from the probability grid map classified into three types of cells, i.e., free, occupied and unknown, includes the following steps:

[0110] extracting frontier points from free cells adjacent to unknown cells for marking an exploration frontier, and extracting boundary points from free cells adjacent to occupied cells for representing boundaries of passable areas;

[0111] calculating a Hamilton-Jacobi skeleton based on a Euclidean signed distance field of the probability grid map, and extracting skeleton points from the Hamilton-Jacobi skeleton;

[0112] calculating a viewpoint that can maximize observation of unknown areas;

[0113] wherein the frontier points, the boundary points and the viewpoint are the vertices.

[0114] Optionally, the assigning attribute vectors for decision-making to different types of the vertices in the heterogeneous topological graph includes the following steps:

[0115] assigning corresponding two-dimensional spatial coordinates to the frontier points and the boundary points in the heterogeneous topological graph as first attribute vectors;

[0116] assigning a second attribute vector to the skeleton point in the heterogeneous topological graph; wherein the second attribute vector comprises the attribute vector of the two-dimensional spatial coordinates and the clearance distance information based on ESDF of the skeleton point;

[0117] assigning a third attribute vector to the viewpoint in the heterogeneous topological graph; wherein the third attribute vector comprises the two-dimensional spatial coordinates of the viewpoint and the potential information gain that the viewpoint can bring to exploration.

[0118] Optionally, the step of inputting the heterogeneous topological graph into a policy network to obtain a target viewpoint output by the policy network comprises the following steps:

[0119] performing type-specific multi-layer perceptron mapping on the attribute vector of each vertex in the heterogeneous topological graph to obtain an initial embedding;

[0120] The expression of the initial embedding is:

[0121] ;

[0122] wherein, is the initial embedding, is the attribute vector, indicates the type to which the vertex belongs, is the multi-layer perceptron corresponding to the type; and

[0123] performing linear transformation on the initial embedding respectively to obtain a query matrix , a key matrix and a value matrix :

[0124] ;

[0125] wherein, indicates the Frobenius norm, and is used for overall normalization of ;

[0126] constructing a diagonal normalization matrix :

[0127] ;

[0128] wherein, indicates an all-one vector, indicates putting a vector into a diagonal to form a diagonal matrix;

[0129] calculating a first global node-level embedding :

[0130] ;

[0131] wherein, is a hyper-parameter for balancing global attention output and residual connection;

[0132] fusing the first global node-level embedding with a local graph convolution result based on an adjacency matrix to obtain a second global node-level embedding;

[0133] ;

[0134] wherein, is the second global node-level embedding; is a fusion coefficient, denotes an updated result of a local graph convolution operation with as input and as the adjacency matrix;

[0135] obtaining a global graph-level embedding according to a current point embedding as a query input and the second global node-level embedding;

[0136] ;

[0137] wherein, is the global graph-level embedding; is the current point embedding, ;

[0138] selecting, according to an action mask of a viewpoint, the vertexes with observation values greater than a preset threshold for unknown regions as candidate viewpoints;

[0139] calculating a correlation score between the global graph-level embedding and an embedding of each of the candidate viewpoints;

[0140] ;

[0141] wherein, is the correlation score, denotes a set of the candidate viewpoints;

[0142] applying softmax normalization to each of the correlation scores to obtain an action and the target viewpoint corresponding to the action.

[0143] Optionally, the method further comprises the following steps:

[0144] calculating an expected return of the current state of the robot according to the global graph-level embedding by using a multi-layer perception machine.

[0145] Optionally, the method further comprises a step of training an exploration policy of the policy network using a proximal policy optimization algorithm, the step of training an exploration policy of the policy network using a proximal policy optimization algorithm comprising the steps of:

[0146] In phase 1 of training, the action is corresponded to complete viewpoint traversal, so as to simplify the action space and stabilize the initial value estimation;

[0147] In phases 2 to 3 of training, the decision frequency is gradually increased from 0.125 Hz to 0.5 Hz, so that the policy can smoothly adapt to more fine-grained control;

[0148] In phase 4 of training, the training turns to the 2 Hz frequency condition of the target, so as to be consistent with the operation rhythm of the robot.

[0149] Next, the scheme of the embodiments of the present application will be described and explained in detail in combination with specific application examples.

[0150] I. Problem modeling

[0151] The autonomous exploration task aims to find a shortest and safe path to cover all passable areas in an unknown environment, and constantly incrementally update the belief map through sensors. The present embodiment formalizes this task into a partially observable Markov decision process in reinforcement learning, which is defined by a tuple , wherein:

[0152] represents the state space, containing the complete space information of the environment;

[0153] represents the observation space, and the robot obtains partial observations through sensors such as laser radars;

[0154] is a discrete action space, corresponding to an optional exploration viewpoint;

[0155] describes the probability of transitioning to state after executing action in state ;

[0156] is an observation model, depicting the mapping relationship between the real state and the sensor observation ;

[0157] is a reward function, used to evaluate the exploration process;

[0158] ​ is a discount factor to balance the weight of immediate reward and future reward.

[0159] represent the environment as a topological graph to narrow the gap between simulation and reality when processing raw sensor data. State corresponds to a global topological map capturing the complete spatial information, nodes represent key attributes (e.g. frontier value, feasibility measure), edges then represent feasible paths, thus efficiently encoding the environment structure. Observation at each decision is the current topological map constructed by the robot based on partial knowledge; Action space is then composed of selecting the next viewpoint to visit, simplifying the planning process compared to dense grid maps. The reward function combines path cost and completion reward:

[0160] ;

[0161] where is the path cost, is the reward for completing the exploration. Parameters satisfy to ensure that the exploration goal is prioritized over merely minimizing path length, denotes the truncated horizon of sampling or decision making.

[0162] II. Method flow.

[0163] Figure 2 is the overall flow of the method of the present embodiment, comprising the following steps:

[0164] 1. Obtain point cloud data through sensors and incrementally accumulate it into an occupancy probability grid. Upon arrival of each new scan, the system fuses the measurement with existing grid information using a Bayesian probability update mechanism to calculate the confidence of each grid cell being "free" or "occupied", thus constructing a dense probability map reflecting the environment structure and uncertainty. (Point cloud → probability grid map).

[0165] 2. Extract a sparse topological graph from the dense probability grid map to narrow the gap between simulation and reality and improve the efficiency of subsequent reasoning. In this process, open areas or target landmarks are identified as graph nodes, and feasible path edges between nodes are established according to the connectivity of free space in the grid, thus obtaining a topological representation that is both compact and retains spatial semantics. (Probability grid map → heterogeneous topological graph).

[0166] 3. The current generated topology graph is passed as input to the agent, i.e. the policy network. The network analyzes the features of each node in the graph (e.g. connectivity, frontier value, etc.), evaluates the exploration value of different positions in a global view, and finally outputs a high-level viewpoint goal, indicating the most valuable observation position for the robot in the next step. (Heterogeneous topology graph → policy network).

[0167] 4. After obtaining the goal, the corresponding viewpoint is set as the short-term subgoal of the local planner. Based on the existing topology graph information, the local planner calculates the short-range path from the current position of the robot to the viewpoint, generates the corresponding speed instructions, and ensures that the robot can both avoid obstacles and maintain a smooth and executable motion trajectory during the movement to the target. (Policy network → local planner).

[0168] 5. The linear and angular velocity instructions output by the local planner are sent to the simulation or real robot chassis model. On the differential drive platform, the robot drives the motor according to these motion instructions to complete the whole process of autonomous exploration task from the original point cloud perception to the actual execution. (Local planner → simulation model).

[0169] Three, heterogeneous topology graph.

[0170] Figure 3 An example of the heterogeneous topology graph provided for this embodiment.

[0171] In order to retain rich environmental information while minimizing redundant nodes, this embodiment represents the environment as a heterogeneous topology graph , where V represents the set of vertices, E represents the edges connecting these vertices , and T represents the predefined vertex types. Each vertex is associated with an attribute vector customized for its specific role. In this embodiment, we only focus on distinguishing different vertex types, and do not consider edge attributes.

[0172] 1. Vertex extraction.

[0173] Starting from a two-dimensional occupancy grid map that has been classified into three types of cells: "free", "occupied", and "unknown", this embodiment extracts key vertices and constructs a heterogeneous graph through a "three-step walk": defining boundaries, extracting skeletons, and inserting viewpoints.

[0174] Define boundaries: extract frontier points from the free cells adjacent to the unknown cells to mark the exploration frontier; extract boundary points from the free cells adjacent to the occupied cells to represent the boundaries of the passable area.

[0175] Skeleton extraction: Hamilton-Jacobi skeleton is first computed based on the Euclidean Signed Distance Field (ESDF) of the grid map (as shown in Figure 3 ), and then skeleton points are extracted from it, which characterize the main passages in free space.

[0176] Insertion viewpoints: following the classic frontier point exploration strategy, this embodiment computes viewpoints that can maximize the observation of unknown regions, so as to strategically select key observation positions. This structured process not only ensures the compactness of the topological graph, but also improves the semantic clarity, which is helpful for subsequent efficient exploration planning.

[0177] 2. Edge construction.

[0178] After the vertex set is determined, this embodiment needs to establish the edge set of each region in the connected environment. Then, the triangulation operation is performed on all vertices, and the edges that do not meet the feasibility or are judged to be redundant are filtered, so as to obtain a sparse topological structure that can reflect the connectivity of the environment and remove redundant connections.

[0179] 3. Vertex attributes.

[0180] Different types of vertices are assigned a simple and decision-oriented attribute vector :

[0181] Frontier points and boundary points only store two-dimensional spatial coordinates , which are sufficient to mark the transition position to the unknown or obstacle boundary.

[0182] Skeleton points contain clearance distance information based on ESDF in addition to coordinates , which is used to indicate the required spatial width through the passage.

[0183] Viewpoints encode observation utility as , where represents the potential information gain that the viewpoint can bring to exploration.

[0184] Through the above targeted attribute design, the topological graph enriches the environment expression without introducing redundant data, effectively enhancing the quality of exploration decision and planning efficiency.

[0185] Four, lightweight global reasoning architecture.

[0186] 1. Architecture overview.

[0187] Referring to Figure 4 , the embodiment proposes a scalable, heterogeneous topology graph-based lightweight global reasoning scheme to support robot exploration tasks. The embodiment scheme adopts an encoder-decoder structure:

[0188] Encoder: combines linear global attention mechanism and local graph convolution to efficiently encode global and local spatial context. The linear attention mechanism used reduces the computational complexity from traditional to , thus enabling the capture of information essential to long-range dependencies on large-scale graph structures.

[0189] Decoder: integrates these embedding representations and adopts an attention-based action selection strategy, while constraining the available actions through a viewpoint mask to ensure robust and efficient decision-making in complex environments.

[0190] The overall design not only achieves linear scalability but also significantly improves reasoning quality, enabling better performance for robots exploring large-scale, complex environments.

[0191] 2. Encoder.

[0192] First, the attribute vector of each vertex is mapped through a type-specific multi-layer perceptron (MLP):

[0193] ;

[0194] where denotes the type to which the vertex belongs, is the MLP corresponding to that type. The resulting embedding matrix lays the foundation for subsequent calculations.

[0195] To efficiently capture the global interactions between nodes in the graph, the embodiment adopts a linearized global attention mechanism. Traditional softmax attention requires the construction of an attention matrix of size , with a computational complexity of . In contrast, linear global attention can complete the same global information fusion in time, making it particularly suitable for large-scale graphs. The specific scheme is as follows:

[0196] The initial (node-level) embedding is transformed through linear transformations to obtain the query matrix , key matrix , and value matrix :

[0197] ;

[0198] where denotes the Frobenius norm (standard norm of a matrix) for overall normalization of .

[0199] A diagonal normalization matrix is constructed as

[0200] ;

[0201] where denotes an all-one vector, denotes putting a vector into a diagonal to form a diagonal matrix.

[0202] The global (node-level) embedding is computed as

[0203] ;

[0204] where is a hyper-parameter to balance the global attention output and the residual connection. This step can capture long-range dependencies and preserve the initial feature distribution of the input, thus achieving efficient global information aggregation on large-scale graphs.

[0205] To take into account the local structure constraints of the graph while reasoning globally, the global embedding obtained in the previous step needs to be fused with the local graph convolution (GCN) result based on the adjacency matrix . The following simple and efficient weighting scheme is adopted in this embodiment:

[0206] ;

[0207] where is the fusion coefficient, denotes the update obtained by performing graph convolution operation with as input and as the adjacency matrix. Finally contains both long-range and local structure information, providing rich and reliable feature representation for the subsequent decoder to generate action decisions.

[0208] 3. Decoder.

[0209] The goal of the decoder is to generate a global (graph-level) embedding from the node-level embedding . To intuitively aggregate global information, a global attention mechanism is adopted in this embodiment, where the current point embedding of the robot is taken as the query input:

[0210] ;

[0211] The embodiment effectively summarizes the context information of the entire topology graph by weighting each node according to its relevance to the current position of the robot, retaining key structures and extracting semantic clues, thereby providing an important basis for subsequent decision-making. The graph-level embedding obtained will be used by the policy head to generate the next action and by the value head to estimate the expected return, thereby ensuring the consistency and effectiveness of the exploration policy.

[0212] 4. Action selection and value estimation.

[0213] After obtaining the graph-level embedding of the decoder output , the embodiment uses a single-head attention mechanism to determine the next action. Specifically, the embodiment calculates the relevance scores between the embedding and each candidate viewpoint:

[0214] ;

[0215] where represents a set of candidate viewpoints. Then, by applying softmax normalization to these relevance scores, the action can be obtained. In order to improve learning efficiency and alleviate the problem of over-compression commonly seen in high-density graphs, the embodiment introduces a viewpoint-based action mask. Unlike traditional adjacency-based masks, the action mask of a viewpoint only retains those vertices that are considered to have important observation value for unknown regions, thereby obtaining a compact and semantically meaningful action set. Since decisions are only made between key viewpoints with significant information gain, this targeted screening speeds up policy convergence and improves exploration effectiveness.

[0216] At the same time, based on the shared graph-level embedding, the embodiment also uses a lightweight multi-layer perceptron (MLP) as the value head to estimate the expected return of the current state. By having the policy head and the value head share the same global representation, the method can balance the pursuit of immediate action quality with long-term exploration performance, thereby improving policy consistency and sampling efficiency. Probability distribution.

[0217] 5. Reinforcement learning based on curriculum learning strategy.

[0218] This example uses the proximal policy optimization algorithm (PPO) to train the exploration policy, which is known for its stability and efficiency. To improve the stability of training and the generalization ability of the policy, this example uses a curriculum learning method to gradually increase the decision frequency during training. In the initial stage 1, the action corresponds to a complete view traversal, which can simplify the action space and stabilize the initial value estimation; then in stages 2-3, this example gradually increases the decision frequency from 0.125 Hz to 0.5 Hz, allowing the policy to smoothly adapt to more fine-grained control; finally, in stage 4, the training shifts to the high-frequency condition (2 Hz) of the target, which is highly consistent with the operation rhythm of the real robot. This structured curriculum learning design significantly enhances the training convergence, robustness, and overall exploration effect.

[0219] More specific implementations are as follows:

[0220] This example is based on the PPO implementation of the CleanRL library to ensure the transparency and repeatability of the code base. Training and evaluation are performed on a server equipped with an AMD EPYC 7773X 64-core CPU and two NVIDIA RTX 4090 GPUs. The simulation is performed using MARSIM, a lightweight, point-based realistic simulation environment designed specifically for LiDAR-based systems. The policy network and value network share a universal encoder architecture based on heterogeneous graphs. Each node is represented by a 6-dimensional feature vector and embedded into a 64-dimensional latent space through two layers of MLP. This example enhances spatial generalization and stabilizes curriculum-based training through two-stage coordinate normalization. First, Min-Max scaling is applied to all coordinates based on a predefined bounding box, normalizing them to the [0, 1] range; second, the second normalization is achieved by converting absolute coordinates to a local coordinate system with the robot's current position as the reference. This dual normalization method alleviates the out-of-distribution problem when scaling the environment, improves sample efficiency, and accelerates policy transfer.

[0221] I. Simulation Environment

[0222] To evaluate exploration efficiency and versatility, this example is evaluated in three randomly generated simulation environments (random pillars, random barriers, and mazes, as shown in Figure 5Experiments are conducted in the scenes shown in Fig. 1. The complexity of these scenes is defined as the ratio of the shortest collision-free path length to the straight-line distance; the closer this ratio is to 1, the simpler the navigation, and the larger the ratio, the more detours caused by obstacles. This embodiment uses three indicators to quantify the performance difference between the evaluation methods: running time, exploration time, and moving distance. In all experiments, the robot is modeled as a differential drive vehicle with a maximum linear speed of 2 m / s and a linear acceleration of 2.5 m / s2, and the LiDAR sensor has a detection range of 40 m. The exploration process is terminated when the coverage reaches 95%. Each method is run 10 times from the same random starting position in each scene.

[0223] Table 1 records the results of the experiments, and the following embodiment analyzes them from the aspects of generality, exploration efficiency, and responsiveness:

[0224] Generality: This embodiment can stably complete the exploration task in different types of random environments (such as having pillars, partitions, or maze structures) without the need for additional parameter tuning for specific scenes. The use of a heterogeneous graph encoder to uniformly model local geometric information enables it to quickly adapt to changes in environment size and shape, and it has strong generalization ability.

[0225] Exploration efficiency: By modeling the exploration problem as an end-to-end sequence decision, this embodiment can consider global information when planning a path, significantly reducing exploration time and total moving distance. In multiple random experiments, the mean and variance of exploration time and moving distance remain at a low level, indicating high sample efficiency and stability under different starting points and environmental arrangements.

[0226] Responsiveness: The network structure of this embodiment is pruned and lightweight designed, and can achieve millisecond-level inference (decision time) for each step of decision-making on a single machine, basically meeting the real-time closed-loop control requirements. Compared with some solutions that require frequent global graph search or large-scale matrix operations, the inference process is more efficient and can adjust the flight path in time when the environment changes rapidly (such as the sudden appearance of obstacles).

[0227]

[0228] Table 1

[0229] II. Real environment

[0230] To verify the effectiveness and robustness of the proposed method in real scenes, this embodiment uses a wheeled biped robot equipped with a Mid-360 laser radar and an Orin NX onboard computer to conduct a large number of field experiments in indoor and outdoor environments. It is worth noting that the strategy trained in simulation can be directly deployed to the real environment without any fine-tuning or adaptation. All calculations are completed on the onboard device, and the dynamics constraints are set to and .

[0231] In the indoor scenario, the embodiment explores a large and complex floor spanning three interconnected buildings with a total area of approximately 2412.75 m². The robot starts from the edge of the unknown area without prior knowledge of the environment. The entire exploration process takes 629 s, and the length of the travel trajectory is 514.72 m. Figure 6 The final generated map and exploration path are shown, demonstrating the effectiveness of the embodiment scheme in covering complex indoor structures such as corridors, rooms, and cluttered areas.

[0232] In the outdoor scenario, the test area is significantly larger, with a total area of approximately 9846.50 m², including open areas, sidewalks, and buildings, typical campus elements. The detection process also takes 629 s, and the trajectory length is 351.93 m. The above figure shows the accurate and complete map generated by the embodiment method, verifying its reliable performance in challenging outdoor conditions.

[0233] Overall, these experiments in real environments highlight the scalability, efficiency, and zero-sample transfer performance of the proposed autonomous exploration method in realistic and complex environmental conditions.

[0234] The beneficial effects of the embodiment are:

[0235] Based on the prior art, the embodiment introduces a heterogeneous topological graph representation and a lightweight global reasoning architecture, combined with a curriculum learning-based deep reinforcement learning strategy, significantly improving the performance and practicality of the robot autonomous exploration system. The specific manifestations are as follows:

[0236] Optimality guarantee: The embodiment strictly models the exploration task as a partially observable Markov decision process, uses path cost as the main reward signal, and introduces complete global topological graph information, so that the reinforcement learning strategy can consider global coverage and path length minimization in end-to-end training. Compared with traditional greedy or heuristic algorithms that can only obtain suboptimal solutions, the learned strategy has a coverage path performance close to or reaching the global optimum, effectively reducing invalid detours and backtracking, and improving exploration quality.

[0237] Scalability and strong generalization ability: Through the conversion of dense probability grid map to sparse heterogeneous topological graph, the embodiment can keep the graph size controllable in large-scale or complex environments, avoiding the problem of excessive computational complexity of traditional grid-based or pure deep network methods. The introduction of two-stage coordinate normalization effectively alleviates the distributional shift problem caused by environmental scale expansion, so that the strategy can remain stable under different map resolutions and sizes.

[0238] Efficiency is significantly improved: this embodiment compresses the environment semantically through a heterogeneous topology graph, only selecting nodes related to the front, boundary, skeleton and key viewpoints, greatly reducing the information redundancy required for decision-making. Combined with a lightweight linear global attention and graph convolution fusion encoder, it can efficiently absorb local and global structure information, further guiding the strategy network to generate efficient exploration paths.

[0239] Real-time and high resource utilization: the lightweight global reasoning architecture reduces the traditional square complexity of attention calculation to linear complexity, enabling millisecond-level reasoning on large-scale graph structures. Compared with those solutions that require frequent global path search or large-scale matrix operation, this embodiment has faster reasoning speed, with a decision frequency of more than 2 Hz on low-power platforms such as Orin NX, meeting the real-time control requirements. The architecture only retains the minimum graph representation related to exploration, further reducing memory usage and computational power consumption, allowing deployment on low-power platforms and expanding application scenarios.

[0240] Robustness and practicality: in real indoor three interconnected buildings and outdoor typical campus scenes, this embodiment can stably complete the exploration task without any fine-tuning. Whether it is a complex indoor corridor and room structure, or an outdoor open area and building decoration environment, this embodiment has shown reliable mapping and positioning capabilities.

[0241] Reference Figure 7 The embodiment of the present application provides a robot exploration device based on a heterogeneous topology graph and a lightweight global reasoning, which comprises:

[0242] Point cloud acquisition data is used to acquire point cloud data of an environment;

[0243] The grid graph construction unit is used to incrementally update the point cloud data to the occupancy probability grid, and then construct the probability grid graph;

[0244] The topology graph construction unit is used to take the open area or target landmark in the environment as a vertex, and the connectivity of the occupancy probability grid in the probability grid graph as an edge, to construct a heterogeneous topology graph;

[0245] The target viewpoint determination unit is used to input the heterogeneous topology graph into a strategy network to obtain a target viewpoint output by the strategy network;

[0246] The motion instruction generation unit is used to calculate a path of a robot from a current position to the target viewpoint according to the heterogeneous topology graph, and then generate a motion instruction corresponding to the path;

[0247] The driving unit is used to drive the robot according to the motion instruction.

[0248] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0249] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flow chart of the present application are provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logical flows presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0250] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present application. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present application as set forth in the claims using ordinary techniques without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the appended claims and their equivalents.

[0251] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts of the prior art that make contributions or parts of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0252] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logic functions, which can be embodied in any computer readable medium for use by or in connection with an instruction execution system, apparatus or device, such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute the instructions, or in conjunction with these instruction execution systems, apparatus or devices. For the purpose of this specification, "computer readable medium" can be any device that can contain, store, communicate, propagate or transport programs for use by or in connection with an instruction execution system, apparatus or device, or in conjunction with these instruction execution systems, apparatus or devices.

[0253] More specific examples (non-exhaustive list) of the computer readable medium include the following: an electrical connection having one or more wires (electrical devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CD ROM). In addition, the computer readable medium can even be paper or other suitable medium on which the program can be printed, because the program can be obtained electronically, for example, by optical scanning of the paper or other medium, followed by editing, interpreting or otherwise processing, if necessary, in other suitable ways, to be stored in the computer memory, and then stored in the computer memory.

[0254] It should be understood that portions of the present application can be realized with hardware, software, firmware or a combination thereof. In the foregoing embodiments, a number of steps or methods can be realized as software or firmware to be executed by a suitable instruction-executing system. For example, if realized with hardware, and as in another embodiment, any one or a combination of the following technologies known in the art can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0255] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0256] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made thereto without departing from the principles and spirit of the present application, the scope of which is defined by the claims and their equivalents.

[0257] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the described embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present application, and these equivalent modifications or substitutions are included in the scope defined by the claims of the present application.

Claims

1. A robot exploration method based on heterogeneous topological graphs and lightweight global reasoning, characterized by: The method comprises the following steps: Get point cloud data of the environment; Incrementally updating the point cloud data into an occupancy probability grid, thereby constructing a probability grid map; Constructing a heterogeneous topological graph using open areas or target landmarks in the environment as vertices and the connectivity of the occupancy probability grid in the probability grid graph as edges; Inputting the heterogeneous topology graph into a policy network to obtain a target viewpoint output by the policy network; Calculating a path from a current position of the robot to the target viewpoint according to the heterogeneous topological graph, and then generating a motion instruction corresponding to the path; The robot is driven according to the motion instruction.

2. A robot exploration method based on heterogeneous topological graphs and lightweight global reasoning according to claim 1, characterized in that: The method of constructing a heterogeneous topological graph using open areas or target landmarks in the environment as vertices and the connectivity of the occupancy probability grid in the probability grid graph as edges includes the following steps: Selecting different types of vertices from the probability grid graph that has been classified as free, occupied, and unknown units; Constructing the edges between the corresponding vertices according to the connectivity between the vertices; Performing a triangulation operation on all the vertices, thereby filtering out the edges that do not meet feasibility requirements or are determined to be redundant, to obtain the heterogeneous topological graph; An attribute vector for decision-making is allocated to each of the vertices of different types in the heterogeneous topological graph.

3. The robot exploration method based on heterogeneous topological graphs and lightweight global reasoning according to claim 2, characterized in that: The step of selecting different types of vertices from the probability grid graph classified as free, occupied, and unknown units comprises the following steps: Extract frontier points from free cells adjacent to unknown cells to mark the exploration frontier; extract boundary points from free cells adjacent to occupied cells to represent the boundary of the traversable area; Calculating a Hamilton–Jacobi skeleton based on the Euclidean signed distance field of the probability grid map, and then extracting skeleton points from the Hamilton–Jacobi skeleton; Calculate the viewpoint that maximizes observation of the unknown area; The frontier point, the boundary point and the viewpoint serve as the vertices.

4. The robot exploration method based on heterogeneous topological graphs and lightweight global reasoning according to claim 3, characterized in that: The step of allocating an attribute vector for decision-making to each of the vertices of different types in the heterogeneous topological graph comprises the following steps: Assigning corresponding two-dimensional spatial coordinates to the frontier point and the boundary point in the heterogeneous topological graph as a first attribute vector; Assigning a second attribute vector to the skeleton point in the heterogeneous topological graph; wherein the second attribute vector includes the two-dimensional spatial coordinates of the skeleton point and the attribute vector based on the clearance distance information of the ESDF; A third attribute vector is assigned to the viewpoint in the heterogeneous topological graph; wherein the third attribute vector includes the two-dimensional spatial coordinates of the viewpoint and the potential information gain that the viewpoint can bring to exploration.

5. The robot exploration method based on heterogeneous topological graphs and lightweight global reasoning according to claim 1, characterized in that: The step of inputting the heterogeneous topology graph into a policy network to obtain a target viewpoint output by the policy network comprises the following steps: Performing type-specific multi-layer perceptron mapping on the attribute vector of each vertex in the heterogeneous topological graph to obtain an initial embedding; The initial embedding expression is: ； in, For the initial embedding, is the attribute vector, Represents a vertex The type of is the multilayer perceptron of the corresponding type; The initial embedding is transformed by linear transformation Get the query matrix , key matrix Sum Matrix : ； in, Represents the Frobenius norm, which is used to convert Do overall normalization; Construct a diagonal normalized matrix : ； in, represents a vector of all 1s, It means putting the vector into the diagonal to form a diagonal matrix; Compute the first global node-level embedding : ； in, is a hyperparameter used to balance the global attention output and residual connection; Fusing the first global node-level embedding with the local graph convolution result based on the adjacency matrix to obtain a second global node-level embedding; ； in, is the second global node-level embedding; is the fusion coefficient, Indicates For input, The update result obtained by performing a local graph convolution operation on the adjacency matrix; Based on the current point embedding as the query input and the second global node-level embedding, a global graph-level embedding is obtained; ； in, is the global graph-level embedding; is the embedding of the current point, ; Selecting the vertex whose observation value of the unknown area is greater than a preset threshold as a candidate viewpoint according to the action mask of the viewpoint; Calculating a correlation score between the global graph-level embedding and the embedding of each of the candidate viewpoints; ； in, is the correlation score, represents a set of each of the candidate viewpoints; Softmax normalization is applied to each of the related scores to obtain an action and the target viewpoint corresponding to the action.

6. The robot exploration method based on heterogeneous topological graphs and lightweight global reasoning according to claim 5, characterized in that: The method further comprises the following steps: A multilayer perceptron is used to calculate the expected reward of the robot's current state based on the global graph-level embedding.

7. A robot exploration method based on heterogeneous topological graphs and lightweight global reasoning according to any one of claims 1 to 6, characterized in that: The method further includes a step of training the exploration strategy of the policy network using a proximal policy optimization algorithm, wherein the step of training the exploration strategy of the policy network using the proximal policy optimization algorithm includes the following steps: In stage 1 of training, actions are mapped to complete viewpoint traversals to simplify the action space and stabilize the initial value estimates; In stages 2 and 3 of training, the decision frequency is gradually increased from 0.125 Hz to 0.5 Hz, allowing the policy to smoothly adapt to finer-grained control; In stage 4 of training, the 2 Hz frequency condition of the steering target was trained to match the operating rhythm of the robot.

8. A robotic exploration device based on heterogeneous topological graphs and lightweight global reasoning, characterized in that: The device comprises: Point cloud acquisition data, used to obtain point cloud data of the environment; A grid map construction unit, configured to incrementally update the point cloud data into an occupancy probability grid, thereby constructing a probability grid map; a topology map construction unit, configured to construct a heterogeneous topology map using open areas or target landmarks in the environment as vertices and the connectivity of the occupancy probability grids in the probability grid map as edges; a target viewpoint determining unit, configured to input the heterogeneous topology graph into a policy network to obtain a target viewpoint output by the policy network; a motion instruction generating unit, configured to calculate a path of the robot from a current position to the target viewpoint according to the heterogeneous topological graph, and then generate a motion instruction corresponding to the path; A driving unit is used to drive the robot according to the motion instruction.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images