Neural topology exploration method for multiple unmanned vehicles of unmeasured map

Abstract

The invention discloses a multi-unmanned-vehicle neural topology exploration method of an unmeasured map, is applied to the field of multi-unmanned-vehicle collaborative exploration, and aims at solving the problem of measurement map dependence in an existing multi-unmanned-vehicle collaborative exploration algorithm. In the mapping stage, environment data are collected through a panoramic RGB-D sensor, pure topological representation is directly constructed from visual features and depth information by applying a topological mapping algorithm based on path points, and dependence on a traditional measurement map is not needed; in the decision-making stage, firstly, a topological map structure is analyzed through a global decision-making module based on a graph neural network (GNN), and an optimal navigation target is intelligently determined; then, planning a path on the topological graph by adopting an A * algorithm and generating a local navigation target; and finally, the abstract target is converted into a specific execution action by the local decision module. In the execution stage, a self-adaptive escape mechanism is integrated, and the state of the unmanned vehicle is monitored in real time and a behavior strategy is intelligently adjusted.

Description

Technical Field

[0001] The present invention belongs to the field of multi-unmanned vehicle collaborative exploration technology, and in particular relates to a multi-unmanned vehicle neural topology exploration technology. Background Art

[0002] With the rapid development of reinforcement learning and robotics, collaborative exploration systems for multiple unmanned vehicles based on reinforcement learning have shown promising applications in disaster relief, military reconnaissance, and environmental monitoring. Compared to traditional algorithms, reinforcement learning can continuously and adaptively learn through interaction with the environment, eliminating the need for manual design of complex rules. Reinforcement learning methods based on deep neural networks have excellent generalization capabilities, can adapt to unprecedented environmental changes, and significantly improve the system's adaptability in real-world applications.

[0003] When autonomous vehicles explore unknown environments, building maps is crucial for expressing spatial structure. While traditional metric maps can accurately represent environmental geometry, they suffer from significant drawbacks in multi-AV systems, including high communication overhead, high computational resource consumption, strong sensor dependency, and weak scene generalization. Topological maps, with their simplified structure, efficient communication, and good environmental adaptability, have become an ideal alternative to metric maps for many exploration algorithms. However, existing multi-AV exploration algorithms often rely on metric maps to prune the topological map to improve accuracy. This results in the effectiveness of these algorithms being largely constrained by the accuracy of the metric map. Summary of the Invention

[0004] In order to solve the metric map dependency problem in existing multi-unmanned vehicle collaborative exploration algorithms and reduce algorithm complexity and computing resource consumption, the present invention proposes a metric map-free multi-unmanned vehicle neural topology exploration method.

[0005] The technical solution adopted by the present invention is: a neural topology exploration method for multiple unmanned vehicles on a metric-free map, comprising:

[0006] S1, receiving panoramic RGB-D image;

[0007] S2: Input the panoramic RGB-D image into the path point-based topology mapping module for mapping, and simultaneously receive the local topology maps of other unmanned vehicles and merge them into a global topology map;

[0008] S3. Extract the current and historical topology maps of the autonomous vehicle, master node, and virtual node from the global topology map, totaling six sub-topology maps.

[0009] S4. Input the sub-topology graph into the global decision module based on the graph neural network to determine the optimal global navigation target;

[0010] S5. Use the A* algorithm to calculate the optimal path from the current position of the unmanned vehicle to the global target, and select the local navigation target on the optimal path;

[0011] S6: Input the current RGB data and local navigation target of the unmanned vehicle into the local decision module based on the recurrent neural network, and output the specific execution action;

[0012] S7. When the exploration task is completed or the exploration time is over, the exploration ends; otherwise, the exploration returns to S1 to continue.

[0013] The execution action outputted in step S6 includes forward, left turn or right turn.

[0014] When the execution action output by step S6 is forward, it also includes calculating the displacement difference between the current position of the unmanned vehicle and the position at the previous moment. If the displacement difference is less than the threshold value of 0.001 meters, it is determined that the unmanned vehicle is stagnant and the execution action of the unmanned vehicle is changed to turn right; otherwise, step S7 is executed.

[0015] Beneficial effects of the present invention: The present invention applies a topological mapping algorithm based on path points to construct a pure topological representation directly from visual features and depth information, without relying on traditional metric maps, which significantly reduces computational complexity and storage requirements. Secondly, the topological map structure is analyzed by a global decision module based on a graph neural network to intelligently determine the optimal navigation target. At the same time, the abstract target is converted into a specific execution action based on the A* algorithm and recurrent neural network to achieve precise control. Thirdly, the present invention integrates an adaptive escape mechanism, which effectively prevents and solves the problem of local exploration stagnation by monitoring the status of the unmanned vehicle in real time and intelligently adjusting the behavior strategy, thereby greatly improving the exploration efficiency and robustness of the system in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Flowchart of a neural topology exploration method for multiple unmanned vehicles on a metric-free map according to an embodiment.

[0017] Figure 2 This is a flow chart of the topological mapping method based on path points in the present invention. DETAILED DESCRIPTION

[0018] To facilitate those skilled in the art to understand the technical content of the present invention, the present invention is further explained below with reference to the accompanying drawings.

[0019] like Figure 1 As shown, the specific steps of the embodiment are as follows:

[0020] S1. Receive a panoramic RGB-D (Red Green Blue Depth) image. The panoramic RGB-D data is obtained by RGB-D cameras spliced ​​at 12 different angles relative to the unmanned vehicle itself. The setting of the RGB-D camera in actual application can refer to the prior art Yang X, Yang Y, Yu C, et al. Active neural topological mapping for multi-agent exploration [J]. IEEE Robotics and Automation Letters, 2023, 9(1): 303-310. The present invention will not be elaborated in detail here.

[0021] Those skilled in the art will know that an RGB-D camera generates an RGB image and a depth image simultaneously.

[0022] S2: The panoramic RGB-D image is input into the path point-based topology mapping module for mapping, and the local topology maps of other unmanned vehicles are received and merged to obtain a global topology map.

[0023] The flowchart of the topological mapping method based on path points is as follows Figure 2 As shown, the specific method is:

[0024] The RGB data and depth data are input into the ViT-B / 32 and ResNet-50 encoders to obtain visual and depth feature vectors respectively, and the direction feature vector is generated according to the angles of 12 RGB-D cameras.

[0025] Then, the depth and direction feature vectors are input into a linear layer network to obtain the fused feature vector v1.

[0026] Then input v1 into a two-layer Transformer network to obtain the depth feature points of the current environment.

[0027] The depth feature points are then input into a multi-layer perceptron to obtain a heat map, and the non-maximum suppression method is used to sample K path points from the heat map. Setting the K value to 5 can avoid excessive computational burden while providing sufficient options for the unmanned vehicle to meet its navigation needs in complex scenarios.

[0028] Then, the visual, depth and direction feature vectors are input into a linear layer network to obtain the fused feature vector v2.

[0029] V2 is then fed into a panoramic encoder based on a multi-layer Transformer network to obtain the visual feature points of the current environment. The average value of these visual feature points is used to represent the current node, and the visual feature points of the corresponding viewpoint are used to represent the path points. The multi-layer Transformer network here refers to a Transformer network with more than two layers.

[0030] Then, the topological map and path points at the previous moment are input into the path point positioning function to calculate the position of the path point in the topological map at the current moment until all path points are calculated; the path point positioning function is an existing known technology, and reference can be made to An D, Wang H, Wang W, et al. Etpnav: Evolving topological planning for vision-language navigation in continuous environments[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. This invention will not be elaborated in detail here.

[0031] At this point, the local topological map of the unmanned vehicle can be obtained, and then the local topological maps of all unmanned vehicles can be merged to obtain a global topological map;

[0032] S3. Six sub-topology maps are extracted from the global topology map, including the current unmanned vehicle topology map, the main node topology map, the virtual node topology map and their corresponding historical unmanned vehicle topology map, the historical main node topology map and the historical virtual node topology map. The extraction process analyzes the nodes and edges in the global map, identifies different types of nodes and their historical information, and constructs them. In practical applications, different types of nodes can be marked, such as using semantic labels, and marking different types of nodes with different values ​​of the binary vector corresponding to the semantic label.

[0033] S4. Input the sub-topological map into the global decision module based on the graph neural network. The module determines the optimal global navigation target through memory fusion and matching score calculation. The specific implementation process of the global decision module based on the graph neural network is an existing known technology. Please refer to Yang X, Yang Y, Yu C, et al. Active neural topological mapping for multi-agent exploration [J]. IEEE Robotics and Automation Letters, 2023, 9 (1): 303-310. The present invention will not be elaborated in detail here.

[0034] S5. Use the A* algorithm to calculate the optimal path from the current position of the unmanned vehicle to the global target, and select a point on the path with a Euclidean distance of 0.25 meters from the current position of the unmanned vehicle as the local navigation target;

[0035] S6. Input the current RGB data and local navigation target of the unmanned vehicle into the local decision module based on the recurrent neural network, and output a specific execution action (forward, left turn or right turn). The specific implementation process of the local decision module based on the recurrent neural network is a known technology. Please refer to Chaplot DS, Gandhi D, Gupta S, et al. Learning to explore using active neural slam [J]. arXiv preprint arXiv: 2004.05155, 2020. The present invention will not be elaborated in detail here.

[0036] S7. Execute an adaptive escape algorithm to optimize the unmanned vehicle's execution action. The specific method is as follows: if the unmanned vehicle's current execution action is forward, calculate the displacement difference between the unmanned vehicle's current position and the position at the previous moment. If the displacement difference is less than a threshold value of 0.001 meters, it is determined that the unmanned vehicle is stagnant, and the unmanned vehicle's execution action is changed to a right turn. The goal of the adaptive escape algorithm is to enable the unmanned vehicle to escape the stagnant state. The default steering strategy of the current algorithm is a right turn. If turning right still cannot escape, or if the environment restricts the right turn, then a left turn can be selected as the steering strategy. That is, the choice of turning left or right should depend on the specific environment and the stagnant state.

[0037] S8. When the exploration task is completed or the exploration time is over, the exploration ends; otherwise, return to S1 to continue the exploration task.

[0038] In this embodiment, the simulation environment is Habitat, and the datasets are Gibson and Matterport3D. The method of the present invention uses the Gibson dataset for training and the Gibson and Matterport3 datasets for verification. The specific effects are as follows:

[0039] Training process:

[0040] 1. Dataset: The Gibson dataset is used to train the algorithm to provide diverse scene and environment features, helping the model learn effective exploration strategies.

[0041] 2. Loss function: During the training process, the MAPPO (Multi-Agent Proximal Policy Optimization) reinforcement learning algorithm is used to train the entire model. The MAPPO algorithm is an existing technology, and its loss function includes three parts: policy loss, value loss, and entropy loss. Please refer to Yu C, Velu A, Vinitsky E, et al. The surprising effectiveness of ppo in cooperative multi-agent games [J]. Advances in neural information processing systems, 2022, 35: 24611-24624. This invention will not be elaborated in detail here.

[0042] 3. Network parameter update: A multi-agent proximal policy optimization algorithm is used to update network parameters, by calculating gradients and adjusting weights using the Adam optimizer.

[0043] 4. Iteration stopping condition: The training iteration stops when the preset maximum number of steps is reached or the loss function converges. The preset maximum number of steps is 600. Refer to the existing technology Yang X, Yang Y, Yu C, et al. Active neural topological mapping for multi-agent exploration [J]. IEEE Robotics and Automation Letters, 2023, 9(1): 303-310.

[0044] Verification process:

[0045] 1. Dataset: The Gibson dataset is used to verify the performance of the model on the training dataset, and the Matterport3D dataset is used to verify the performance of the model on the generalization dataset and evaluate its generalization ability.

[0046] 2. Verification Results: The algorithm was tested on the Gibson and Matterport3D datasets, recording the number of exploration steps required to achieve a 90% exploration rate and the exploration rate. A lower number of exploration steps indicates a faster algorithm exploration speed, while a higher exploration rate indicates a higher exploration completeness. The results show that this method performs better than the MANTM (Multi-Agent Neural Topological Mapping) algorithm on both datasets, as shown in Tables 1 and 2. The number of exploration steps in Tables 1 and 2 refers to the number of steps required to achieve a 90% exploration rate. A smaller number indicates a higher exploration efficiency. Clearly, the NMM-MVNTE method of the present invention has a lower exploration step rate than the MANTM method.

[0047] Table 1 Performance comparison of collaborative exploration algorithms based on the Gibson dataset (mean ± standard deviation)

[0048]

[0049] Table 2 Performance comparison of collaborative exploration algorithms based on the Matterport3D dataset (mean ± standard deviation)

[0050]

[0051] In summary, the method of the present invention completely abandons the dependence on metric maps, realizes the efficient encoding and transmission of environmental information through pure topological representation, greatly reduces the communication overhead between multiple unmanned vehicle systems, and enhances the generalization ability of the algorithm across different scenarios.

[0052] Those skilled in the art will appreciate that the embodiments described herein are intended to aid the reader in understanding the principles of the present invention, and it should be understood that the scope of the present invention is not limited to such specific descriptions and embodiments. Various modifications and variations are readily apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims.

Claims

1. A neural topology exploration method for multiple unmanned vehicles on a metric-free map, characterized by: include: S1, receiving panoramic RGB-D image; S2: Input the panoramic RGB-D image into the path point-based topology mapping module for mapping, and simultaneously receive the local topology maps of other unmanned vehicles and merge them into a global topology map; S3. Extract the current and historical topology maps of the autonomous vehicle, master node, and virtual node from the global topology map, totaling six sub-topology maps. S4. Input the sub-topology graph into the global decision module based on the graph neural network to determine the optimal global navigation target; S5. Use the A* algorithm to calculate the optimal path from the current position of the unmanned vehicle to the global target, and select the local navigation target on the optimal path; S6: Input the current RGB data and local navigation target of the unmanned vehicle into the local decision module based on the recurrent neural network, and output the specific execution action; S7. When the exploration task is completed or the exploration time is over, the exploration ends; otherwise, the exploration returns to S1 to continue.

2. The neural topology exploration method for multiple unmanned vehicles on a metric-free map according to claim 1 is characterized in that: Step S2 specifically includes the following sub-steps: S21. Input the RGB data and depth data into two encoders respectively to obtain the visual feature vector and the depth feature vector, and generate the direction feature vector at the same time. S22, fusing the depth feature vector and the direction feature vector to obtain a first fused feature vector v1; S23, extracting depth feature points of the current environment according to the first fused feature vector v1; S24, generating a heat map according to the depth feature points, and sampling K path points from the heat map; S25, fusing the visual feature vector, the depth feature vector, and the direction feature vector to obtain a second fused feature vector v2; S26. Obtain visual feature points of the current environment based on the second fused feature vector v2, and use the average value of the visual feature points to represent the current node, and use the visual feature points of the corresponding viewing angle to represent the path points; S27. Based on the topological map and path points at the previous moment, calculate the position of the path points in the topological map at the current moment until all path points are calculated; thereby obtaining a local topological map of the unmanned vehicle, and then merge the local topological maps of all unmanned vehicles to obtain a global topological map.

3. The neural topology exploration method for multiple unmanned vehicles on a metric-free map according to claim 2 is characterized in that: The two encoders in step S21 are respectively a ViT-B / 32 encoder and a ResNet-50 encoder.

4. The neural topology exploration method for multiple unmanned vehicles on a metric-free map according to claim 3 is characterized in that: Step S22 and step S25 both use a linear layer network for fusion processing.

5. The neural topology exploration method for multiple unmanned vehicles on a metric-free map according to claim 4 is characterized in that: Step S24 obtains a heat map by inputting the depth feature points into a multi-layer perceptron.

6. The neural topology exploration method for multiple unmanned vehicles on a metric-free map according to claim 5, characterized in that: The execution action outputted in step S6 includes forward, left turn or right turn.

7. The neural topology exploration method for multiple unmanned vehicles on a metric-free map according to claim 6, characterized in that: When the execution action outputted in step S6 is forward, the method further includes calculating the displacement difference between the current position of the unmanned vehicle and the position at the previous moment. If the displacement difference is less than a threshold, it is determined that the unmanned vehicle is stagnant and the execution action of the unmanned vehicle is changed to turn right. Otherwise, execute step S7.

Images