Unmanned system visual navigation method and device

Abstract

The application provides an unmanned system visual navigation method and device. The method comprises the following steps: acquiring an image sequence of a target environment, reconstructing a topology graph according to the correlation similarity and exploration sequence of the image sequence; acquiring a target point image of a navigation task, determining a target node and an adjacent node of the target point image in the topology graph, connecting the target node and the adjacent node in the topology graph, and calculating the shortest path from other nodes to the target node; acquiring a current observation image of the unmanned system, determining a positioning node of the observation image in the topology graph, calculating the distance and turning gap between the current observation image and the image corresponding to the positioning node, and performing queue point to topology graph node navigation or topology graph node to path point navigation according to the distance and turning gap; and repeating the navigation process shown in the above steps at the beginning of the subsequent navigation steps until the unmanned system reaches the target point of the navigation task. The application realizes autonomous navigation of the unmanned system by using only visual information, and has low complexity.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) navigation, and more particularly to a visual navigation method and device for unmanned systems. Background Technology

[0002] The traditional approach to autonomous navigation for unmanned systems employs a framework of "perception-localization-mapping-planning-obstacle avoidance-control." It uses external sensors (such as GPS) or onboard sensors (such as visual-inertial sensors) to estimate the unmanned system's motion, build and update a 3D map of the environment, and plan a safe, traversable path based on the unmanned system's pose and the environmental map to achieve navigation. However, maintaining an environmental map with scale information consumes significant computational resources, and this navigation method differs from the way the human brain navigates. Summary of the Invention

[0003] This application aims to at least partially address one of the technical problems in the related art.

[0004] Therefore, for the image target navigation problem of unmanned systems, the first objective of this application is to propose a visual navigation method for unmanned systems. It realizes a robust and computationally inefficient positioning method and a small-scale topological map memory structure using only visual information, and the proposed method has been successfully verified in a simulation environment.

[0005] The second objective of this application is to propose a visual navigation device for unmanned systems.

[0006] To achieve the above objectives, a first aspect of this application proposes a visual navigation method for an unmanned system, comprising:

[0007] Obtain an image sequence of the target environment, and reconstruct a topology map based on the correlation similarity and exploration order of the image sequence;

[0008] Obtain the target point image of the navigation task, determine the target node and adjacent nodes of the target point image in the topology graph, connect the target node and the adjacent nodes in the topology graph, and calculate the shortest path from other nodes to the target node;

[0009] The system acquires the current observation image of the unmanned system, determines the positioning node of the observation image in the topology map, calculates the distance and turning difference between the current observation image and the image corresponding to the positioning node, and performs navigation from the queue point to the topology map node or from the topology map node to the path point based on the distance and turning difference, thus completing one navigation step.

[0010] The navigation process described above is repeated at the start of subsequent navigation steps until the unmanned system reaches the target point of the navigation task.

[0011] Optionally, acquiring the image sequence of the target environment and reconstructing the topology map based on the association similarity and exploration order of the image sequence includes:

[0012] In the target environment, the unmanned system is used for teaching exploration, and image sequences of the entire target environment are collected using prior information;

[0013] Extract the first local descriptor, the first feature point, and the first global descriptor from the image sequence;

[0014] The first global descriptor is used to perform global image recall, and each image in the image sequence is associated with a first neighboring image set. The SuperGlue network is used to match each image and its associated first neighboring image set with the first local descriptor and the first feature point to obtain matching information. The first global descriptor similarity between each image and its associated first neighboring image set is higher than the first global descriptor similarity threshold.

[0015] Based on the open-source software COLMAP, the matching information and descriptor information are used to perform 3D reconstruction of the image sequence, and a local database is generated. The local database contains the 3D position information of the camera projection center estimated in the reconstructed scene for the successfully matched images, as well as the camera pose.

[0016] A topology map is constructed according to the exploration order of the image sequence, wherein images that do not appear in the local database are skipped when constructing the topology map.

[0017] Optionally, obtaining the first local descriptor, the first feature point, and the first global descriptor of the image sequence includes:

[0018] The first local descriptor and the first feature point of each image in the image sequence are extracted using the SuperPoint decoding layer of deep learning feature point detection.

[0019] The image sequence is encoded using a NetVLAD network layer that aggregates local image features, and the first global descriptor of the encoded image sequence is extracted.

[0020] Optionally, when constructing the topology graph, the method further includes:

[0021] After establishing the connection relationships between all nodes and edges, for each image node in the topology graph, calculate and select the image node whose global descriptor similarity is higher than the second global descriptor similarity threshold and whose distance from the camera projection center position is less than the first camera projection center position distance threshold, and add a shortcut edge between the image node and the corresponding image node.

[0022] Optionally, determining the target node and adjacent nodes of the target point image in the topology graph, connecting the target node and the adjacent nodes in the topology graph, and calculating the shortest path from other nodes to the target node includes:

[0023] Extract the second local descriptor, the second feature point, and the second global descriptor from the target point image;

[0024] Determine the similarity between the first global descriptor of the image represented by the node in the topology graph and the second global descriptor of the second global descriptor; select images whose similarity to the second global descriptor is higher than the second global descriptor similarity threshold and construct a second neighboring image set; select the node corresponding to the image with the highest similarity to the second global descriptor as the neighboring node of the target point image in the topology graph.

[0025] By using the second neighboring image set, the second local descriptor, and the second feature point to perform feature matching, the camera projection center position and pose quaternion of the target point image in the local database are estimated.

[0026] Based on the camera projection center position and attitude quaternion of the target point image in the local database, a target node representing the target point image is constructed in the topology graph, and the target node is connected to the connected node.

[0027] After the target node is connected to the connected nodes, the Dixtro algorithm is used to calculate the shortest path from each other node to the target node in the topology graph.

[0028] Optionally, determining the location node of the observed image in the topology map, calculating the distance and turning difference between the current observed image and the image corresponding to the location node, and performing navigation from queue point to topology map node or from topology map node to path point based on the distance and turning difference, includes:

[0029] Extract the third local descriptor, the third feature point, and the third global descriptor from the observed image;

[0030] Determine the similarity between the first global descriptor of the image represented by the node in the topology graph and the third global descriptor of the third global descriptor; select images whose third global descriptor similarity is higher than the third global descriptor similarity threshold and construct a third neighboring image set;

[0031] By using the third neighboring image set, the third local descriptor, and the third feature point to perform feature matching, the camera projection center position and attitude quaternion of the observed image in the local database are estimated.

[0032] In the third adjacent image set, images whose camera projection center position in the local database is greater than the distance threshold of the second camera projection center position are excluded, and the image node with the highest similarity to the third global descriptor is selected from the remaining image nodes as the positioning node of the observed image in the topology map;

[0033] The distance and turning difference are calculated based on the camera projection center position and attitude quaternion of the observed image and the image corresponding to the positioning node in the local database;

[0034] If the distance and turning difference are greater than a preset difference threshold, navigation from the queue point to the topology graph node is performed.

[0035] If the distance and turning difference are not greater than a preset difference threshold, navigation from the topology node to the path point is performed.

[0036] Optionally, if the distance and turning difference is greater than a preset difference threshold, the navigation from the queue point to the topology graph node includes:

[0037] Based on the attitude quaternions of the observed image queue points and the positioning node, calculate the first and second included angles between the two cameras relative to the same axis of the world coordinate system;

[0038] Based on the observation image queue points and the camera projection center positions of the positioning node, calculate the straight-line distance between the observation image queue points and the positioning node, as well as the third angle relative to the world coordinate axis;

[0039] A movement strategy is generated based on the straight-line distance, the first included angle, the second included angle, and the third included angle to control the movement and orientation of the unmanned system.

[0040] Optionally, the step of generating a movement strategy based on the straight-line distance, the first included angle, the second included angle, and the third included angle to control the movement and orientation of the unmanned system includes:

[0041] The unmanned system is controlled to rotate from the current observation point to face the positioning node, and the rotation angle is the difference between the third included angle and the first included angle;

[0042] The unmanned system is controlled to move towards the positioning node, and the moving distance is the straight-line distance between the observation image queue points and the positioning node;

[0043] The unmanned system is controlled to turn from facing the positioning node to the original orientation of the image corresponding to the positioning node, and the selected angle is the difference between the second included angle and the third included angle.

[0044] Optionally, if the distance and turning difference is not greater than a preset difference threshold, the navigation from the topology node to the waypoint includes:

[0045] Determine the shortest path between the location node and the target node;

[0046] Determine the next node on the shortest path from the location node to the target node;

[0047] Control the unmanned system to move to the next node on the shortest path.

[0048] To achieve the above objectives, a second aspect of this application provides a visual navigation device for an unmanned system, comprising:

[0049] The topology graph construction module is used to acquire image sequences of the target environment and reconstruct a topology graph based on the association similarity and exploration order of the image sequences.

[0050] An initialization module is used to acquire the target point image of the navigation task, determine the target node and adjacent nodes of the target point image in the topology graph, connect the target node and the adjacent nodes in the topology graph, and calculate the shortest path from other nodes to the target node;

[0051] The first navigation module is used to acquire the current observation image of the unmanned system, determine the positioning node of the observation image in the topology map, calculate the distance and turning difference between the current observation image and the image corresponding to the positioning node, and perform navigation from the queue point to the topology map node or from the topology map node to the path point based on the distance and turning difference, thus completing one navigation step.

[0052] The second navigation module is used to repeat the navigation process described above at the beginning of subsequent navigation steps until the unmanned system reaches the target point of the navigation task.

[0053] The technical solutions provided by the embodiments of this application have at least the following beneficial effects:

[0054] For unmanned systems that can only use RGB cameras as perception units, in unknown environments, it is only necessary to construct a topology map offline from a certain number of exploration image sequences, extract environmental features offline using neural networks, and perform 3D reconstruction to estimate the pose information of the corresponding topology map nodes. This information is then used for planning and motion control during online navigation. This lightweight approach achieves autonomous navigation of the unmanned system by relying solely on visual information and the topology map as real-time navigation guidance. In addition, this application has good generalization ability for different scene features and can still achieve good results in testing without scene-specific training. Furthermore, the pose information estimated by 3D reconstruction in this method serves as a guide for the movement strategy, providing more intuitive and strong pose information compared to end-to-end movement strategies learned by the network, thus enabling more accurate movement strategies.

[0055] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0056] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0057] Figure 1 This is a flowchart illustrating a visual navigation method for an unmanned system according to an embodiment of this application;

[0058] Figure 2 This is a flowchart illustrating another visual navigation method for an unmanned system according to an embodiment of this application;

[0059] Figure 3 This is a flowchart illustrating another visual navigation method for an unmanned system according to an embodiment of this application;

[0060] Figure 4 This is a flowchart illustrating another visual navigation method for an unmanned system according to an embodiment of this application;

[0061] Figure 5(a) is a schematic diagram illustrating a navigation test in the scene Foyi according to an embodiment of this application;

[0062] Figure 5(b) is a schematic diagram illustrating another navigation test in the Foyi scene according to an embodiment of this application;

[0063] Figure 5(c) is a schematic diagram illustrating a navigation test in the Kinneyl scene according to an embodiment of this application;

[0064] Figure 5(d) is a schematic diagram illustrating another navigation test in the Kinneyl scene according to an embodiment of this application;

[0065] Figure 6 This is a block diagram illustrating a visual navigation device for an unmanned system according to an embodiment of this application. Detailed Implementation

[0066] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0067] The following description, with reference to the accompanying drawings, illustrates an unmanned system visual navigation method and apparatus according to embodiments of this application.

[0068] Figure 1 This is a flowchart illustrating a visual navigation method for an unmanned system according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0069] Step 101: Obtain the image sequence of the target environment, and reconstruct the topology map based on the correlation similarity of the image sequence and the exploration order.

[0070] like Figure 2 As shown, step 101 further includes:

[0071] Step 201: In the target environment, use an unmanned system for teaching exploration and collect image sequences of the entire target environment using prior information.

[0072] In this embodiment, the unmanned system is first placed in an unknown new environment for teaching and exploration. Prior information is used to collect RGB image sequences of the entire target environment so that the image content covers the entire area and angle of the environment as much as possible, and avoids images that are directly facing a solid color wall or where the lens is too close to an object. Then the images are saved locally.

[0073] Step 202: Extract the first local descriptor, the first feature point, and the first global descriptor of the image sequence.

[0074] As one possible implementation, the SuperPoint decoding layer of deep learning feature point detection is used to extract the first local descriptor and the first feature point of each image in the image sequence; the NetVLAD network layer of image local feature aggregation is used to encode the image sequence and extract the first global descriptor of the encoded image sequence.

[0075] Step 203: Use the first global descriptor to perform global image recall, associate each image in the image sequence with a first neighboring image set, and use the SuperGlue network to match each image and its associated first neighboring image set with the first local descriptor and the first feature point to obtain matching information.

[0076] In this embodiment of the application, each image is associated with a first set of neighboring images with high global descriptor similarity. That is, for any image, when the first global descriptor similarity of other images with the image is higher than the first global descriptor similarity threshold, the other images are added to the first set of neighboring images.

[0077] It is understandable that the first global descriptor similarity threshold is determined based on the actual scenario, and this application does not impose specific limitations on it.

[0078] It should be noted that after matching, the matching information and descriptor information are saved locally.

[0079] Step 204: Based on the open-source software COLMAP, 3D reconstruction of the image sequence is performed using matching information and descriptor information to generate a local database.

[0080] Understandably, some images in the image sequence may not appear in the database due to matching failures. The database contains the 3D position information of the camera projection center estimated in the reconstructed scene for each image, as well as the camera pose (expressed in quaternion form).

[0081] Step 205: Construct a topology map based on the local database, following the exploration order of the image sequence.

[0082] In this embodiment of the application, during the construction of the topology graph, each image is treated as a node, and there are edges between adjacent images to represent the connection relationship.

[0083] It is understandable that for images that do not appear in the local database, the image is skipped and the connection is made directly. The nodes only store the image name information as an index to the local database, and the edges only represent the connection relationship without expressing other information.

[0084] In one possible embodiment, if image 2 does not appear in the reconstruction database, then the topology graph connects image nodes 1 and 3.

[0085] Additionally, it should be noted that after establishing the connection relationships between all nodes and edges, for each image node in the topology graph, calculate and select the image node whose global descriptor similarity is higher than the second global descriptor similarity threshold and whose distance from the camera projection center is less than the first camera projection center distance threshold, and add a shortcut edge between the image node and the corresponding image node.

[0086] By adding shortcut edges, the unmanned system is assumed to be able to move freely between similar and close nodes, which reduces the complexity of path planning.

[0087] It should also be noted that the entire process of step 101 was completed offline, and a reconstructed local database that can be read at any time was generated.

[0088] Step 102: Obtain the target point image of the navigation task, determine the target node and adjacent nodes of the target point image in the topology graph, connect the target node and adjacent nodes in the topology graph, and calculate the shortest path from other nodes to the target node.

[0089] In this embodiment of the application, an image of the target point is first given before the navigation task begins.

[0090] like Figure 3 As shown, step 102 also includes:

[0091] Step 301: Extract the second local descriptor, the second feature point, and the second global descriptor from the target point image.

[0092] It is understood that the process of extracting the second local descriptor, the second feature point, and the second global descriptor of the target point image is described in step 202, and will not be elaborated further here.

[0093] Step 302: Determine the similarity between the first global descriptor and the second global descriptor of the image represented by the node in the topology graph. Select images whose second global descriptor similarity is higher than the second global descriptor similarity threshold and construct a second neighboring image set. Select the node corresponding to the image with the highest second global descriptor similarity as the neighboring node of the target point image in the topology graph.

[0094] In this embodiment of the application, the neighboring image set and neighboring nodes of the target point image are found based on the second global descriptor similarity.

[0095] Step 303: Use the second neighboring image set, the second local descriptor, and the second feature point to perform feature matching, and estimate the camera projection center position and pose quaternion of the target point image in the local database.

[0096] Step 304: Based on the camera projection center position and attitude quaternion of the target point image in the local database, construct a target node representing the target point image in the topology graph, and connect the target node to the connected node.

[0097] Step 305: After the target node is connected to its connected nodes, use the Dixtra algorithm to calculate the shortest path from each other node to the target node in the topology graph.

[0098] It is understandable that step 102 initializes the online navigation task, identifies the target node of the target point image in the topology map, and calculates the shortest path between other nodes and the target node, thus facilitating the subsequent navigation.

[0099] Step 103: Obtain the current observation image of the unmanned system, determine the positioning node of the observation image in the topology map, calculate the distance and turning difference between the current observation image and the corresponding image of the positioning node, and perform navigation from the queue point to the topology map node or from the topology map node to the path point based on the distance and turning difference, and complete one navigation step.

[0100] In this embodiment of the application, the unmanned system is randomly placed at a certain location in the target environment to perform autonomous navigation. The process of implementing one navigation step is described below.

[0101] like Figure 4 As shown, step 103 also includes:

[0102] Step 401: Extract the third local descriptor, the third feature point, and the third global descriptor of the observed image.

[0103] In this embodiment of the application, at the start of the navigation steps, the unmanned system needs to acquire the current observed RGB image and extract the third local descriptor, the third feature point and the third global descriptor from the image using a method similar to that shown in step 202.

[0104] Step 402: Determine the similarity between the first global descriptor and the third global descriptor of the image represented by the node in the topology graph, select images whose third global descriptor similarity is higher than the third global descriptor similarity threshold, and construct the third neighboring image set.

[0105] In this embodiment, when constructing the third neighboring image set, priority is given to selecting from the neighboring nodes with a radius of 3 in the topology graph of the previously located node (selective local recall). Global recall (selection from all node images) is only performed when all of the selected third neighboring image sets fail to meet the threshold requirements. This effectively improves the efficiency of image recall.

[0106] Step 403: Use the third neighboring image set, the third local descriptor, and the third feature point to perform feature matching, and estimate the camera projection center position and pose quaternion of the observed image in the local database.

[0107] Step 404: In the third adjacent image set, exclude images whose camera projection center position in the local database is greater than the distance threshold of the second camera projection center position, and select the image node with the highest similarity to the third global descriptor from the remaining image nodes as the positioning node of the observation image in the topology graph.

[0108] Step 405: Calculate the distance and steering difference based on the camera projection center position and attitude quaternion of the observed image and the corresponding image of the positioning node in the local database.

[0109] It is understandable that the values ​​assigned to the various thresholds mentioned above need to be determined based on the actual scenario.

[0110] In this embodiment, if the distance and turning difference is greater than a preset difference threshold, navigation from queue point to topology node is performed; if the distance and turning difference is not greater than the preset difference threshold, navigation from topology node to path point is performed.

[0111] It is understandable that when the distance and turning difference is greater than the preset difference threshold, it means that the current pose of the unmanned system is still different from the positioning node. When the distance and turning difference is not greater than the preset difference threshold, it means that the current pose of the unmanned system is already on the positioning node.

[0112] As one possible implementation, when navigating from a queue point to a node in the topology graph, it is necessary to calculate the first and second included angles between the two cameras relative to the same axis of the world coordinate system based on the attitude quaternions of the queue point and the positioning node in the observed image. Then, based on the projection center positions of the queue point and the positioning node camera, the straight-line distance between the queue point and the positioning node and the third included angle relative to the world coordinate axis are calculated. Finally, a movement strategy is generated based on the straight-line distance, the first included angle, the second included angle, and the third included angle to control the movement and orientation of the unmanned system.

[0113] It should be noted that when controlling the movement and orientation of the unmanned system, firstly, the unmanned system is controlled to rotate from the current observation point to face the positioning node, and the rotation angle is the difference between the third included angle and the first included angle; then, the unmanned system is controlled to move forward towards the positioning node, and the forward distance is the straight-line distance between the observation image queue points and the positioning node; finally, the unmanned system is controlled to rotate from facing the positioning node to the original orientation of the image corresponding to the positioning node, and the selected angle is the difference between the second included angle and the third included angle.

[0114] It is understandable that the purpose of navigating from queue points to nodes in the topology graph is to locate the nodes when making the next navigation step; it is a process of finding and locating the nodes.

[0115] Once the pose of the unmanned system is already on the positioning node, navigation from the topology graph node to the path point is performed.

[0116] As another possible implementation, when performing topology node-to-path point navigation, since the unmanned system is already on the positioning node, it is only necessary to determine the shortest path between the positioning node and the target node, as well as the next node on the shortest path from the positioning node to the target node, and then control the unmanned system to move to the next node on the shortest path.

[0117] Thus, one navigation step is completed.

[0118] Step 104: Repeat the navigation process described above at the start of subsequent navigation steps until the unmanned system reaches the target point of the navigation task.

[0119] In this embodiment of the application, at the start of each new navigation step, the unmanned system obtains the observed RGB image at this moment and navigates the observed image using the method shown in step 103 until the unmanned system reaches the target point of the navigation task and ends the navigation task.

[0120] Furthermore, to test the effectiveness of this application, navigation experiments were conducted in two Gibson datasets, Foyi and Kinneyl, in one embodiment. The experiments were deployed in the HabitatSim simulation environment and tested on the Gibson datasets. HabitatSim is a flexible, high-performance 3D simulator with configurable robots, multiple sensors, and generative 3D dataset processing capabilities. The Gibson datasets are datasets collected from real-world indoor environments using 3D scanning and reconstruction. Additionally, the code implementation in this embodiment is based on the Python language and uses PyTorch as the deep learning framework.

[0121] Figures 5(a) and 5(b) are schematic diagrams of the paths for two navigation tasks in the Foyi scene, and Figures 5(c) and 5(d) are schematic diagrams of the paths for two navigation tasks in the Kinneyl scene.

[0122] In the experiment, the unmanned system moved forward at a rate of 0.1 m / step and turned left or right at a rate of 10 degrees / step. The line connecting two nodes in the diagram represents one navigation step, and one navigation step contains multiple action steps. The experimental results are shown in Table 1, which demonstrates that this application performs well in indoor environments.

[0123] Table 1

[0124]

[0125]

[0126] It can be seen that, given a sequence of teaching exploration images, this method can achieve autonomous navigation of unmanned systems using the action space of forward, left turn, and right turn, and has a success rate of over 70% in an indoor space of approximately 100 square meters.

[0127] In summary, for unmanned systems that can only use RGB cameras as perception units, this application only requires offline topology map construction from a certain number of exploration image sequences in unknown environments. It then uses neural networks to extract environmental features offline and perform 3D reconstruction, estimating the pose information of corresponding topology map nodes. This information is then used for planning and motion control during online navigation. This lightweight approach achieves autonomous navigation of the unmanned system relying solely on visual information and the topology map as real-time navigation guidance. Furthermore, this application demonstrates good generalization to different scene features, achieving good results even without scene-specific training in testing. On the other hand, the pose information estimated by 3D reconstruction in this method serves as a guide for movement strategies, providing more intuitive and robust pose information compared to end-to-end movement strategies learned through networks, leading to more accurate movement strategies.

[0128] Figure 6 This is a block diagram of an unmanned system visual navigation device 10 according to an embodiment of this application, comprising:

[0129] The topology graph construction module 100 is used to acquire image sequences of the target environment and reconstruct a topology graph based on the correlation similarity and exploration order of the image sequences.

[0130] The initialization module 200 is used to acquire the target point image of the navigation task, determine the target node and adjacent nodes of the target point image in the topology graph, connect the target node and adjacent nodes in the topology graph, and calculate the shortest path from other nodes to the target node.

[0131] The first navigation module 300 is used to acquire the current observation image of the unmanned system, determine the positioning node of the observation image in the topology map, calculate the distance and turning difference between the current observation image and the corresponding image of the positioning node, and perform navigation from the queue point to the topology map node or from the topology map node to the path point based on the distance and turning difference, thus completing one navigation step.

[0132] The second navigation module 400 is used to repeat the navigation process described above at the beginning of subsequent navigation steps until the unmanned system reaches the target point of the navigation task.

[0133] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0134] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0135] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A visual navigation method for an unmanned system, characterized in that, include: Obtain an image sequence of the target environment, and reconstruct a topology map based on the correlation similarity and exploration order of the image sequence; Obtain the target point image of the navigation task, determine the target node and adjacent nodes of the target point image in the topology graph, connect the target node and the adjacent nodes in the topology graph, and calculate the shortest path from other nodes to the target node; The system acquires the current observation image of the unmanned system, determines the positioning node of the observation image in the topology map, calculates the distance and turning difference between the current observation image and the image corresponding to the positioning node, and performs navigation from the queue point to the topology map node or from the topology map node to the path point based on the distance and turning difference, thus completing one navigation step. The navigation process described above is repeated at the start of subsequent navigation steps until the unmanned system reaches the target point of the navigation task. The process of determining the location node of the observed image in the topology map, calculating the distance and turning difference between the current observed image and the image corresponding to the location node, and performing navigation from queue point to topology map node or from topology map node to path point based on the distance and turning difference includes: Extract the third local descriptor, the third feature point, and the third global descriptor from the observed image; Determine the similarity between the first global descriptor of the image represented by the node in the topology graph and the third global descriptor of the third global descriptor; select images whose third global descriptor similarity is higher than the third global descriptor similarity threshold and construct a third neighboring image set; By using the third neighboring image set, the third local descriptor, and the third feature point for feature matching, the camera projection center position and pose quaternion of the observed image in the local database are estimated; In the third adjacent image set, images whose camera projection center position in the local database is greater than the distance threshold of the second camera projection center position are excluded, and the image node with the highest similarity to the third global descriptor is selected from the remaining image nodes as the positioning node of the observed image in the topology map; The distance and turning difference are calculated based on the camera projection center position and attitude quaternion of the observed image and the image corresponding to the positioning node in the local database; If the distance and turning difference are greater than a preset difference threshold, navigation from the queue point to the topology graph node is performed. If the distance and turning difference are not greater than a preset difference threshold, navigation from the topology node to the path point is performed.

2. The method according to claim 1, characterized in that, The process of acquiring image sequences of the target environment and reconstructing a topological map based on the correlation similarity and exploration order of the image sequences includes: In the target environment, the unmanned system is used for teaching exploration, and image sequences of the entire target environment are collected using prior information; Extract the first local descriptor, the first feature point, and the first global descriptor from the image sequence; The first global descriptor is used to perform global image recall, and a first neighboring image set is associated with each image in the image sequence. The SuperGlue network is used to match the first local descriptor and the first feature point of each image and its associated first neighboring image set to obtain matching information. The first global descriptor similarity of each image and its associated first neighboring image set is higher than the first global descriptor similarity threshold. Based on the open-source software COLMAP, the matching information and descriptor information are used to perform 3D reconstruction of the image sequence, and a local database is generated. The local database contains the 3D position information of the camera projection center estimated in the reconstructed scene for the successfully matched images, as well as the camera pose. A topology map is constructed according to the exploration order of the image sequence, wherein images that do not appear in the local database are skipped when constructing the topology map.

3. The method according to claim 2, characterized in that, The extraction of the first local descriptor, the first feature point, and the first global descriptor of the image sequence includes: The first local descriptor and the first feature point of each image in the image sequence are extracted using the SuperPoint decoding layer of deep learning feature point detection. The image sequence is encoded using a NetVLAD network layer that aggregates local image features, and the first global descriptor of the encoded image sequence is extracted.

4. The method according to claim 2, characterized in that, The construction of the topology graph also includes: After establishing the connection relationships between all nodes and edges, for each image node in the topology graph, calculate and select the image node whose global descriptor similarity is higher than the second global descriptor similarity threshold and whose distance from the camera projection center position is less than the first camera projection center position distance threshold, and add a shortcut edge between the image node and the corresponding image node.

5. The method according to claim 4, characterized in that, The process of determining the target node and adjacent nodes of the target point image in the topology graph, connecting the target node and the adjacent nodes in the topology graph, and calculating the shortest path from other nodes to the target node includes: Extract the second local descriptor, the second feature point, and the second global descriptor from the target point image; Determine the similarity between the first global descriptor of the image represented by the node in the topology graph and the second global descriptor of the second global descriptor; select images whose similarity to the second global descriptor is higher than the second global descriptor similarity threshold and construct a second neighboring image set; select the node corresponding to the image with the highest similarity to the second global descriptor as the neighboring node of the target point image in the topology graph. By using the second neighboring image set, the second local descriptor, and the second feature point to perform feature matching, the camera projection center position and pose quaternion of the target point image in the local database are estimated. Based on the camera projection center position and attitude quaternion of the target point image in the local database, a target node representing the target point image is constructed in the topology graph, and the target node is connected to the adjacent node; After the target node is connected to its neighboring nodes, the Dixtro algorithm is used to calculate the shortest path from each of the other nodes to the target node in the topology graph.

6. The method according to claim 5, characterized in that, If the distance and turning difference are greater than a preset difference threshold, the navigation from the queue point to the topology graph node is performed, including: Based on the attitude quaternions of the observed image queue points and the positioning node, calculate the first and second included angles between the two cameras relative to the same axis of the world coordinate system; Based on the observation image queue points and the camera projection center position of the positioning node, calculate the straight-line distance between the observation image queue points and the positioning node, as well as the third angle relative to the world coordinate axis. A movement strategy is generated based on the straight-line distance, the first included angle, the second included angle, and the third included angle to control the movement and orientation of the unmanned system.

7. The method according to claim 6, characterized in that, The step of generating a movement strategy based on the straight-line distance, the first included angle, the second included angle, and the third included angle to control the movement and orientation of the unmanned system includes: The unmanned system is controlled to rotate from the current observation point to face the positioning node, and the rotation angle is the difference between the third included angle and the first included angle; The unmanned system is controlled to move towards the positioning node, and the moving distance is the straight-line distance between the observation image queue points and the positioning node; The unmanned system is controlled to turn from facing the positioning node to the original orientation of the image corresponding to the positioning node, and the selected angle is the difference between the second included angle and the third included angle.

8. The method according to claim 7, characterized in that, If the distance and turning difference are not greater than a preset difference threshold, the navigation from the topology node to the path point is performed, including: Determine the shortest path between the location node and the target node; Determine the next node on the shortest path from the location node to the target node; Control the unmanned system to move to the next node on the shortest path.

9. A visual navigation device for an unmanned system, characterized in that, include: The topology graph construction module is used to acquire image sequences of the target environment and reconstruct a topology graph based on the association similarity and exploration order of the image sequences. An initialization module is used to acquire the target point image of the navigation task, determine the target node and adjacent nodes of the target point image in the topology graph, connect the target node and the adjacent nodes in the topology graph, and calculate the shortest path from other nodes to the target node; The first navigation module is used to acquire the current observation image of the unmanned system, determine the positioning node of the observation image in the topology map, calculate the distance and turning difference between the current observation image and the image corresponding to the positioning node, and perform navigation from the queue point to the topology map node or from the topology map node to the path point based on the distance and turning difference, thus completing one navigation step. The second navigation module is used to repeat the navigation process described above at the beginning of subsequent navigation steps until the unmanned system reaches the target point of the navigation task. The process of determining the location node of the observed image in the topology map, calculating the distance and turning difference between the current observed image and the image corresponding to the location node, and performing navigation from queue point to topology map node or from topology map node to path point based on the distance and turning difference includes: Extract the third local descriptor, the third feature point, and the third global descriptor from the observed image; Determine the similarity between the first global descriptor of the image represented by the node in the topology graph and the third global descriptor of the third global descriptor; select images whose third global descriptor similarity is higher than the third global descriptor similarity threshold and construct a third neighboring image set; By using the third neighboring image set, the third local descriptor, and the third feature point for feature matching, the camera projection center position and pose quaternion of the observed image in the local database are estimated; In the third adjacent image set, images whose camera projection center position in the local database is greater than the distance threshold of the second camera projection center position are excluded, and the image node with the highest similarity to the third global descriptor is selected from the remaining image nodes as the positioning node of the observed image in the topology map; The distance and turning difference are calculated based on the camera projection center position and attitude quaternion of the observed image and the image corresponding to the positioning node in the local database; If the distance and turning difference are greater than a preset difference threshold, navigation from the queue point to the topology graph node is performed. If the distance and turning difference are not greater than a preset difference threshold, navigation from the topology node to the path point is performed.

Images