Visual navigation method and system based on unordered set of rgb images

Abstract

This application discloses a visual navigation method and system based on an unordered RGB image set, belonging to the field of visual navigation technology. The method includes: extracting global visual descriptors from the unordered RGB image set, the target image, and the real-time RGB observation image; determining candidate retrieval distance thresholds based on the global visual descriptors; filtering candidate image pairs based on the candidate retrieval distance thresholds and performing local geometric verification on the candidate image pairs to obtain edge weights; performing global structured pruning and strong loop re-insertion on the topology graph with set edge weights to obtain a robust topology graph; performing belief propagation localization on the robust topology graph to obtain posterior beliefs; determining the current position node on the robust topology graph based on the posterior beliefs and performing belief-aware target planning to obtain a navigation path. This application can solve the problem of difficulty in directly processing unordered image sets, reduce deployment costs and calibration complexity, and can accurately and efficiently achieve visual navigation.

Description

Technical Field

[0001] This application relates to the field of visual navigation technology, and in particular to visual navigation methods and systems based on unordered RGB image sets. Background Technology

[0002] Visual navigation is a crucial capability for mobile robots, embodied agents, and service robots to achieve autonomous movement. Existing solutions can generally be categorized into four types: geometric navigation based on metric maps or depth / laser sensors, navigation based on visual position recognition and topological maps, visual navigation based on time-ordered video or trajectory learning, and end-to-end image-driven target control.

[0003] The first type of solution can obtain relatively fine geometric information, but it usually relies on sensor calibration, continuous trajectory acquisition, depth cameras, LiDAR, or visual inertial odometry, which limits its low-cost deployment, use of crowdsourced images, and cross-scene migration.

[0004] The second type of approach uses graph nodes to represent location images and edges to represent the adjacency relationships between locations, which can reduce the cost of dense geometry reconstruction. However, if only global descriptor similarity is used to build edges, it is easily affected by duplicate appearances, changes in viewpoint, occlusion, and changes in lighting; if only local geometry verification is used, the graph structure may break due to overly strict thresholds.

[0005] The third type of scheme usually assumes that the images are continuously collected by the robot along the trajectory, and that adjacent frames are naturally approximately spatially adjacent; however, when the input images come from manual shooting, crowdsourced images, interior photos of buildings, or public image libraries, this temporal prior does not exist, and the sequential navigation model is prone to incorrect connections, oscillating decisions, or local repetitions.

[0006] The fourth type of end-to-end vision strategy can directly output actions from the current observation and target image, but it usually lacks lightweight structural memory and is prone to short-sighted planning in intersections, loops, repetitive rooms, long-distance paths or yaw recovery scenarios, and the reasons for errors are difficult to explain. Summary of the Invention

[0007] In view of this, embodiments of this application provide a visual navigation method and system based on an unordered RGB image set to achieve visual navigation accurately and efficiently.

[0008] One aspect of this application provides a visual navigation method based on an unordered RGB image set, the method comprising the following steps:

[0009] Global visual descriptors are extracted from the unordered RGB image set, the target image, and the real-time RGB observation image, respectively.

[0010] Determine the candidate retrieval distance threshold based on the global visual descriptor;

[0011] Candidate image pairs are selected from the unordered RGB image set based on the candidate retrieval distance threshold, and local geometric verification is performed on the candidate image pairs to obtain edge weights;

[0012] Global structured pruning and strong loop re-insertion are performed on the topology graph with the edge weights set to obtain a robust topology graph.

[0013] Belief propagation and localization are performed on the robust topology to obtain posterior beliefs;

[0014] Based on posterior beliefs, the current position node is determined on the robust topology graph, and belief-aware goal planning is performed to obtain the navigation path.

[0015] In some embodiments, determining the candidate retrieval distance threshold based on the global visual descriptor includes the following steps:

[0016] Select a reference anchor image, and then select the effective neighbor image that is the furthest from the candidate search results from the reference anchor image;

[0017] Candidate images are retrieved from the reference anchor image and the effective neighbor image respectively, and local feature matching and geometric verification are performed to obtain the number of inliers and the descriptor distance of the global visual descriptor;

[0018] Clustering based on the number of inliers yields high-confidence clusters and second-highest-confidence clusters;

[0019] The geometric verification inlier threshold is determined based on the high-confidence cluster and the second-highest-confidence cluster;

[0020] The candidate retrieval distance threshold is determined based on the descriptor distance of the candidate image pairs in the high-confidence cluster.

[0021] In some embodiments, performing local geometric verification on the candidate image pairs to obtain edge weights includes the following steps:

[0022] For any candidate image pair, if the descriptor distance is less than or equal to the candidate retrieval distance threshold, local feature matching and geometric consistency verification are performed; if the number of inliers is greater than the inlier threshold, the graph edge weight is set to the number of inliers, otherwise it is set to no edge or weight is 0.

[0023] In some embodiments, the step of performing global structured pruning and strong loop re-insertion based on the topology graph with the edge weights to obtain a robust topology graph includes the following steps:

[0024] Calculate the maximum spanning forest for the initial weighted topology graph with the edge weights set, so that the retained edges are composed of strong geometric correspondences in each connected component;

[0025] Cluster the edge weights in the initial weighted topology graph and select the highest confidence clusters to determine the strong loop re-insertion threshold;

[0026] Edges that were not retained by the maximum generated forest but whose edge weights are higher than the strong loop re-insertion threshold are re-inserted into the initial weighted topology graph to obtain the robust topology graph.

[0027] In some embodiments, the step of performing belief propagation localization on the robust topology to obtain posterior beliefs includes the following steps:

[0028] The graph nodes on the robust topology graph are used as candidate positions to maintain the node belief distribution;

[0029] Construct a multi-hop reachability matrix based on the graph adjacency matrix, and then normalize it in parallel to obtain the transition matrix.

[0030] The predicted belief is obtained from the belief at the previous moment and the transition matrix;

[0031] The observation likelihood is calculated based on the descriptor distance between the real-time RGB observation image and the image of each node, and the prediction weight and observation weight are adaptively determined based on the normalized entropy of the belief distribution at the previous time step.

[0032] The posterior belief is obtained by weighted geometric mean normalization based on the prediction weight, the observation weight, and the prediction belief.

[0033] In some embodiments, determining the current location node on the robust topology graph based on posterior beliefs and performing belief-aware goal planning to obtain a navigation path includes the following steps:

[0034] The starting node is determined based on the current posterior belief, and the target node is determined based on the similarity between the global visual descriptor of the target image and the global visual descriptor of the graph node.

[0035] Candidate paths are determined based on the starting node and the target node;

[0036] For each candidate path, the path confidence is determined by the minimum edge weight on the path, and the widest path that maximizes the path confidence is selected as the navigation path.

[0037] In some embodiments, the method further includes the following steps:

[0038] During visual navigation, node beliefs are continuously updated. If the node with the maximum a posteriori belief deviates from the current path, or if the graph distance from the node with the maximum a posteriori belief to the current target node exceeds a preset threshold, the navigation path is re-searched from the node with the current maximum a posteriori belief to the target node.

[0039] Another aspect of this application embodiment provides a visual navigation system based on an unordered RGB image set, the system comprising:

[0040] The descriptor extraction unit is used to extract global visual descriptors from the unordered RGB image set, the target image, and the real-time RGB observation image, respectively.

[0041] A threshold determination unit is used to determine a candidate retrieval distance threshold based on the global visual descriptor;

[0042] The edge weight determination unit is used to filter candidate image pairs from the unordered RGB image set based on the candidate retrieval distance threshold, and perform local geometric verification on the candidate image pairs to obtain edge weights;

[0043] The topology graph construction unit is used to perform global structured pruning and strong loop re-insertion based on the topology graph with the edge weights set, to obtain a robust topology graph;

[0044] A belief determination unit is used to perform belief propagation localization on the robust topology graph to obtain posterior beliefs.

[0045] The path navigation unit is used to determine the current position node on the robust topology graph based on posterior beliefs and to perform belief-aware goal planning to obtain a navigation path.

[0046] Another aspect of this application embodiment provides an electronic device, including a processor and a memory;

[0047] The memory is used to store programs;

[0048] The processor executes the program to implement any of the methods described above.

[0049] Another aspect of this application provides a computer-readable storage medium storing a program that is executed by a processor to implement the method described in any of the above embodiments.

[0050] This application includes at least the following beneficial effects:

[0051] This application can extract global visual descriptors from unordered RGB image sets, target images, and real-time RGB observation images respectively; determine candidate retrieval distance thresholds based on the global visual descriptors; filter candidate image pairs from the unordered RGB image set based on the candidate retrieval distance thresholds, and perform local geometric verification on the candidate image pairs to obtain edge weights; perform global structured pruning and strong loop re-insertion on the topology graph with set edge weights to obtain a robust topology graph; perform belief propagation localization on the robust topology graph to obtain posterior beliefs; determine the current position node on the robust topology graph based on the posterior beliefs, and perform belief-aware target planning to obtain a navigation path. This application can solve the problem of directly processing unordered image sets and can solve the strong dependence on auxiliary sensors such as depth, LiDAR, odometry, or inertial navigation, reducing deployment costs and calibration complexity, and can achieve accurate and efficient visual navigation. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart illustrating the visual navigation method based on an unordered RGB image set provided in this application embodiment;

[0054] Figure 2 An optional system architecture diagram is provided for embodiments of this application;

[0055] Figure 3 A flowchart illustrating the robust topology graph construction process provided in this application embodiment;

[0056] Figure 4 A flowchart illustrating the belief propagation and localization process provided in this application embodiment;

[0057] Figure 5 A flowchart of sub-target search and replanning provided for embodiments of this application;

[0058] Figure 6 An optional system module structure diagram provided for an embodiment of this application;

[0059] Figure 7 This is a structural block diagram of a visual navigation system based on an unordered RGB image set provided in an embodiment of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] Before providing a detailed description of the embodiments of this application, some related technologies involved in the embodiments of this application will be described first, as follows:

[0062] RGB image: refers to a common visual image composed of three color channels: red, green, and blue. This solution mainly uses RGB information to complete mapping, positioning, and navigation.

[0063] Unordered RGB image set: refers to a group of environmental landmark images that do not require acquisition time sequence, continuous trajectory, camera pose, or odometry information, denoted as .

[0064] Global visual descriptor: refers to the vector z used to characterize the location features of the entire image, which can be extracted by visual location recognition models, convolutional networks, or visual base models.

[0065] Local geometric verification refers to performing local feature matching on candidate image pairs and using a geometric model to estimate and eliminate incorrect matches in order to obtain the number of inliers. .

[0066] Robust topology graph: refers to a navigable graph structure consisting of image nodes and weighted edges, where nodes represent landmark images and edges represent reachable or adjacent relationships that have been verified visually and geometrically.

[0067] Widest path: refers to the path with the largest minimum edge weight, used to avoid choosing navigation routes with weak visual connections.

[0068] Yaw detection: refers to determining whether the robot needs to replan based on whether the maximum a posteriori node has deviated from the planned path or whether the distance to the current sub-target exceeds a threshold during execution.

[0069] The shortcomings of existing technologies:

[0070] 1. It has a strong dependence on time order and is difficult to directly process unordered image collections.

[0071] 2. It relies heavily on auxiliary sensors such as depth, lidar, odometer or inertial navigation, resulting in high deployment costs and calibration complexity.

[0072] 3. Disordered image composition is prone to producing false edges or graph breaks, making it difficult to simultaneously guarantee the reliability and connectivity of the graph structure.

[0073] 4. It lacks a global positioning mechanism without odometer that adapts to two-dimensional topology maps, making it difficult to handle forks, intersections, and loops.

[0074] 5. The planning and execution lack a dynamic recovery mechanism, making it difficult to detect and replan the robot in a timely manner after it deviates from the path.

[0075] Reference Figure 1 This application provides a visual navigation method based on an unordered RGB image set, specifically including the following steps S100~S150:

[0076] S100: Extract global visual descriptors from the unordered RGB image set, the target image, and the real-time RGB observation image, respectively;

[0077] S110: Determine the candidate retrieval distance threshold based on the global visual descriptor;

[0078] S120: Based on the candidate retrieval distance threshold, candidate image pairs are selected from the unordered RGB image set, and local geometric verification is performed on the candidate image pairs to obtain edge weights;

[0079] S130: Perform global structured pruning and strong loop re-insertion on the topology graph with the edge weights set to obtain a robust topology graph;

[0080] S140: Perform belief propagation localization on the robust topology to obtain posterior beliefs;

[0081] S150: Determine the current position node on the robust topology graph based on posterior beliefs, and perform belief-aware goal planning to obtain the navigation path.

[0082] The following section will provide a detailed introduction and explanation of the solutions in the embodiments of this application, using specific application examples.

[0083] This application provides a visual navigation method based on an unordered RGB image set, including robust topology mapping, on-image belief propagation localization, belief-aware sub-target search, and closed-loop replanning after yaw. The specific steps are as follows:

[0084] S1, acquire the unordered RGB image set, the target image, and real-time RGB observation images during robot execution. The unordered RGB image set is denoted as... Each of them A landmark image in the environment; the target image is denoted as... Real-time observed images are denoted as .

[0085] S2, extract global visual descriptors from the images, target image, and real-time observation image in the unordered RGB image set. Preferably, the global visual descriptor is an L2 normalized descriptor, which can be generated by a visual position recognition model, a convolutional network, or a visual base model.

[0086] S3, a candidate matching index is built based on the global descriptor, and a one-time data-driven threshold calibration is performed. A baseline anchor image is selected. And select from candidate search results that match The largest effective neighbor ;right and Candidate images are retrieved separately, and local feature matching and geometric verification are performed to obtain the number of inliers. and descriptor distance Clustering based on the number of inliers yields high-confidence clusters. and second-highest confidence cluster ;according to and Determine the interior point threshold for geometric verification ,For example ;according to The global descriptor distance of candidate pairs determines the candidate retrieval distance threshold. ,For example .

[0087] S4: Image pairs are selected based on a candidate retrieval distance threshold, and local geometric verification is performed on the candidate image pairs to obtain edge weights. For any image pair... If the global descriptor distance is not greater than Then perform local feature matching and geometric consistency verification; if the number of interior points... Greater than Then set the graph edge weights. Otherwise, set it to no border or weight 0.

[0088] S5 performs global structured pruning and strong loop re-insertion on the initial weighted topology obtained from S4 to obtain a robust topology. Specifically, this includes: calculating the maximum spanning forest for the initial weighted topology, ensuring that the retained edges are composed of strong geometric correspondences within each connected component; clustering the edge weights in the initial graph, and selecting the highest-confidence clusters to determine the strong loop re-insertion threshold. Those not included in the maximum generated forest but with higher edge weights will be retained. The edges are reinserted into the graph.

[0089] S6. Perform belief propagation localization on a robust topology graph. Use graph nodes as candidate locations and maintain the node belief distribution. According to the graph adjacency matrix Constructing a multi-hop reachability matrix Parallel normalization yields the transition matrix. From the belief of the previous moment and Gain predictive beliefs .

[0090] S7, calculate the observation likelihood based on the descriptor distance between the real-time observed image and the images of each node. And based on the normalized entropy of the belief distribution at the previous moment. The prediction weights and observation weights are adaptively determined. Preferably, the prediction weights... Observation weight Then, weighted geometric mean normalization is used to obtain posterior beliefs. .

[0091] S8 determines the current position node based on posterior beliefs and executes belief-aware sub-goal planning. The starting node is determined based on the current posterior beliefs. Target nodes are determined based on the similarity between the target image descriptor and the graph node descriptor. For candidate paths The path confidence is determined by the minimum edge weight on the path. , choose to The widest and largest path is used as the navigation path.

[0092] S9, the robot tracks the visual sub-targets sequentially along the planned path. During real-time execution, node beliefs are continuously updated. If the maximum a posteriori node deviates from the current path, or the graph distance from the maximum a posteriori node to the current sub-target exceeds a preset threshold, replanning is triggered, and the widest path is searched again from the current maximum a posteriori node to the target node.

[0093] S10. When the belief probability of the target node or the belief probability of nodes near the target exceeds a preset threshold, or when the robot reaches the target sub-target, navigation is determined to be successful.

[0094] Specifically, the embodiments of this application include the following technical solutions:

[0095] 1. Input and preprocessing.

[0096] Receive a set of unordered RGB images This image collection can be sourced from offline robot acquisition, manual photography, crowdsourced images, interior photos of buildings, screenshots of simulated environments, or public image libraries. Images are not required to have acquisition timestamps, adjacent frame relationships, pose annotations, or odometry data.

[0097] Also receive target image And real-time observation images obtained by a monocular RGB camera during robot operation. Extract global visual descriptors for each image. And preferably perform L2 normalization.

[0098] 2. Robust topology graph construction.

[0099] First, a global descriptor index is established, for example, using an approximate nearest neighbor index, to quickly retrieve candidate image pairs. To avoid the fixed top-k or fixed distance threshold becoming ineffective in different scenarios, a one-time threshold calibration is performed.

[0100] Select an anchor point And select the furthest valid candidate in the global description subspace. .Will and As representative anchor points, local feature matching and geometric verification are performed with candidate images respectively to obtain the number of inliers for each candidate pair. The number of inliers is then clustered to estimate the boundary threshold between strong and weak matching distributions.

[0101] After threshold calibration, geometric verification is performed on candidate pairs that meet the descriptor distance threshold. If the number of inliers is greater than the geometric threshold, the image pair is treated as an edge in the topological graph, and the number of inliers or its normalized value is used as the edge weight. Subsequently, the maximum spanning forest is calculated and strong loop re-insertion is performed to reduce the interference of erroneous edges on belief propagation and planning, while preserving alternative paths and topological loop closures.

[0102] 3. Positioning of belief dissemination on the map.

[0103] The constructed topology graph Maintaining Belief Distribution Each node This corresponds to a reference image or a landmark. Based on the graph adjacency matrix... Constructing a multi-hop reachability matrix ,right The transition matrix is ​​obtained by row-by-row normalization. and the belief of the previous moment and Multiply to obtain the predictive belief. .

[0104] Calculate real-time observations With each node image The descriptor distance is calculated and converted into observation likelihood. For example... ,in This can be configured according to the scenario. It is based on the normalized entropy of the belief from the previous moment. Dynamically adjust prediction weights and observation weights The predicted beliefs and observed likelihoods are then fused using a weighted geometric mean to obtain the posterior beliefs. .

[0105] 4. Sub-target search and closed-loop navigation.

[0106] Determine the starting node by identifying the node with the highest current belief. Target nodes are determined by the similarity between the target image and the graph node descriptors. Searching for sources in the weighted topology graph arrive The path is determined, and the path confidence is defined as the minimum edge weight in the path, thus selecting the widest path.

[0107] After obtaining the path, the path nodes are sequentially passed as visual sub-targets to the local vision controller. During execution, the belief is continuously updated; if the maximum posterior node at the current position is not on the currently planned path, or the topological distance between the current position node and the current sub-target node exceeds a threshold... If the robot deviates from its course or is disturbed, it will be determined that the robot has been disturbed and will search for the widest path again from the current position node to the target node.

[0108] For example, Figure 2 The diagram shows the overall system architecture, illustrating the relationships between the unordered RGB image set, the target image, and the real-time observation image after they enter the global descriptor extraction module, driving robust topology mapping, belief propagation localization, sub-target search, and local visual control, respectively.

[0109] Figure 3 A flowchart for constructing robust topology graphs is provided, illustrating the graph construction process from unordered image input, descriptor extraction, anchor threshold calibration, candidate selection, local geometry verification, to maximum generating forest and strong loop re-insertion.

[0110] Figure 4 The flowchart for belief propagation shows the process by which the previous belief, graph structure transition matrix, real-time observation likelihood, and entropy adaptive weights enter the weighted geometric mean fusion module to obtain the posterior belief.

[0111] Figure 5 The flowchart for sub-target search and replanning shows the closed-loop process of determining the starting point and target node based on the current posterior belief and target image, selecting the widest path, executing visual sub-targets, and triggering replanning when yaw or distance threshold is exceeded.

[0112] Figure 6 The system module structure diagram shows the system structure relationships between the image input module, descriptor extraction module, threshold calibration module, topology graph construction module, graph structure optimization module, belief propagation and localization module, sub-goal planning module, closed-loop control module, and output module.

[0113] In summary, the embodiments of this application include the following key technical solutions:

[0114] 1. A method for constructing a navigable topology map using an unordered set of RGB images, wherein image acquisition time sequence, pose, odometry, depth or lidar are not required.

[0115] 2. A one-time data-driven threshold calibration method, which jointly determines the candidate retrieval distance threshold and the geometric verification inlier threshold by anchor point image, global retrieval, local geometric verification and inlier clustering.

[0116] 3. A topological graph skeleton extraction method that combines local geometric verification of edge weights with maximum spanning forest to suppress false edges and maintain the reliability of graph structure.

[0117] 4. A strong loop re-insertion method, which determines the re-insertion threshold by clustering the original edge weights, and re-adds high-confidence loop edges to the topology graph to balance skeleton reliability and path redundancy.

[0118] 5. A method for odometry-free global localization on a two-dimensional topology graph, which constructs a transition matrix through a multi-hop reachability matrix and recursively calculates node beliefs on the graph.

[0119] 6. An entropy-adaptive observation fusion method that dynamically adjusts the weights of graph structure prediction and visual observation likelihood based on the normalized entropy of the belief at the previous moment.

[0120] 7. A graphical posterior belief update method based on the fusion of predicted beliefs and observed likelihood using weighted geometric mean.

[0121] 8. A method for determining target nodes based on target image descriptors and searching for visual sub-target paths with the goal of maximizing the minimum edge weight of the path.

[0122] 9. A closed-loop yaw detection and replanning method, wherein when the maximum a posteriori node is off the path or the graph distance to the current sub-target exceeds a threshold, the widest path is re-searched from the current node to the target node.

[0123] Some alternatives are as follows:

[0124] 1. Global visual descriptors can be generated by visual location recognition models, convolutional neural networks, visual Transformers, or basic visual models, as long as they can output vector representations for image similarity retrieval.

[0125] 2. Local feature matchers can employ SIFT, SuperPoint, DISK, LightGlue, or other local feature matching algorithms; geometric verification can employ homography matrix, fundamental matrix, RANSAC, MAGSAC, or other robust estimation methods.

[0126] 3. The candidate retrieval distance threshold, geometric verification threshold, and strong loop re-insertion threshold can be determined using clustering, quantiles, mixture models, or other data-driven statistical methods.

[0127] 4. Graph structure optimization can be achieved using maximum spanning forest, maximum spanning tree, connected component constraints, edge weight sparsification, or loop confidence filtering, etc.

[0128] 5. Sub-goal path search can be implemented using the widest path algorithm, a modified Dijkstra's algorithm, the maximum bottleneck path algorithm, or a priority queue search.

[0129] 6. The local vision controller can be a vision servo controller, an image target navigation controller, a learning controller, or other modules that can output action commands based on the current observation and sub-target images.

[0130] In specific implementation, the visual descriptor model, local feature matcher, graph search algorithm, threshold clustering method, and yaw determination conditions can be replaced with equivalent ones based on the robot platform, camera type, computing resources, scene scale, and controller type.

[0131] The beneficial effects of the embodiments of this application include at least the following:

[0132] 1. It does not rely on image acquisition order, pose, odometer, depth sensor or LiDAR, and can directly use unordered RGB image sets to build a navigable topology map, reducing the deployment threshold.

[0133] 2. By using one-time data-driven threshold calibration, the global candidate retrieval threshold and local geometric verification threshold are automatically determined, reducing manual parameter tuning and improving cross-scene adaptability.

[0134] 3. Combining local geometric verification with global structural constraints of maximum spanning forest can maintain graph connectivity and critical loops while suppressing visual false edges.

[0135] 4. By spreading beliefs on a two-dimensional topological graph through a multi-hop graph transition matrix, it is able to handle intersections, forks, and loops, rather than being limited to one-dimensional time chains.

[0136] 5. Based on the adaptive fusion of structural prediction and visual observation according to positioning uncertainty, it can still maintain stable positioning under changes in illumination, noise, occlusion and viewing angle.

[0137] 6. Using the widest path criterion to select visual sub-target paths makes the weakest visual connections on the path as reliable as possible, which helps to improve the success rate of closed-loop execution.

[0138] 7. It can detect path deviations caused by robot yaw or external disturbances, and automatically replan from the current position, improving the robustness of real robot deployment.

[0139] Reference Figure 7 This application provides a visual navigation system based on an unordered RGB image set, including:

[0140] The descriptor extraction unit is used to extract global visual descriptors from the unordered RGB image set, the target image, and the real-time RGB observation image, respectively.

[0141] A threshold determination unit is used to determine a candidate retrieval distance threshold based on the global visual descriptor;

[0142] The edge weight determination unit is used to filter candidate image pairs from the unordered RGB image set based on the candidate retrieval distance threshold, and perform local geometric verification on the candidate image pairs to obtain edge weights;

[0143] The topology graph construction unit is used to perform global structured pruning and strong loop re-insertion based on the topology graph with the edge weights set, to obtain a robust topology graph;

[0144] A belief determination unit is used to perform belief propagation localization on the robust topology graph to obtain posterior beliefs.

[0145] The path navigation unit is used to determine the current position node on the robust topology graph based on posterior beliefs and to perform belief-aware goal planning to obtain a navigation path.

[0146] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0147] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0148] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the described functions and / or features may be integrated into a single physical system and / or software module, or one or more functions and / or features may be implemented in a separate physical system or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the system disclosed herein, the actual implementation of the module will be understood within the scope of conventional technology for an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0149] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0150] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, system, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, system, or device). For the purposes of this specification, "computer-readable medium" can mean any system that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, system, or device.

[0151] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic systems) with one or more wires, portable computer disk drives (magnetic systems), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic systems, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0152] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0153] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0154] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0155] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A visual navigation method based on an unordered RGB image set, characterized in that, The method includes the following steps: Global visual descriptors are extracted from the unordered RGB image set, the target image, and the real-time RGB observation image, respectively. Determine the candidate retrieval distance threshold based on the global visual descriptor; Candidate image pairs are selected from the unordered RGB image set based on the candidate retrieval distance threshold, and local geometric verification is performed on the candidate image pairs to obtain edge weights; Global structured pruning and strong loop re-insertion are performed on the topology graph with the edge weights set to obtain a robust topology graph. Belief propagation and localization are performed on the robust topology to obtain posterior beliefs; Based on posterior beliefs, the current position node is determined on the robust topology graph, and belief-aware goal planning is performed to obtain the navigation path.

2. The visual navigation method based on an unordered RGB image set according to claim 1, characterized in that, Determining the candidate retrieval distance threshold based on the global visual descriptor includes the following steps: Select a reference anchor image, and then select the effective neighbor image that is the furthest from the candidate search results from the reference anchor image; Candidate images are retrieved from the reference anchor image and the effective neighbor image respectively, and local feature matching and geometric verification are performed to obtain the number of inliers and the descriptor distance of the global visual descriptor; Clustering based on the number of inliers yields high-confidence clusters and second-highest-confidence clusters; The geometric verification inlier threshold is determined based on the high-confidence cluster and the second-highest-confidence cluster; The candidate retrieval distance threshold is determined based on the descriptor distance of the candidate image pairs in the high-confidence cluster.

3. The visual navigation method based on an unordered RGB image set according to claim 2, characterized in that, The process of performing local geometric verification on the candidate image pairs to obtain edge weights includes the following steps: For any candidate image pair, if the descriptor distance is less than or equal to the candidate retrieval distance threshold, local feature matching and geometric consistency verification are performed; if the number of inliers is greater than the inlier threshold, the graph edge weight is set to the number of inliers, otherwise it is set to no edge or weight is 0.

4. The visual navigation method based on an unordered RGB image set according to claim 1, characterized in that, The process of performing global structured pruning and strong loop re-insertion on the topology graph with the edge weights set to obtain a robust topology graph includes the following steps: Calculate the maximum spanning forest for the initial weighted topology graph with the edge weights set, so that the retained edges are composed of strong geometric correspondences in each connected component; Cluster the edge weights in the initial weighted topology graph and select the highest confidence clusters to determine the strong loop re-insertion threshold; Edges that were not retained by the maximum generated forest but whose edge weights are higher than the strong loop re-insertion threshold are re-inserted into the initial weighted topology graph to obtain the robust topology graph.

5. The visual navigation method based on an unordered RGB image set according to claim 1, characterized in that, The step of performing belief propagation localization on the robust topology to obtain posterior beliefs includes the following steps: The graph nodes on the robust topology graph are used as candidate positions to maintain the node belief distribution; Construct a multi-hop reachability matrix based on the graph adjacency matrix, and then normalize it in parallel to obtain the transition matrix. The predicted belief is obtained from the belief at the previous moment and the transition matrix; The observation likelihood is calculated based on the descriptor distance between the real-time RGB observation image and the image of each node, and the prediction weight and observation weight are adaptively determined based on the normalized entropy of the belief distribution at the previous time step. The posterior belief is obtained by weighted geometric mean normalization based on the prediction weight, the observation weight, and the prediction belief.

6. The visual navigation method based on an unordered RGB image set according to claim 1, characterized in that, The process of determining the current position node on the robust topology graph based on posterior beliefs and performing belief-aware goal planning to obtain a navigation path includes the following steps: The starting node is determined based on the current posterior belief, and the target node is determined based on the similarity between the global visual descriptor of the target image and the global visual descriptor of the graph node. Candidate paths are determined based on the starting node and the target node; For each candidate path, the path confidence is determined by the minimum edge weight on the path, and the widest path that maximizes the path confidence is selected as the navigation path.

7. The visual navigation method based on an unordered RGB image set according to any one of claims 1 to 6, characterized in that, The method further includes the following steps: During visual navigation, node beliefs are continuously updated. If the node with the maximum a posteriori belief deviates from the current path, or if the graph distance from the node with the maximum a posteriori belief to the current target node exceeds a preset threshold, the navigation path is re-searched from the node with the current maximum a posteriori belief to the target node.

8. A visual navigation system based on an unordered RGB image set, characterized in that, The system includes: The descriptor extraction unit is used to extract global visual descriptors from the unordered RGB image set, the target image, and the real-time RGB observation image, respectively. A threshold determination unit is used to determine a candidate retrieval distance threshold based on the global visual descriptor; The edge weight determination unit is used to filter candidate image pairs from the unordered RGB image set based on the candidate retrieval distance threshold, and perform local geometric verification on the candidate image pairs to obtain edge weights; The topology graph construction unit is used to perform global structured pruning and strong loop re-insertion based on the topology graph with the edge weights set, to obtain a robust topology graph; A belief determination unit is used to perform belief propagation localization on the robust topology graph to obtain posterior beliefs. The path navigation unit is used to determine the current position node on the robust topology graph based on posterior beliefs and to perform belief-aware goal planning to obtain a navigation path.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the method as described in any one of claims 1 to 7.

Images