A method of policy directed visual loop detection

Abstract

The application provides a method for strategy-guided visual loop detection, which firstly utilizes scene data stream time correlation, learns the rule of loop frame distribution in a specific scene, so that the most similar loop candidate frame can be directly obtained from the current frame and the historical frame, and tedious comparison one by one is avoided; then a loop candidate frame refinement strategy is introduced, the reliability of the selected candidate frame is obtained, and the historical frame picture with the maximum probability is selected as the final loop candidate frame; it can be seen that the application provides a new method for loop detection in visual real-time positioning and mapping, a loop candidate frame selection strategy is trained, which will respond according to the current frame and the frame that has been selected, select the most similar frame as the next loop candidate frame, can find the loop frame of the current frame in visual real-time positioning and mapping, and combine the optimization algorithm to reduce the cumulative error, so that the positioning and mapping are more accurate.

Description

Technical Field

[0001] This invention belongs to the field of perception and localization technology in robotics, and particularly relates to a strategy-guided visual loop closure detection method. Background Technology

[0002] Visual loop closure detection is an essential module in Visual Simultaneous Localization and Mapping (VSLAM). VSLAM is a technique that uses camera perception data to simultaneously estimate camera motion and build a map. It's a vision-based simultaneous localization and mapping technology that enables autonomous navigation and localization in unknown environments. In VSLAM, the camera acquires perception data by capturing images of its surroundings. Image processing and computer vision techniques are then used to extract useful information, such as feature points and edges, and these are tracked against previous images to determine camera motion. Simultaneously, a map is built, which can be a point cloud, a topological map, or other forms of mapping. The map can represent the camera's previous trajectory, estimate camera motion, and be updated by matching previously detected feature points with new observations.

[0003] The general framework is divided into two parts: front-end and back-end. The front-end's main responsibilities are tracking and mapping. It primarily relies on computer vision techniques, such as extracting ORB and SIFT feature points as representations of the current video frame, and then using feature point descriptors to achieve feature point registration, thereby enabling feature point tracking. Simultaneously, it utilizes multi-view geometry techniques to calculate the 3D coordinates of the feature points and the current camera pose, completing both tracking and mapping. Feature points extracted by VSLAM based on feature points are relatively sparse, resulting in a sparse map. In contrast, direct VSLAM methods utilize pixel-level information as the basis for tracking between consecutive frames and reconstruct a point cloud representation of the scene, resulting in a denser map.

[0004] The VSLAM front-end calculates camera pose and feature point 3D coordinates based on sensor data. However, sensor input is noisy, causing the calculated camera pose and feature point 3D coordinates to also be noisy. Furthermore, as the camera moves, the noise in each result accumulates, causing the difference between the camera pose and the ground truth to widen. This leads to a growing discrepancy between the constructed map and the actual map; this error is called cumulative error. The back-end's role is to use loop closure detection to reduce the impact of cumulative error. When the robot explores a previously explored location, the positions of the two should be similar in the global coordinate system. However, due to cumulative error, the newly obtained pose differs from the previously obtained pose. Graph optimization techniques can distribute the cumulative error across the poses between two frames, resulting in a more accurate map. Therefore, finding accurate loop closure frames in the current frame through loop closure detection is crucial. Currently, the general framework for loop closure detection consists of two stages: the first stage finds candidate loop closure frames similar to the current frame from historical frames, and the second stage further selects more accurate loop closure frames from these candidate frames. Current methods differ primarily in the first stage, while the second stage mainly relies on local feature points for geometric verification. Loop closure candidate frame selection methods are mainly divided into three categories: traditional feature point-based, deep learning-based, and semantic-based methods. Traditional feature point-based methods utilize a bag-of-words model to obtain bag-of-words vectors for the image as global features. Based on these global features, they search for the most similar frames from historical frames as loop closure candidates. For example, methods like ORB-SLAM2 and FAB-MAP2.0 use a visual bag-of-words model for loop closure candidate detection. The bag-of-words model typically performs unsupervised learning on images of various categories to obtain a visual dictionary. This visual dictionary can be used to create a visual word histogram that can serve as a global feature representation. The advantage of this offline training method is its applicability across various scenarios, but it may fail in certain specific scenarios. Therefore, some methods train the dictionary online, resulting in different dictionaries for different scenarios, thus achieving higher accuracy. Deep learning-based methods use neural networks to extract global or local features as the representation of the current frame and use these features to search for loop closure candidate frames from historical frames. Liu et al. proposed a method to simultaneously extract local and global features for loop closure candidate detection. Furthermore, semantic information exists in most scenes, which can be used to improve the model's robustness to lighting, dynamic objects, and angle changes. Based on the ORB-SLAM framework, Li et al. created a semantic map and trained a network capable of inferring the orientation of object blocks in images. This model was able to detect loop closures in two images where the object angles changed by 125 degrees.

[0005] The main purpose of loop closure detection is to find reliable loop closure frames, and then to correct the camera pose and the positions of the involved 3D map points between two loop closure frames by solving an optimization problem, thereby reducing accumulated errors. Existing loop closure detection methods mainly consist of two stages. The first stage selects some reliable candidate frames from historical frames, and the second stage selects more accurate loop closure frames from the candidate frames selected in the first stage using feature point information. Therefore, the selection of candidate frames in the first stage is very important. If the selection criteria are too low, it will lead to an increase in the computational space of the second stage and consume more computational resources.

[0006] In the process of detecting loop closure candidate frames, both bag-of-words models and methods using neural networks compare the global features of the current frame and historical frames one by one to filter for relatively reliable loop closure candidate frames. However, among a large number of historical frames, usually only a small portion are reliable loop closure candidate frames. This indicates that a large number of invalid comparisons are performed in historical frames to obtain reliable loop closure candidate frames, and these methods do not take into account the temporal information in the historical sequence. Summary of the Invention

[0007] To address the aforementioned issues, this invention provides a strategy-guided visual loop closure detection method that utilizes temporal contextual information to predict the possible locations of loop closure candidates, thereby avoiding inefficient frame-by-frame comparisons.

[0008] A strategy-guided visual loop closure detection method sequentially acquires loop closure frames by treating each frame as the current frame in chronological order. The loop closure frame acquisition operation is as follows:

[0009] S1: Input the current frame image and the historical frame images before the current frame image into the feature representation network to obtain the feature vectors of the current frame image and the historical frame images;

[0010] S2: The images in the historical frames that have not been selected as loop closure candidate frames are formed into a candidate set, and the images that have been selected as loop closure candidate frames are formed into a selected set. At the same time, the feature vectors corresponding to the candidate set, the feature vectors corresponding to the current frame image, and the feature vectors corresponding to the selected set are input into the policy network to obtain the probability distribution of each historical frame image in the candidate set being selected as a loop closure candidate frame, and the historical frame image with the highest probability is selected as the loop closure candidate frame for the current frame image.

[0011] S3: Remove the currently selected images as loop closure candidate frames from the candidate set and add them to the selected set to complete the update of the candidate set and the selected set. Repeat step S2 with the updated candidate set and the selected set until the set number of loop closure candidate frames are obtained.

[0012] S4: Perform geometric verification and temporal consistency verification on all loopback candidate frames, and select the frame with the highest inlier rate from the verified loopback candidate frames as the loopback frame of the current frame image.

[0013] Furthermore, the method for obtaining the feature representation network is as follows:

[0014] S11: Combine all frame images in the training set into any two pairs to obtain all possible image pairs, and each image pair has similar or dissimilar labels.

[0015] S12: Input the photos from each image pair into the feature representation network in sequence to obtain the feature vector pairs corresponding to each image pair;

[0016] S13: Calculate the similarity between the two feature vectors in each feature vector pair. Obtain the loss function of the feature representation network based on the similarity of each image pair and the label. Determine whether the loss function is less than the set value. If yes, the feature representation network under the current network parameters is the final feature representation network. If no, update the network parameters of the feature representation network based on backpropagation of the loss function.

[0017] S14: Repeat steps S12 to S13 using the updated feature representation network until the final feature representation network is obtained.

[0018] Furthermore, the loss function for:

[0019]

[0020] in, The tags representing image pairs; if they are similar tags, then... If the tags are dissimilar, then The method for determining whether any image pair is labeled as similar or dissimilar is as follows: if the difference in frame number between the two images in the image pair is not greater than 20, then the image pair is labeled as similar; otherwise, it is labeled as dissimilar. d represents the cosine similarity of the feature vectors of the two images in the image pair; m represents a set constant greater than 0.

[0021] Furthermore, the method for obtaining the policy network is as follows: a feature representation network is used to obtain the feature vectors of all frame images in the training set, and each feature vector is used as the feature vector of the current frame to perform the following operations:

[0022] S21: Input the feature vectors of the historical frames before the current frame feature vector, the feature vector of the current frame, and the feature vectors that have been selected as candidate frames for loop closure into the policy network to obtain the probability distribution of each historical frame feature vector being selected as the most similar feature vector. Based on the probability distribution, randomly select a frame feature vector as the candidate frame feature vector.

[0023] S22: Obtain the reward value corresponding to the candidate feature vector.

[0024]

[0025] Where t represents the time to find the candidate frame feature vector corresponding to the current frame feature vector, and f q Represents the feature vector of the current frame. Represents the feature vector of the candidate frame. This represents the similarity between the feature vector of the current frame and the feature vector of the candidate frame, and the similarity is calculated as follows:

[0026]

[0027] Where α represents the set weight. This represents the cosine value between the feature vector of the current frame and the feature vector of the candidate frame. This indicates the number of feature points that match between the current frame's feature vector and the candidate frame's feature vector;

[0028] S23: Record reward value Then update the selected set and candidate set, repeating S21 to S22 until N candidate feature vectors are selected, and calculate the expected reward. Determine expected reward If the value is greater than the set value, then the policy network under the current network parameters is the final policy network; otherwise, the expected reward is used. The network parameters of the policy network are updated using gradient ascent.

[0029] S24: Repeat steps S21 to S23 using the updated policy network until the expected reward is achieved. If the value is greater than the set value, the final policy network is obtained.

[0030] Furthermore, in step S21, the feature vectors of each historical frame are selected as the probability distribution π of the most similar feature vector. w (a t |s t The calculation method for ) is as follows:

[0031]

[0032] Among them, a tThis represents the action of selecting candidate frame feature vectors from historical frame feature vectors, s t This indicates the current state, which includes the selection of the current frame feature vector, historical frame feature vectors, and candidate frame feature vectors. The action space represents the current state, and the action space is composed of feature vectors from the historical frame feature vectors that have not been selected as loopback candidate frames. w is the network parameter of the policy network, and T represents the transpose.

[0033] Furthermore, before performing geometric verification and temporal consistency verification on all loopback candidate frames, the credibility of all loopback candidate frames is first determined. Only loopback candidate frames with a credibility greater than a set value are subjected to geometric verification and temporal consistency verification. The method for determining the credibility of each loopback candidate frame is as follows:

[0034] Arrange the current loop closure candidate frames according to their own frame numbers on the original image frame sequence. Expand the window by a set number of frames before and after the current loop closure candidate frame to obtain an expansion window of a set frame length. The sum of the other loop closure candidate frames that fall into the expansion window and the current loop closure candidate frame is used as the confidence level of the current loop closure candidate frame.

[0035] Furthermore, before performing geometric verification and temporal consistency verification on all loopback candidate frames, all loopback candidate frames are first expanded and merged to obtain candidate segments. Then, geometric verification and temporal consistency verification are performed on the candidate segments. The expansion and merging method is as follows:

[0036] Each loopback candidate frame is expanded by W frames before and after it to form a candidate segment of length 2W.

[0037] Determine whether there is frame overlap among the candidate segments. If so, merge the overlapping candidate segments.

[0038] Furthermore, in step S3, the number of loopback candidate frames is set to 10.

[0039] Beneficial effects:

[0040] 1. This invention provides a strategy-guided visual loop closure detection method. First, it utilizes the temporal correlation of scene data streams to learn the distribution patterns of loop closure frames in a specific scene. Given the current frame and historical frames, it can directly obtain the most similar loop closure candidate frame, avoiding tedious one-by-one comparisons. Then, it introduces a strategy for refining loop closure candidate frames, obtaining the reliability of the selected candidate frames, and selecting the historical frame image with the highest probability as the final loop closure candidate frame. Therefore, this invention provides a novel method for loop closure detection in real-time visual localization and mapping. It trains a loop closure candidate frame selection strategy that reacts based on the current frame and already selected frames, selecting the most similar frame as the next loop closure candidate frame. This enables the identification of loop closure frames in the current frame during real-time visual localization and mapping, and, combined with optimization algorithms, reduces accumulated errors, thereby making localization and mapping more accurate.

[0041] 2. This invention provides a strategy-guided visual loop closure detection method, which further filters the initially selected loop closure candidate frames based on their credibility, leaving only those that meet the credibility requirements for the next stage, thereby improving the accuracy of loop closure detection.

[0042] 3. This invention provides a strategy-guided visual loop closure detection method, which expands loop closure candidate frames into candidate segments, and then performs geometric verification. Only frames that pass multiple verifications can be used as loop closure frames. Finally, the loop closure frame with the highest inlier rate is selected as the final loop closure frame, which can further improve the recall rate and balance computational efficiency and algorithm performance.

[0043] 4. This invention provides a strategy-guided visual loop closure detection method, which sets the number of loop closure candidate frames to 10 frames. This number can ensure that the number of loop closure candidate frames left after geometric verification is sufficient to calculate the relative pose to eliminate the cumulative error in real-time visual localization and mapping, and can also reduce the computational load of subsequent geometric verification and improve the efficiency of loop closure detection. Attached Figure Description

[0044] Figure 1 This forms the implementation framework for the present invention;

[0045] Figure 2 A flowchart of a strategy-guided visual loop closure detection method provided by the present invention;

[0046] Figure 3 This is a comparison between the training results and ground truth values ​​of the feature representation network of this invention;

[0047] Figure 4 This is the process of loopback candidate frame screening in this invention;

[0048] Figure 5 This invention describes the process of calculating the confidence level of loopback candidate frames and constructing candidate segments based on loopback candidate frames. Detailed Implementation

[0049] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0050] This invention consists of two stages. The first stage is the selection of loop closure candidate frames, and the second stage is image-level refinement. The first stage identifies reliable candidate frame regions based on the learned strategy and selects reliable candidate frames from these regions. The second stage uses feature points to perform geometric verification on the candidate frames obtained in the first stage, calculates the fundamental matrix and inlier rate to further select loop closure candidate frames, and simultaneously calculates the relative pose between the current frame and the loop closure frame as the target for subsequent optimization.

[0051] First, the problem model for the policy-based loop closure candidate frame detection of this invention is as follows:

[0052] Traditional methods for selecting loop closure candidate frames evaluate the similarity between the current frame and historical frames one by one, and then select historical frames that meet certain conditions as loop closure candidates based on the similarity. Traditional retrieval methods do not utilize the long-term temporal continuity and the distribution of loops in the scene. Therefore, this invention aims to integrate the distribution of loops in the scene into the selection strategy in specific scenarios. The process of selecting loop closure candidate frames is viewed as a Markov decision process, which includes five parts: state, action, state transition function, reward, and policy. The process of selecting loop closure candidate frames can be seen as follows: given the current state, the most reasonable action needs to be selected and executed according to the policy. That is, a loop closure candidate frame is continuously selected from the historical frames. After executing an action, the state changes and a corresponding reward is obtained. Then, the action is selected again according to the policy and executed until a certain number of loop closure candidate frames are obtained.

[0053] The state consists of four parts: the current frame, historical frames, the previous selection, and the candidate frame selected last time. All states constitute the state space. An action represents selecting a frame from the historical frames as a candidate frame, given the current frame, such as... Figure 1 As shown, given a state, performing a series of actions yields a set of candidate frames for loop closure. When a given state is given, performing an action changes the state according to the state transition function, resulting in a new state. Specifically, the previously selected frame changes compared to the last selected frame. The reward reflects the value of the current action and is related to the ranking of the selected candidate frame and the similarity between the candidate frames.

[0054] The strategy represents how to select an action given the current state. The action selection space is the frames that have not been selected in the history frames. We mask the previously selected frames in the history frames. This invention represents the strategy as a probability distribution of the history frames, and finally selects the frame with the highest probability as the action that should be executed now.

[0055] Therefore, the first stage of this invention involves two parts. The first part is the feature representation part, which maps the image set to a vector space. The goal is for similar images to have similar vector representations, and dissimilar images to have significantly different vector representations. This invention uses a convolutional neural network to achieve this through adversarial learning, resulting in good feature representations. Training the strategy is then necessary. This invention uses reinforcement learning to train the loop closure candidate frame selection strategy. Furthermore, the method of this invention has the following main contributions: (a) It proposes a novel visual loop closure detection method for specific scenarios; (b) It trains a loop closure candidate frame selection strategy that utilizes temporal contextual information to predict the possible locations of loop closure candidates, avoiding inefficient frame-by-frame comparisons; (c) It introduces a refined loop closure candidate frame strategy, which uses temporal information to select loop closure candidate frames and performs certain expansion to ensure sufficient recall.

[0056] Based on the above problem model, the present invention provides a method such as... Figure 2 As shown, the strategy-guided visual loop closure detection method sequentially uses each frame image as the current frame image to perform a loop closure frame acquisition operation, obtaining the loop closure frames corresponding to each frame image. The loop closure frame acquisition operation is as follows:

[0057] S1: Input the current frame image and the historical frame images before the current frame image into the feature representation network to obtain the feature vectors of the current frame image and the historical frame images;

[0058] S2: The images in the historical frames that have not been selected as loop closure candidate frames are formed into a candidate set, and the images that have been selected as loop closure candidate frames are formed into a selected set. At the same time, the feature vectors corresponding to the candidate set, the feature vectors corresponding to the current frame image, and the feature vectors corresponding to the selected set are input into the policy network to obtain the probability distribution of each historical frame image in the candidate set being selected as a loop closure candidate frame, and the historical frame image with the highest probability is selected as the loop closure candidate frame for the current frame image.

[0059] S3: Remove the currently selected images as loop closure candidate frames from the candidate set and add them to the selected set to complete the update of the candidate set and the selected set. Repeat step S2 with the updated candidate set and the selected set until the set number of loop closure candidate frames are obtained.

[0060] It should be noted that the set quantity is denoted as N. The value of N affects the quality of the candidate frames and the time consumed in subsequent geometric verification. If N is too small, the selected loop closure candidates may not be of high quality, and the loop closure frames left after geometric verification will be difficult to use to calculate the relative pose to eliminate the accumulated error. Conversely, if N is too large, the computational load of subsequent geometric verification will be very large and inefficient. Through experiments, the present invention selects N=10 to achieve a better effect.

[0061] S4: Perform geometric verification and temporal consistency verification on all loopback candidate frames, and select the frame with the highest inlier rate from the verified loopback candidate frames as the loopback frame of the current frame image.

[0062] It should be noted that after selecting the top N most similar loop closure candidates, this invention performs a candidate frame refinement operation to filter out some noisy data. Because the entire exploration environment image is temporally continuous, loops in the current frame tend to cluster in a certain segment, and loop closure candidates should also exist around the current loop closure candidate. Therefore, this invention calculates the number of loop closure candidates within a given-width window, which becomes the confidence level of the loop closure candidate frame. Finally, only those loop closure candidate frames that meet the confidence requirements are retained for the next stage. Simultaneously, after obtaining N loop closure candidate frames, it is necessary to filter out more accurate loop closure frames based on these frames and calculate their relative poses to provide parameters for the optimization equation. Loop closure candidate frames filtered according to the learned strategy have similar feature representations to the current frame, but the loop closure frames that can actually be used to calculate the relative pose may not be among the selected N loop closure candidate frames. Furthermore, the loop closure candidate frames filtered by confidence level indicate that their positions are likely similar to those of the current frame, so more accurate loop closure frames are likely to be near the loop closure candidate frames with higher confidence levels. Therefore, this invention first expands the candidate loopback frame by a portion before and after it to form a candidate segment, and then searches for the loopback frame in each candidate segment.

[0063] Based on this, before performing geometric verification and temporal consistency verification on all loopback candidate frames, the credibility of all loopback candidate frames is first determined. Then, loopback candidate frames with a credibility greater than a set value are expanded and merged to obtain candidate segments. Geometric verification and temporal consistency verification are then performed on these candidate segments. The method for determining the credibility of each loopback candidate frame is as follows:

[0064] The current loop closure candidate frame is arranged in the original image frame sequence according to its own frame number. An expanded window of a set number of frames is then added before and after the current loop closure candidate frame, resulting in an expanded window of a set frame length. The sum of the scores of the remaining loop closure candidate frames falling within the expanded window and the current loop closure candidate frame is used as the confidence level of the current loop closure candidate frame. Furthermore, through experiments, this invention has shown that a window width of 30 and a minimum confidence level of 2 yield good results.

[0065] The method for expansion and merging is as follows:

[0066] Each loopback candidate frame is expanded by W frames before and after it to form a candidate segment of length 2W.

[0067] Determine whether there is frame overlap among the candidate segments. If so, merge the overlapping candidate segments.

[0068] In other words, after obtaining N candidate loopback frames, we expand each candidate loopback frame by W frames before and after it to form a candidate segment of length 2W. If two candidate segments intersect, they are merged into a single candidate segment. Figure 5 As shown, when W=3, Five loop-loop candidate frames form two candidate segments. The candidate segments formed after the expansion of the first four loop-loop candidate frames intersect and are thus merged into one candidate segment.

[0069] Once each candidate segment is determined, the search space for geometric verification is also defined. Each frame in a candidate segment needs to undergo geometric verification with the current frame to filter for more accurate loop closure frames. ORB feature points are extracted from each frame in the candidate segment and the current frame. If there are fewer than 8 pairs of feature points, the frame is discarded; otherwise, the fundamental matrix is ​​calculated using RANSAC. Frames that fail to calculate the fundamental matrix or whose inlier rate is less than a threshold are also discarded. The remaining frames then need to undergo temporal consistency verification before they have a chance to be selected as loop closure frames.

[0070] Because the camera's data stream is continuous in time, the loopback frame of the current frame is very likely to be a loopback frame of a frame surrounding it. Considering that the NewCollege and CityCentre datasets are arranged with alternating left and right camera views, if the loopback frame of the previous frame is near the currently checked frame, then the currently checked frame is considered to have passed the temporal consistency check. Finally, the frame with the highest inlier rate is selected from the frames that have passed the geometric check as the loopback frame of the current frame.

[0071] The following describes the methods for obtaining feature representation networks, such as... Figure 4 As shown, the specific steps include:

[0072] S11: Combine all frame images in the training set into any two pairs to obtain all possible image pairs, and each image pair has similar or dissimilar labels.

[0073] It should be noted that the feature representation network and the decision network are trained using the same datasets. This invention selects the NewCollege and CityCentre datasets, which respectively include 1073 and 1273 pairs of photos taken by two cameras roaming in the scene. The photos taken by the left and right cameras are arranged alternately, and each photo is numbered starting from 1. A ground truth matrix is ​​provided for each dataset. The ground truth matrix is ​​a two-dimensional binary matrix, where a value of 0 indicates that the two frames corresponding to the index are not in a loop, and a value of 1 indicates that the two frames corresponding to the index are in a loop. Specifically, the feature representation part of this invention uses 30% of the data from both datasets as the training set, while the policy decision part uses 50% of the data as the training set.

[0074] S12: Input the photos from each image pair into the feature representation network in sequence to obtain the feature vector pairs corresponding to each image pair;

[0075] It should be noted that the feature representation network of this invention can use the classic ResNet50 as the backbone network, with two fully connected layers following ResNet50 to reduce the feature dimension of the ResNet50 output from 2048 to 128. Feature representation is performed through adversarial learning, i.e., a pair of photos is input, labeled as similar or dissimilar, and the output is two 128-dimensional feature vectors. The loss function is calculated based on the similarity between the labels and the two feature vectors, and the network parameters are updated through backpropagation. Furthermore, since the training data of the feature representation network is pair of photos, if the value of a photo in the ground truth matrix is ​​1 or the two images are close in number (within 20), then the two frames are considered close, and the label of the pair of photos is similar; otherwise, they are considered dissimilar, and equal amounts of positive and negative samples are used for training.

[0076] S13: Calculate the similarity between the two feature vectors in each feature vector pair. Based on the similarity of each image pair and the label, obtain the loss function of the feature representation network. Determine if the loss function is less than a set value. If yes, the feature representation network with the current network parameters is the final feature representation network; otherwise, update the network parameters of the feature representation network through backpropagation based on the loss function. Wherein, the loss function... for:

[0077]

[0078] in, The tags representing image pairs; if they are similar tags, then... If the tags are dissimilar, then The method for determining whether an image pair is labeled as similar or dissimilar is as follows: if the difference in frame number between the two images in the pair is no greater than 20, the pair is labeled as similar; otherwise, it is labeled as dissimilar. `d` represents the cosine similarity of the feature vectors of the two images in the pair; `m` represents a constant greater than 0, used to limit the cosine similarity. The smaller the expected loss function, the greater the cosine similarity of similar images should be, and the less the cosine similarity of dissimilar images should be, the less than -m. Through a series of experiments, this invention ultimately selected `m = 0.79`. Figure 3 As shown, Figure 3 (b) is the label of an image segment in the CityCentre dataset, where the white portion indicates that the two frames form a loop, meaning that the two frames are similar. Figure 3 (a) shows the cosine similarity of the image feature representations in CityCentre after training. It can be seen that the similarity of the image feature representations is similar to the ground truth, indicating that the trained image feature representations can better measure the similarity of images.

[0079] S14: Repeat steps S12 to S13 using the updated feature representation network until the final feature representation network is obtained.

[0080] After obtaining a good feature representation, the policy decision-making part needs to be trained. The following describes the method for obtaining the policy network. This invention uses reinforcement learning for training, specifically including the following steps:

[0081] A feature representation network is used to obtain feature vectors for all frames in the training set. Each feature vector is then used as the feature vector for the current frame to perform the following operations:

[0082] S21: Input the feature vectors of previous historical frames, the feature vector of the current frame, and the feature vectors of the selected candidate frames for loop closure into the policy network to obtain the probability distribution of each historical frame feature vector being selected as the most similar feature vector. Based on the probability distribution, randomly select a frame feature vector as the candidate frame feature vector; the probability distribution π of each historical frame feature vector being selected as the most similar feature vector in step S21. w (a t |s t The calculation method for ) is as follows:

[0083]

[0084] Among them, a t This represents the action of selecting candidate frame feature vectors from historical frame feature vectors, s t This indicates the current state, which includes the selection of the current frame feature vector, historical frame feature vectors, and candidate frame feature vectors. The action space represents the current state, and the action space consists of feature vectors from the historical frame feature vectors that have not been selected as the most similar feature vectors. w is the network parameter of the policy network, and T represents the transpose.

[0085] It should be noted that this invention uses three fully connected layers to represent the policy, and the parameters of the fully connected layers are updated using the REINFORCE algorithm, the purpose of which is to maximize the cumulative reward.

[0086]

[0087] in It is an action selected in one round. It is a cumulative reward for one round:

[0088]

[0089] Therefore, the gradient of J(w) is:

[0090]

[0091] Among them G t This is the cumulative discounted reward from time t to the end of this round. This invention updates parameters based on the gradient of the cumulative reward for a round and the learning rate:

[0092]

[0093] S22: Obtain the reward value corresponding to the candidate feature vector.

[0094]

[0095] Where t represents the time to find the candidate frame feature vector corresponding to the current frame feature vector, and f q Represents the feature vector of the current frame. Represents the feature vector of the candidate frame. This represents the similarity between the feature vector of the current frame and the feature vector of the candidate frames. It can be seen that the more similar the frames are, the earlier they are found, and the larger the reward function will be. The similarity is calculated as follows:

[0096]

[0097] Where α represents the set weight. This represents the cosine value between the feature vector of the current frame and the feature vector of the candidate frame. It represents the number of feature points that match between the feature vector of the current frame and the feature vector of the candidate frame. The more similar the feature representations of the two frames are, the more similar the two frames are. The more feature points that can be extracted from the two frames, the higher the reliability of the similarity measurement between the two frames.

[0098] S23: Record reward value Then update the selected set and candidate set, repeating S21 to S22 until N candidate feature vectors are selected, and calculate the expected reward. Determine expected reward If the value is greater than the set value, then the policy network under the current network parameters is the final policy network; otherwise, the expected reward is used. The network parameters of the policy network are updated using gradient ascent.

[0099] S24: Repeat steps S21 to S23 using the updated policy network until the expected reward is achieved. If the value is greater than the set value, the final policy network is obtained.

[0100] At this point, both the feature representation network and the policy network have completed training. This invention will utilize the trained network to detect loop closure candidate frames online. Specifically, each action's probability distribution is represented using a probability distribution formula, and the action with the highest probability is selected for execution, until the top N most similar loop closure candidate frames are selected. It should be noted that experiments on two publicly available datasets have shown that the results outperform current state-of-the-art methods, demonstrating the effectiveness of this invention.

[0101] In summary, previous visual loop closure detection algorithms, whether offline or online bag-of-words or deep learning-based algorithms, all require comparing the feature representations of the current frame with those of historical frames one by one, even though most of these comparisons are unnecessary. This is because previous methods did not utilize the semantic information related to the preceding and following data streams in the scene. This invention proposes a novel framework for visual loop closure detection in specific scenarios. It models the selection of loop closure candidate frames as a Markov decision process and utilizes the temporal correlation of scene data streams to learn the distribution pattern of loop closure frames in specific scenarios. Thus, given the current frame and historical frames, it can directly obtain the most similar loop closure candidate frames, avoiding the tedious one-by-one comparisons.

[0102] Meanwhile, this invention comprehensively considers time efficiency and performance, proposing a method for screening loop closure candidate frames based on their credibility. It also expands the loop closure candidate frames into segments to increase the search space. Through layers of screening, loop closure candidate frames with high credibility can be obtained. To further improve recall, this invention expands the loop closure candidate frames into candidate segments, and then performs geometric verification, etc. Only frames that pass multiple layers of verification can be used as loop closure frames. Finally, the loop closure frame with the highest inlier rate is selected as the loop closure frame. The steps of screening loop closure candidate frames mainly consider accuracy and computational efficiency, while screening loop closure frames based on candidate frames mainly considers recall. This approach ultimately achieves good performance in both recall and accuracy.

[0103] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A strategy-guided visual loop closure detection method, characterized in that, The loopback frame acquisition operation is performed sequentially, treating each frame as the current frame to obtain the corresponding loopback frame. The loopback frame acquisition operation is as follows: S1: Input the current frame image and the historical frame images before the current frame image into the feature representation network to obtain the feature vectors of the current frame image and the historical frame images; S2: The images in the historical frames that have not been selected as loop closure candidate frames are formed into a candidate set, and the images that have been selected as loop closure candidate frames are formed into a selected set. At the same time, the feature vectors corresponding to the candidate set, the feature vectors corresponding to the current frame image, and the feature vectors corresponding to the selected set are input into the policy network to obtain the probability distribution of each historical frame image in the candidate set being selected as a loop closure candidate frame, and the historical frame image with the highest probability is selected as the loop closure candidate frame for the current frame image. The method for obtaining the policy network is as follows: a feature representation network is used to obtain the feature vectors of all frames in the training set, and each feature vector is used as the feature vector of the current frame to perform the following operations: S21: Input the feature vectors of the historical frames before the current frame feature vector, the feature vector of the current frame, and the feature vectors that have been selected as candidate frames for loop closure into the policy network to obtain the probability distribution of each historical frame feature vector being selected as the most similar feature vector. Based on the probability distribution, randomly select a frame feature vector as the candidate frame feature vector. S22: Obtain the reward value corresponding to the candidate feature vector. : in, This indicates the time required to find the candidate frame feature vector corresponding to the current frame feature vector. Represents the feature vector of the current frame. Represents the feature vector of the candidate frame. This represents the similarity between the feature vector of the current frame and the feature vector of the candidate frame, and the similarity is calculated as follows: in, This indicates that weights are set. This represents the cosine value between the feature vector of the current frame and the feature vector of the candidate frame. This indicates the number of feature points that match between the current frame's feature vector and the candidate frame's feature vector; S23: Record reward value Then update the selected set and the candidate set, repeating S21~S22 until N candidate feature vectors are selected, and calculate the expected reward. Determine expected reward If the value is greater than the set value, then the policy network under the current network parameters is the final policy network; otherwise, the expected reward is used. The network parameters of the policy network are updated using gradient ascent. S24: Repeat steps S21-S23 using the updated policy network until the expected reward is achieved. If the value is greater than the set value, the final policy network is obtained; S3: Remove the currently selected images as loop closure candidate frames from the candidate set and add them to the selected set to complete the update of the candidate set and the selected set. Repeat step S2 with the updated candidate set and the selected set until the set number of loop closure candidate frames are obtained. S4: Perform geometric verification and temporal consistency verification on all loopback candidate frames, and select the frame with the highest inlier rate from the verified loopback candidate frames as the loopback frame of the current frame image.

2. The strategy-guided visual loop closure detection method as described in claim 1, characterized in that, The method for obtaining the feature representation network is as follows: S11: Combine all frame images in the training set into any two pairs to obtain all possible image pairs, and each image pair has similar or dissimilar labels. S12: Input the photos from each image pair into the feature representation network in sequence to obtain the feature vector pairs corresponding to each image pair; S13: Calculate the similarity between the two feature vectors in each feature vector pair. Obtain the loss function of the feature representation network based on the similarity of each image pair and the label. Determine whether the loss function is less than the set value. If yes, the feature representation network under the current network parameters is the final feature representation network. If no, update the network parameters of the feature representation network based on backpropagation of the loss function. S14: Repeat steps S12-S13 using the updated feature representation network until the final feature representation network is obtained.

3. The strategy-guided visual loop closure detection method as described in claim 2, characterized in that, The loss function for: in, The tags representing image pairs; if they are similar tags, then... If the tags are dissimilar, then The method for determining whether any image pair is labeled as similar or dissimilar is as follows: if the difference in frame number between the two images in the image pair is not greater than 20, then the image pair is labeled as similar; otherwise, it is labeled as dissimilar. This represents the cosine similarity of the feature vectors of two images in an image pair. This represents a set constant that is greater than 0.

4. The strategy-guided visual loop closure detection method as described in claim 1, characterized in that, In step S21, the probability distribution of each historical frame feature vector being selected as the most similar feature vector is... The calculation method is as follows: in, This represents the action of selecting candidate frame feature vectors from the historical frame feature vectors. This indicates the current state, which includes the selection of the current frame feature vector, historical frame feature vectors, and candidate frame feature vectors. The action space represents the current state, and it consists of feature vectors from historical frames that have not been selected as candidate frames for loop closure. w Here are the network parameters for the policy network, and T represents the transpose.

5. The strategy-guided visual loop closure detection method as described in claim 1, characterized in that, Before performing geometric verification and temporal consistency verification on all loopback candidate frames, the credibility of all loopback candidate frames is first determined. Only loopback candidate frames with a credibility greater than a set value are subjected to geometric verification and temporal consistency verification. The method for determining the credibility of each loopback candidate frame is as follows: Arrange the current loop closure candidate frames according to their own frame numbers on the original image frame sequence. Expand the window by a set number of frames before and after the current loop closure candidate frame to obtain an expansion window of a set frame length. The sum of the other loop closure candidate frames that fall into the expansion window and the current loop closure candidate frame is used as the confidence level of the current loop closure candidate frame.

6. The strategy-guided visual loop closure detection method as described in claim 1, characterized in that, Before performing geometric verification and temporal consistency verification on all loopback candidate frames, all loopback candidate frames are first expanded and merged to obtain candidate segments. Then, geometric verification and temporal consistency verification are performed on the candidate segments. The expansion and merging method is as follows: Each loopback candidate frame is expanded by W frames before and after it to form a candidate segment of length 2W. Determine whether there is frame overlap among the candidate segments. If so, merge the overlapping candidate segments.

7. A strategy-guided visual loop closure detection method as described in any one of claims 1 to 6, characterized in that, In step S3, the number of loopback candidate frames is set to 10.

Images