A visual inertial slam loop detection method and system based on geomagnetic information optimization

Abstract

The application discloses a kind of visual inertial SLAM loop detection method and system based on geomagnetic information optimization, first to the data collected is solved coarse matching on low resolution level and obtains first geomagnetic sequence matching result, then first geomagnetic sequence matching result is optimized in local neighborhood and obtains second geomagnetic sequence matching result, then based on second geomagnetic sequence matching result introduces key frame image and calculates the modulus of geomagnetic intensity corresponding to the time stamp of key frame image;The application realizes the function of first coarse screening and then fine matching in loop detection with the way of deep fusion of high-efficiency geomagnetic sequence matching algorithm and visual key frame, and through Fast-EDLines algorithm, gradient calculation acceleration, edge detection, line extraction and verification can be carried out, ensuring the accuracy and high confidence of line feature extraction, and through the use of FastDTW algorithm and geomagnetic sequence matching and dynamic adjustment of key frame buffer, efficient and robust loop detection can be carried out, which is suitable for wide promotion and use.

Description

Technical Field

[0001] This invention relates to the field of SLAM loop closure detection technology, specifically to a visual-inertial SLAM loop closure detection method and system based on geomagnetic information optimization. Background Technology

[0002] Visual SLAM is a key technology that endows machines with "visual perception" and "spatial cognition" capabilities. By analyzing continuous image streams, it calculates its own motion in real time and constructs environmental maps, providing fundamental spatial understanding and localization capabilities for various autonomous systems and interactive applications. Despite numerous challenges, with advancements in algorithmic innovation, computing hardware development, and multi-sensor fusion, visual SLAM is becoming increasingly robust, accurate, and practical, and its application scope continues to expand.

[0003] Currently, visual SLAM technology still faces many challenges in practical applications. Traditional methods, especially in the loop closure detection stage, are prone to false positives and false negatives due to reliance on single visual information, especially in environments with low light, missing textures, and unstable handheld devices. Existing methods generally rely on image matching and spatiotemporal consistency verification, which makes it difficult to guarantee detection accuracy in situations with drastic changes in lighting or high scene repetition. Furthermore, traditional methods that rely on point features for pose estimation are easily interfered with in low-texture or repetitive texture environments, thereby reducing the front-end tracking performance. Therefore, it is necessary to design a visual inertial SLAM loop closure detection method and system based on geomagnetic information optimization. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and to better and more effectively solve the problems of false detection and missed detection in visual SLAM under low light, texture loss, and unstable handheld devices, especially in the loop closure detection stage, which often relies on a single visual information. Existing methods generally rely on image matching and spatiotemporal consistency verification, which makes it difficult to guarantee detection accuracy under conditions of drastic lighting changes or high scene repetition. In addition, traditional methods that rely on point features for pose estimation are easily interfered with in low-texture or repetitive texture environments, thereby reducing the front-end tracking performance. This invention provides a visual inertial SLAM loop closure detection method and system based on geomagnetic information optimization. It realizes the function of first coarse screening and then fine matching in loop closure detection by using an efficient geomagnetic sequence matching algorithm and deep fusion of visual keyframes. Furthermore, the Fast-EDLines algorithm can accelerate gradient calculation, edge detection, line segment extraction and verification, ensuring the accuracy and high confidence of line feature extraction. By adopting the FastDTW algorithm and geomagnetic sequence matching and dynamic adjustment of the keyframe buffer, efficient and robust loop closure detection can be achieved.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A visual-inertial SLAM loop closure detection method optimized based on geomagnetic information includes the following steps:

[0007] Step A: Solve the coarse matching at a low resolution level for the collected data to obtain the first geomagnetic sequence matching result;

[0008] Step B: Optimize the first geomagnetic sequence matching result within a local neighborhood to obtain the second geomagnetic sequence matching result;

[0009] Step C: Based on the second geomagnetic sequence matching result, keyframe images are introduced and the geomagnetic intensity modulus value of the corresponding timestamp of the keyframe image is calculated to obtain keyframe images containing geomagnetic intensity modulus values;

[0010] Step D: Store the keyframe images containing geomagnetic intensity modulus values ​​into the buffer and form the current buffer visual geomagnetic sequence;

[0011] Step E: Calculate the distance matrix between the current buffer visual geomagnetic sequence and the historical buffer visual geomagnetic sequence, and then use dynamic programming to calculate the cumulative distance matrix to obtain the optimal matching path and the final matching distance;

[0012] Step F involves selecting the region from the closest sequence based on the obtained optimal matching path and final matching distance for visual feature matching using the visual bag-of-words model. Then, the visual bag-of-words model is used to perform visual feature matching on the selected region and determine whether there is a loop, thus completing the visual inertial SLAM loop detection task.

[0013] The aforementioned visual-inertial SLAM loop closure detection method based on geomagnetic information optimization includes step A, which involves solving a coarse match at a low resolution level for the acquired data to obtain the first geomagnetic sequence matching result. The specific steps are as follows:

[0014] Step A1 involves downsampling the original time series data and then averaging adjacent sampling points to generate a low-resolution version. The downsampling process halves the sequence length while preserving the overall shape and trend of the original sequence.

[0015] Step A2 involves using the DTW algorithm at the lowest resolution level to calculate the initial coarse matching path and obtain the first geomagnetic sequence matching result. The specific steps are as follows:

[0016] Step A21: Calculate the Euclidean distance between each point in the two sequences and construct the distance matrix, as shown in formula (1).

[0017] D i，j =d(i,j)=|p i -q j | (1)

[0018] Among them, D i,j The cumulative Euclidean distance is given, where i and j are indices, d(i, j) is the local cost function, and p... i For the database sequence P(p1, p2, ..., p...), ... i , ..., p m The element in ) , q i For querying the sequence Q(q1, q2, ..., q... j , ..., q n Elements in )

[0019] Step A22 involves recursively calculating the cumulative minimum distance path using dynamic programming. Then, the similarity between the two sequences is measured by calculating the sum of the Euclidean distances of all points on the matching path. Specifically, a path with the minimum Euclidean distance is found between the starting point (1, 1) and the point (m, n), satisfying boundary constraints, continuity constraints, and monotonic constraints. The boundary constraints specifically state that the starting point of the alignment path is the lower left corner (1, 1) of the cumulative matrix, and the ending point is the upper right corner (m, n) of the cumulative matrix. The continuity constraints specifically state that the points on the alignment path are continuous, and the indices of the two sequences on the time axis can only gradually increase and can only move one step at a time. The monotonic constraints specifically state that the alignment path is monotonically non-decreasing, and the indices of the two sequences cannot regress, as shown in formula (2).

[0020]

[0021] Among them, D i-1,j D i,j-1 and D i-1,j-1 Let be the cumulative Euclidean distances between points (i-1, j), (i, j-1), and (i-1, j-1), respectively.

[0022] The aforementioned visual-inertial SLAM loop closure detection method based on geomagnetic information optimization, step B, optimizes the first geomagnetic sequence matching result within a local neighborhood to obtain the second geomagnetic sequence matching result. The specific steps are as follows.

[0023] Step B1: Project the first geomagnetic sequence matching result obtained at low resolution onto a higher resolution sequence, and then map and extend each low resolution point to a local neighborhood within a preset radius, thereby limiting the search area for high resolution matching.

[0024] Step B2: In the local neighborhood within the preset radius, the DTW algorithm with spatial constraints is used to refine the initial alignment path locally and obtain the second geomagnetic sequence matching result. The second geomagnetic sequence matching result is then used as the initial path for a higher resolution level.

[0025] The aforementioned visual-inertial SLAM loop closure detection method based on geomagnetic information optimization includes step C, which involves introducing keyframe images based on the second geomagnetic sequence matching results and calculating the geomagnetic intensity modulus value of the corresponding timestamp of the keyframe images to obtain keyframe images containing the geomagnetic intensity modulus value. The specific steps are as follows.

[0026] Step C1: Introduce keyframe images. The selection criteria for keyframe images include motion changes, parallax changes, and time intervals. The keyframe images are used to reduce computational redundancy, ensure real-time balance, ensure accuracy balance, and enhance stability in violent motion or harsh environments.

[0027] Step C2: Calculate the geomagnetic intensity modulus of the keyframe image corresponding to the timestamp. Specifically, let the geomagnetic field at a certain point indoors be represented by the components of the three-axis coordinate system (m). x m y m z If the geomagnetic intensity modulus M is as shown in formula (3), then

[0028]

[0029] In the aforementioned visual-inertial SLAM loop closure detection method based on geomagnetic information optimization, step D involves storing keyframe images containing geomagnetic intensity modulus values ​​into a buffer and forming a current buffer visual-geomagnetic sequence. Specifically, keyframe images containing geomagnetic intensity modulus values ​​are stored at the midpoint of the buffer, and information from preceding and following frames is accumulated. If the accumulated information from preceding and following frames meets a set buffer capacity threshold, a current buffer visual-geomagnetic sequence is formed. The buffer capacity threshold is dynamically adjusted according to the magnitude of the carrier parallax change and the stability of the geomagnetic environment.

[0030] The aforementioned visual inertial SLAM loop closure detection method based on geomagnetic information optimization, in step F, selects the region for visual feature matching by the visual bag-of-words model from the closest sequence based on the obtained optimal matching path and final matching distance. Then, the visual bag-of-words model is used to perform visual feature matching on the selected region and determine whether there is a loop, thus completing the visual inertial SLAM loop closure detection task. Specifically, the visual bag-of-words model is used to perform visual feature matching on the selected region, as shown in formula (4).

[0031] T new =T origin ×(1-β·(1-dis Norm ) k (4)

[0032] Among them, T new T is the threshold for the new bag-of-words model. origin The threshold of the original bag-of-words model is β, where β is the weighting factor and dis is the weighting factor. NormThis represents the normalized form of the geomagnetic sequence matching results, where k is a nonlinear factor.

[0033] A visual-inertial SLAM loop closure detection system based on geomagnetic information optimization includes a coarse matching module, a local optimization module, a keyframe introduction module, a visual geomagnetic sequence acquisition module, a cumulative distance matrix calculation module, and a loop closure judgment module. The coarse matching module solves the coarse matching problem on the acquired data at a low resolution level to obtain a first geomagnetic sequence matching result. The local optimization module optimizes the first geomagnetic sequence matching result within a local neighborhood to obtain a second geomagnetic sequence matching result. The keyframe introduction module introduces keyframe images based on the second geomagnetic sequence matching result and calculates the geomagnetic intensity modulus value corresponding to the timestamp of the keyframe image, thereby obtaining a keyframe image containing the geomagnetic intensity modulus value. The visual geomagnetic sequence acquisition module stores keyframe images containing geomagnetic intensity modulus values ​​into a buffer and forms the current buffer visual geomagnetic sequence. The cumulative distance matrix calculation module calculates the distance matrix between the current buffer visual geomagnetic sequence and the historical buffer visual geomagnetic sequence, and then uses dynamic programming to calculate the cumulative distance matrix to obtain the optimal matching path and the final matching distance. The loop closure detection module selects the region for visual feature matching using the visual bag-of-words model from the closest sequences based on the obtained optimal matching path and the final matching distance, and then uses the visual bag-of-words model to perform visual feature matching on the selected region and determine whether there is a loop, thereby completing the visual inertial SLAM loop closure detection task.

[0034] The beneficial effects of this invention are:

[0035] (1) This invention first solves the coarse matching of the collected data at a low resolution level to obtain the first geomagnetic sequence matching result. Then, the first geomagnetic sequence matching result is optimized in the local neighborhood to obtain the second geomagnetic sequence matching result. Next, based on the second geomagnetic sequence matching result, keyframe images are introduced and the geomagnetic intensity modulus value of the corresponding timestamp of the keyframe image is calculated, thereby obtaining keyframe images containing geomagnetic intensity modulus values. Then, the keyframe images containing geomagnetic intensity modulus values ​​are stored in a buffer and form the current buffer visual geomagnetic sequence. Subsequently, the distance matrix between the current buffer visual geomagnetic sequence and the historical buffer visual geomagnetic sequence is calculated. Then, the cumulative distance matrix is ​​calculated through dynamic programming to obtain the optimal matching path and the final matching distance. Then, based on the obtained optimal matching path and the final matching distance, the region is selected from the sequence with the closest distance as the region for visual feature matching using the visual bag-of-words model. Finally, the visual bag-of-words model is used to select the region. This invention performs visual feature matching and determines the presence of loop closures to complete the visual-inertial SLAM loop closure detection task. It effectively realizes that the visual-inertial SLAM loop closure detection method and system have the function of first coarse screening and then fine matching in loop closure detection by utilizing an efficient geomagnetic sequence matching algorithm and deep fusion of visual keyframes. Furthermore, the Fast-EDLines algorithm can accelerate gradient calculation, edge detection, line segment extraction, and verification, effectively fusing visual information and inertial data while ensuring the accuracy and high confidence of line feature extraction. By employing the FastDTW algorithm and dynamically adjusting the geomagnetic sequence matching and keyframe buffer, efficient and robust loop closure detection can be achieved. This invention fully utilizes the geometric information of line features in the image, combines geomagnetic data to improve positioning accuracy, and also takes into account real-time performance and stability, providing solid and reliable technical support for SLAM systems in complex environments.

[0036] (2) This invention innovatively introduces geomagnetic information from a nine-axis IMU to construct a sequence matching mechanism by deeply integrating visual, inertial and geomagnetic multi-sensor data built into a smartphone. This significantly improves the loop closure detection accuracy and positioning stability of the visual-inertial SLAM system in complex indoor environments. It effectively realizes that the visual-inertial SLAM loop closure detection method and system can solve the problem of false detection and missed detection that are easy to occur in low light and low texture scenes by traditional visual bag-of-words model with the assistance of geomagnetic sequence matching. Moreover, by dynamically adjusting the key frame buffer strategy and the adaptive threshold matching algorithm, the success rate of loop closure detection can be increased to more than 90%. Furthermore, by utilizing the spatial specificity and temporal stability of the geomagnetic field to construct a real-time geomagnetic feature database, low-cost deployment without prior equipment can be achieved. This not only significantly reduces the false detection and missed detection rates, but also improves the overall accuracy and robustness of the SLAM system in complex environments. Attached Figure Description

[0037] Figure 1This is an overall flowchart of a visual-inertial SLAM loop closure detection method based on geomagnetic information optimization according to the present invention;

[0038] Figure 2 This is a schematic diagram illustrating the working principle of a visual-inertial SLAM loop closure detection system based on geomagnetic information optimization according to the present invention.

[0039] Figure 3 This is a schematic diagram of the accelerated geomagnetic sequence matching dual screening strategy of the present invention;

[0040] Figure 4 This is a schematic diagram of the VINS_loop and Mag_loop trajectories and their displacements in the X / Y / Z directions in an embodiment of the present invention;

[0041] Figure 5 This is a diagram of newly added loop trajectories detected by the Mag_Loop algorithm in Rviz in an embodiment of the present invention;

[0042] Figure 6 This is a comparison diagram of the VECtor Benchmark units-scooter sequence algorithm trajectories in an embodiment of the present invention;

[0043] Figure 7 This is a schematic diagram of the EVO_APE detection results of the VECtor Benchmark units-scooter sequence in an embodiment of the present invention;

[0044] Figure 8 This is a comparison diagram of the underground space sequence algorithm trajectory in an embodiment of the present invention;

[0045] Figure 9 This is a schematic diagram of the underground space sequence evo_ape detection results in an embodiment of the present invention. Detailed Implementation

[0046] The present invention will now be further described with reference to the accompanying drawings.

[0047] like Figure 1-2 As shown, the present invention provides a visual-inertial SLAM loop closure detection method based on geomagnetic information optimization, comprising the following steps:

[0048] like Figure 3 As shown, step A involves solving a coarse match on the collected data at a low resolution level to obtain the first geomagnetic sequence matching result. The specific steps are as follows.

[0049] In this approach, by first solving the coarse matching at a low-resolution level and then refining and optimizing it in the local neighborhood, the global calculation of the entire distance matrix is ​​avoided, thereby significantly reducing the time and space complexity of the algorithm and ensuring efficient computation when matching time series of different lengths and speeds.

[0050] Step A1 involves downsampling the original time series data and then averaging adjacent sampling points to generate a low-resolution version. The downsampling process halves the sequence length while preserving the overall shape and trend of the original sequence.

[0051] Step A2 involves using the DTW algorithm at the lowest resolution level to calculate the initial coarse matching path and obtain the first geomagnetic sequence matching result. The specific steps are as follows:

[0052] Step A21: Calculate the Euclidean distance between each point in the two sequences and construct the distance matrix, as shown in formula (1).

[0053] D i，j =d(i,j)=|p i -q j | (1)

[0054] Among them, D i,j The cumulative Euclidean distance is given, where i and j are indices, d(i, j) is the local cost function, and p... i For the database sequence P(p1, p2, ..., p...), ... i , ..., p m The element in ) , q i For querying the sequence Q(q1, q2, ..., q... i , ..., q n Elements in )

[0055] Step A22 involves recursively calculating the cumulative minimum distance path using dynamic programming. Then, the similarity between the two sequences is measured by calculating the sum of the Euclidean distances of all points on the matching path. Specifically, a path with the minimum Euclidean distance is found between the starting point (1, 1) and the point (m, n), satisfying boundary constraints, continuity constraints, and monotonic constraints. The boundary constraints specifically state that the starting point of the alignment path is the lower left corner (1, 1) of the cumulative matrix, and the ending point is the upper right corner (m, n) of the cumulative matrix. The continuity constraints specifically state that the points on the alignment path are continuous, and the indices of the two sequences on the time axis can only gradually increase and can only move one step at a time. The monotonic constraints specifically state that the alignment path is monotonically non-decreasing, and the indices of the two sequences cannot regress, as shown in formula (2).

[0056]

[0057] Among them, Di-1,j D i,j-1 and D i-1,j-1 Let be the cumulative Euclidean distances between points (i-1, j), (i, j-1), and (i-1, j-1), respectively.

[0058] like Figure 4 As shown, in step B, the first geomagnetic sequence matching result is optimized within a local neighborhood to obtain the second geomagnetic sequence matching result. The specific steps are as follows.

[0059] Step B1: Project the first geomagnetic sequence matching result obtained at low resolution onto a higher resolution sequence, and then map and extend each low resolution point to a local neighborhood within a preset radius, thereby limiting the search area for high resolution matching.

[0060] Step B2: In the local neighborhood within the preset radius, the DTW algorithm with spatial constraints is used to refine the initial alignment path locally and obtain the second geomagnetic sequence matching result. The second geomagnetic sequence matching result is then used as the initial path for a higher resolution level.

[0061] like Figure 5 As shown, in step C, based on the second geomagnetic sequence matching result, keyframe images are introduced and the geomagnetic intensity modulus value of the corresponding timestamp of the keyframe image is calculated to obtain a keyframe image containing the geomagnetic intensity modulus value. The specific steps are as follows.

[0062] Step C1: Introduce keyframe images. The selection criteria for keyframe images include motion changes, parallax changes, and time intervals. The keyframe images are used to reduce computational redundancy, ensure real-time balance, ensure accuracy balance, and enhance stability in violent motion or harsh environments.

[0063] Step C2: Calculate the geomagnetic intensity modulus of the keyframe image corresponding to the timestamp. Specifically, let the geomagnetic field at a certain point indoors be represented by the components of the three-axis coordinate system (m). x m y m z If the geomagnetic intensity modulus M is as shown in formula (3), then

[0064]

[0065] like Figure 6 As shown, in step D, the keyframe image containing the geomagnetic intensity modulus value is stored in the buffer and the current buffer visual geomagnetic sequence is formed. Specifically, the keyframe image containing the geomagnetic intensity modulus value is stored at the midpoint of the buffer and the information of the preceding and following frames is accumulated. If the accumulated information of the preceding and following frames meets the set buffer capacity threshold, the current buffer visual geomagnetic sequence is formed. The buffer capacity threshold is dynamically adjusted according to the magnitude of the carrier parallax change and whether the geomagnetic environment is stable.

[0066] Specifically, when the parallax of the carrier changes significantly or the geomagnetic environment is unstable, a larger buffer window (n=25, corresponding to a buffer length of 51) is set to better capture dynamic changes; when the parallax change is moderate, a medium buffer window (n=12, corresponding to a buffer length of 25) is set; and when the parallax change is small and the geomagnetic environment is stable, a smaller buffer window (n=5, corresponding to a buffer length of 11) is used.

[0067] like Figure 7 As shown, in step E, the distance matrix between the current buffer visual geomagnetic sequence and the historical buffer visual geomagnetic sequence is calculated, and then the optimal matching path and the final matching distance are obtained by calculating the cumulative distance matrix through dynamic programming.

[0068] like Figure 8 As shown, in step F, based on the obtained optimal matching path and final matching distance, the region to be used for visual feature matching by the visual bag-of-words model is selected from the sequence with the closest distance. Then, the visual bag-of-words model is used to perform visual feature matching on the selected region and to determine whether there is a loop, thus completing the visual inertial SLAM loop detection task. Specifically, the visual bag-of-words model is used to perform visual feature matching on the selected region, as shown in formula (4).

[0069] T new =T origin ×(1-β·(1-dis Norm ) k (4)

[0070] Among them, T new T is the threshold for the new bag-of-words model. origin The threshold of the original bag-of-words model is β, where β is the weighting factor and dis is the weighting factor. Norm This represents the normalized form of the geomagnetic sequence matching results, where k is a nonlinear factor.

[0071] In the matching stage of the visual bag-of-words model, appropriately lowering the matching threshold can compensate for missed detections in loop closure detection, thus better corresponding to the matching results of the geomagnetic sequence. However, an excessively low threshold will introduce more false detections in a local range, thereby affecting the accuracy of loop closure detection. For candidate frames that are closer in the geomagnetic sequence, higher weights can be assigned, allowing the bag-of-words model to have a lower minimum matching threshold, while for candidate frames that are slightly farther away, a higher minimum matching threshold is required.

[0072] like Figure 9As shown, a visual-inertial SLAM loop closure detection system based on geomagnetic information optimization includes a coarse matching module, a local optimization module, a keyframe introduction module, a visual geomagnetic sequence acquisition module, a cumulative distance matrix calculation module, and a loop closure judgment module. The coarse matching module solves the coarse matching problem on the acquired data at a low resolution level to obtain a first geomagnetic sequence matching result. The local optimization module optimizes the first geomagnetic sequence matching result within a local neighborhood to obtain a second geomagnetic sequence matching result. The keyframe introduction module introduces keyframe images based on the second geomagnetic sequence matching result and calculates the geomagnetic intensity modulus value of the corresponding timestamp of the keyframe image, thereby obtaining a keyframe image containing the geomagnetic intensity modulus value. The visual geomagnetic sequence acquisition module is used to store keyframe images containing geomagnetic intensity modulus values ​​into a buffer and form the current buffer visual geomagnetic sequence; the cumulative distance matrix calculation module is used to calculate the distance matrix between the current buffer visual geomagnetic sequence and the historical buffer visual geomagnetic sequence, and then obtain the optimal matching path and the final matching distance by calculating the cumulative distance matrix through dynamic programming; the loop closure judgment module is used to select the region as the visual bag-of-words model for visual feature matching in the sequence with the closest distance according to the obtained optimal matching path and the final matching distance, and then use the visual bag-of-words model to perform visual feature matching on the selected region and judge whether there is a loop, thereby completing the visual inertial SLAM loop closure detection operation.

[0073] In summary, the visual inertial SLAM loop closure detection method and system based on geomagnetic information optimization of the present invention first solves the coarse matching at a low resolution level to obtain a first geomagnetic sequence matching result. Then, the first geomagnetic sequence matching result is optimized within a local neighborhood to obtain a second geomagnetic sequence matching result. Next, keyframe images are introduced based on the second geomagnetic sequence matching result, and the geomagnetic intensity magnitude value of the corresponding timestamp of the keyframe image is calculated, thereby obtaining keyframe images containing geomagnetic intensity magnitude values. These keyframe images containing geomagnetic intensity magnitude values ​​are then stored in a buffer to form the current buffer visual geomagnetic sequence. Subsequently, the distance matrix between the current buffer visual geomagnetic sequence and the historical buffer visual geomagnetic sequences is calculated. Then, the cumulative distance matrix is ​​calculated through dynamic programming to obtain the optimal matching path and the final matching distance. Finally, based on the obtained optimal matching path and the final matching distance, the sequence with the closest distance is selected as the region for visual feature matching using the visual bag-of-words model. The system first selects a region and then uses a visual bag-of-words model to perform visual feature matching on the selected region and determine whether loops exist, thus completing the visual inertial SLAM loop detection task. This effectively realizes that the visual inertial SLAM loop detection method and system has the function of first performing coarse screening and then fine matching in loop detection by using an efficient geomagnetic sequence matching algorithm and deep fusion of visual keyframes. In addition, the Fast-EDLines algorithm can accelerate gradient calculation, edge detection, line segment extraction and verification. It not only effectively integrates visual information and inertial data, but also ensures the accuracy and high confidence of line feature extraction. By using the FastDTW algorithm and geomagnetic sequence matching and dynamic adjustment of keyframe buffers, efficient and robust loop detection can be achieved. This invention makes full use of the geometric information of line features in the image, and combines geomagnetic data to improve positioning accuracy. It also takes into account real-time performance and stability, and can provide solid and reliable technical support for SLAM systems in complex environments.

[0074] Furthermore, this invention innovatively incorporates geomagnetic information from a nine-axis IMU to construct a sequence matching mechanism by deeply integrating visual, inertial, and geomagnetic multi-sensor data built into a smartphone. This significantly improves the loop closure detection accuracy and positioning stability of the visual-inertial SLAM system in complex indoor environments. It effectively enables the visual-inertial SLAM loop closure detection method and system to address the false positives and false negatives that are common in traditional visual bag-of-words models in low-light and low-texture scenes, aided by geomagnetic sequence matching. By dynamically adjusting the keyframe buffer strategy and using an adaptive threshold matching algorithm, the loop closure detection success rate can be increased to over 90%. Moreover, by utilizing the spatial specificity and temporal stability of the geomagnetic field to construct a real-time geomagnetic feature database, low-cost deployment without prior equipment is achieved. This not only significantly reduces false positives and false negatives but also improves the overall accuracy and robustness of the SLAM system in complex environments.

[0075] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A visual-inertial SLAM loop closure detection method based on geomagnetic information optimization, characterized in that: Includes the following steps, Step A: Solve the coarse matching at a low resolution level for the collected data to obtain the first geomagnetic sequence matching result; Step B: Optimize the first geomagnetic sequence matching result within a local neighborhood to obtain the second geomagnetic sequence matching result; Step C: Based on the second geomagnetic sequence matching result, keyframe images are introduced and the geomagnetic intensity modulus value of the corresponding timestamp of the keyframe image is calculated to obtain keyframe images containing geomagnetic intensity modulus values; Step D: Store the keyframe image containing the geomagnetic intensity modulus value into the buffer and form the current buffer visual geomagnetic sequence. Specifically, store the keyframe image containing the geomagnetic intensity modulus value at the midpoint of the buffer and accumulate the information of the preceding and following frames. If the accumulated information of the preceding and following frames meets the set buffer capacity threshold, the current buffer visual geomagnetic sequence is formed. The buffer capacity threshold is dynamically adjusted according to the magnitude of the carrier parallax change and whether the geomagnetic environment is stable. Step E: Calculate the distance matrix between the current buffer visual geomagnetic sequence and the historical buffer visual geomagnetic sequence, and then use dynamic programming to calculate the cumulative distance matrix to obtain the optimal matching path and the final matching distance; Step F: Based on the obtained optimal matching path and final matching distance, select the region from the closest sequence as the visual bag-of-words model for visual feature matching. Then, use the visual bag-of-words model to perform visual feature matching on the selected region and determine whether there is a loop, thus completing the visual inertial SLAM loop detection task. Specifically, the visual bag-of-words model is used to perform visual feature matching on the selected region, as shown in formula (4). （4） in, This is the threshold for the new bag-of-words model. This is the threshold of the original bag-of-words model. As the weighting factor, This is the normalized form of the geomagnetic sequence matching results. This is a non-linear factor.

2. The visual-inertial SLAM loop closure detection method based on geomagnetic information optimization according to claim 1, characterized in that: Step A involves performing a coarse matching on the collected data at a low resolution level to obtain the first geomagnetic sequence matching result. The specific steps are as follows. Step A1 involves downsampling the original time series data and then averaging adjacent sampling points to generate a low-resolution version. The downsampling process halves the sequence length while preserving the overall shape and trend of the original sequence. Step A2 involves using the DTW algorithm at the lowest resolution level to calculate the initial coarse matching path and obtain the first geomagnetic sequence matching result. The specific steps are as follows: Step A21: Calculate the Euclidean distance between each point in the two sequences and construct the distance matrix, as shown in formula (1). （1） in, To accumulate the Euclidean distance, i and j are indices. For local cost function, For database sequences The elements in For query sequence Elements in; Step A22 involves recursively calculating the cumulative minimum distance path using dynamic programming, and then measuring the similarity between the two sequences by calculating the sum of the Euclidean distances of all points on the matching path. Specifically, this starts from the starting point. Location from Find a path between positions that minimizes the sum of Euclidean distances while simultaneously satisfying boundary constraints, continuity constraints, and monotonicity constraints. Specifically, the boundary constraint requires the starting point of the aligned path to be the lower left corner of the cumulative matrix. And the endpoint is the upper right corner of the cumulative matrix. The continuity constraint specifically means that the points on the alignment path are continuous and the indices of the two sequences on the time axis can only increase incrementally and can only move one step at a time. The monotonic constraint specifically means that the alignment path is monotonically non-decreasing and the indices of the two sequences cannot go back, as shown in formula (2). （2） in, , and Points , and The cumulative Euclidean distance.

3. The visual-inertial SLAM loop closure detection method based on geomagnetic information optimization according to claim 2, characterized in that: Step B involves optimizing the first geomagnetic sequence matching result within a local neighborhood to obtain the second geomagnetic sequence matching result. The specific steps are as follows. Step B1: Project the first geomagnetic sequence matching result obtained at low resolution onto a higher resolution sequence, and then map and extend each low resolution point to a local neighborhood within a preset radius, thereby limiting the search area for high resolution matching. Step B2: In the local neighborhood within the preset radius, the DTW algorithm with spatial constraints is used to refine the initial alignment path locally and obtain the second geomagnetic sequence matching result. The second geomagnetic sequence matching result is then used as the initial path for a higher resolution level.

4. The visual-inertial SLAM loop closure detection method based on geomagnetic information optimization according to claim 3, characterized in that: Step C: Based on the second geomagnetic sequence matching result, keyframe images are introduced and the geomagnetic intensity modulus value of the corresponding timestamp of the keyframe image is calculated to obtain keyframe images containing geomagnetic intensity modulus values. The specific steps are as follows. Step C1: Introduce keyframe images. The selection criteria for keyframe images include motion changes, parallax changes, and time intervals. The keyframe images are used to reduce computational redundancy, ensure real-time balance, ensure accuracy balance, and enhance stability in violent motion or harsh environments. Step C2: Calculate the geomagnetic intensity modulus of the keyframe image corresponding to the timestamp. Specifically, let the geomagnetic field at a certain point indoors be represented by the components of the three-axis coordinate system. Then the geomagnetic intensity modulus As shown in formula (3), （3）。 5. A visual-inertial SLAM loop closure detection system optimized based on geomagnetic information, wherein the visual-inertial SLAM loop closure detection process is based on the visual-inertial SLAM loop closure detection method according to any one of claims 1-4, characterized in that: It includes a coarse matching module, a local optimization module, a keyframe introduction module, a visual geomagnetic sequence acquisition module, a cumulative distance matrix calculation module, and a loop closure detection module. The coarse matching module is used to solve the coarse matching of the collected data at a low resolution level and obtain the first geomagnetic sequence matching result. The local optimization module is used to optimize the first geomagnetic sequence matching result in a local neighborhood and obtain the second geomagnetic sequence matching result. The keyframe introduction module is used to introduce keyframe images based on the second geomagnetic sequence matching results and calculate the geomagnetic intensity modulus value of the timestamp corresponding to the keyframe image, thereby obtaining a keyframe image containing the geomagnetic intensity modulus value. The visual geomagnetic sequence acquisition module is used to store keyframe images containing geomagnetic intensity modulus values ​​into a buffer and form a visual geomagnetic sequence in the current buffer. The cumulative distance matrix calculation module is used to calculate the distance matrix between the current buffer visual geomagnetic sequence and the historical buffer visual geomagnetic sequence, and then obtain the optimal matching path and the final matching distance by calculating the cumulative distance matrix through dynamic programming. The loop closure detection module is used to select the region for visual feature matching by the visual bag-of-words model from the closest sequence based on the obtained optimal matching path and final matching distance. Then, the visual bag-of-words model is used to perform visual feature matching on the selected region and determine whether there is a loop, thereby completing the visual inertial SLAM loop closure detection task.

Images