Visual inertia SLAM loopback detection method and system based on geomagnetic information optimization

Abstract

The invention discloses a visual inertia SLAM loopback detection method and system based on geomagnetic information optimization, and the method comprises the steps: carrying out the rough matching of collected data at a low-resolution level, obtaining a first geomagnetic sequence matching result, carrying out the optimization of the first geomagnetic sequence matching result in a local neighborhood, and obtaining a second geomagnetic sequence matching result; introducing a key frame image based on a second geomagnetic sequence matching result, and calculating a geomagnetic intensity module value of a timestamp corresponding to the key frame image; according to the method, the function of firstly performing coarse screening and then performing fine matching in loopback detection by utilizing an efficient geomagnetic sequence matching algorithm and a visual key frame deep fusion mode is realized, gradient calculation acceleration, edge detection and line segment extraction and verification can be performed through a Fast-EDLines algorithm, the accuracy and high confidence of line feature extraction are ensured, and the method is suitable for large-scale popularization and application. Efficient and robust loopback detection can be carried out by adopting a FastDTW algorithm, geomagnetic sequence matching and dynamic adjustment of a key frame buffer area, and the method is suitable for being widely popularized and used.

Description

Technical Field

[0001] The present invention relates to the technical field of SLAM loop detection, and in particular to a visual-inertial SLAM loop detection method and system based on geomagnetic information optimization. Background Art

[0002] Visual SLAM is a key technology that empowers machines with visual perception and spatial cognition. By analyzing continuous image streams, calculating their own motion in real time and building environmental maps, it provides fundamental spatial understanding and positioning capabilities for a variety of autonomous systems and interactive applications. Despite numerous challenges, thanks to algorithmic innovations, advancements in computing hardware, and advances in multi-sensor fusion, visual SLAM is becoming increasingly robust, accurate, and practical, and its application scope is expanding.

[0003] At present, visual SLAM technology still faces many challenges in practical applications. Traditional methods are often prone to false detection and missed detection due to reliance on single visual information in environments such as low light, texture loss, and unstable handheld devices, especially in the loop detection stage. Existing methods generally rely on image matching and spatiotemporal consistency verification, which makes it difficult to ensure detection accuracy when the lighting changes drastically or the scene repetition is high. In addition, the traditional method of pose estimation that relies on point features is easily interfered with in low-texture or repetitive-texture environments, thereby reducing the front-end tracking performance. Therefore, it is necessary to design a visual-inertial SLAM loop detection method and system based on geomagnetic information optimization. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and to better and effectively solve the problems of current visual SLAM in environments such as low light, texture loss and unstable handheld devices, especially in the loop detection stage, which is often prone to false detection and missed detection due to reliance on single visual information. Existing methods generally rely on image matching and spatiotemporal consistency verification, which makes it difficult to ensure the accuracy of detection when the lighting changes drastically or the scene repetition is high. In addition, the traditional method of pose estimation that relies on point features is easily disturbed in low-texture or repetitive-texture environments, thereby reducing the front-end tracking performance. A visual-inertial SLAM loop detection method and system based on geomagnetic information optimization is provided, which realizes the function of first coarse screening and then fine matching in loop detection by using an efficient geomagnetic sequence matching algorithm and a deep fusion of visual key frames, and can perform gradient calculation acceleration, edge detection, line segment extraction and verification through the Fast-EDLines algorithm, ensuring the accuracy and high confidence of line feature extraction, and by adopting the FastDTW algorithm and geomagnetic sequence matching and dynamic adjustment of the key frame buffer, efficient and robust loop detection can be performed.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A visual inertial SLAM loop detection method based on geomagnetic information optimization includes the following steps:

[0007] Step A, performing a coarse matching on the collected data at a low resolution level to obtain a first geomagnetic sequence matching result;

[0008] Step B, optimizing the first geomagnetic sequence matching result in a local neighborhood to obtain a second geomagnetic sequence matching result;

[0009] Step C, based on the second geomagnetic sequence matching result, introducing a key frame image and calculating the geomagnetic intensity modulus value corresponding to the time stamp of the key frame image to obtain a key frame image containing the geomagnetic intensity modulus value;

[0010] Step D, storing the key frame images containing the geomagnetic intensity modulus value into a buffer and forming a visual geomagnetic sequence in the current buffer;

[0011] Step E: Calculate the distance matrix between the visual geomagnetic sequence of the current buffer and the visual geomagnetic sequence of the historical buffer, and then calculate the cumulative distance matrix through dynamic programming to obtain the optimal matching path and the final matching distance;

[0012] In step F, based on the obtained optimal matching path and the final matching distance, the area in the closest distance sequence is selected as the visual feature matching area using the visual word bag model. The visual word bag model is then used to match the visual features of the selected area and determine whether there is a loop, thus completing the visual inertial SLAM loop detection operation.

[0013] The aforementioned visual inertial SLAM loop detection method based on geomagnetic information optimization, step A, solves the coarse matching of the collected data at a low resolution level to obtain the first geomagnetic sequence matching result. The specific steps are as follows:

[0014] Step A1: downsample the original time series in the collected data and then average adjacent sampling points to generate a low-resolution version, wherein the downsampling process is to reduce the sequence length by half while preserving the overall shape and trend of the original sequence;

[0015] Step A2: Calculate the initial rough matching path using the DTW algorithm at the lowest resolution level and obtain the first geomagnetic sequence matching result. The specific steps are as follows:

[0016] Step A21, calculate the Euclidean distance between each point in the two sequences point by point and construct a distance matrix, as shown in formula (1),

[0017] D i，j =d(i, j)=|p i -q j | (1)

[0018] Among them, D i,j is the cumulative Euclidean distance, i and j are indexes, d(i, j) is the local cost function, p i For the database sequence P(p1, p2, ..., p i ,...,p m ), q i For the query sequence Q(q1,q2,...,q j ,...,q n ) in the element;

[0019] In step A22, dynamic programming is used to recursively calculate the cumulative minimum distance path, and then the similarity of the two sequences is measured by calculating the cumulative sum of the Euclidean distances of all points on the matching path. Specifically, a path with the minimum Euclidean distance is found from the starting point (1, 1) to the position (m, n) and simultaneously satisfies the boundary constraint, continuity constraint and monotonic constraint. The boundary constraint is that the starting point of the alignment path is the lower left corner (1, 1) of the accumulation matrix and the end point is the upper right corner (m, n) of the accumulation matrix. The continuity constraint is that the points on the alignment path are continuous and the indexes of the two sequences on the time axis can only be gradually increased and can only move one step at a time. The monotonic constraint is that the alignment path is monotonically non-decreasing and the indexes of the two sequences cannot fall back, as shown in formula (2).

[0020]

[0021] Among them, D i-1,j 、D i,j-1 and D i-1,j-1 are the cumulative Euclidean distances of points (i-1, j), (i, j-1), and (i-1, j-1) respectively.

[0022] The aforementioned visual inertial SLAM loop detection method based on geomagnetic information optimization, step B, optimizes the first geomagnetic sequence matching result in a local neighborhood to obtain the second geomagnetic sequence matching result, the specific steps are as follows:

[0023] Step B1: Projecting the matching results of the first geomagnetic sequence obtained at low resolution into a higher resolution sequence, and then expanding each low-resolution point mapping to a local neighborhood within a preset radius, thereby limiting the search area for high-resolution matching;

[0024] In step B2, the preliminary alignment path is locally refined using a DTW algorithm with spatial constraints in a local neighborhood within a preset radius to obtain a second geomagnetic sequence matching result, which is then used as the initial path at a higher resolution level.

[0025] The aforementioned visual inertial SLAM loop detection method based on geomagnetic information optimization, step C, based on the second geomagnetic sequence matching result, introduces the key frame image and calculates the geomagnetic intensity modulus value of the key frame image corresponding to the time stamp, to obtain the key frame image containing the geomagnetic intensity modulus value, the specific steps are as follows,

[0026] Step C1: introducing keyframe images. The keyframe images are selected based on criteria including motion change, parallax change, and time interval. The keyframe images are used to reduce computational redundancy, ensure a balance between real-time performance and accuracy, and enhance stability during intense exercise or in harsh environments.

[0027] Step C2, calculate the geomagnetic intensity modulus of the key frame image corresponding to the timestamp, specifically assuming that the geomagnetic field at a certain point in the room is represented by the components of the three coordinate axes (m x , m y , m z ), then the geomagnetic intensity modulus M is as shown in formula (3),

[0028]

[0029] The aforementioned visual-inertial SLAM loop detection method based on geomagnetic information optimization, step D, stores the key frame image containing the geomagnetic intensity modulus value into the buffer and constitutes the visual geomagnetic sequence of the current buffer. Specifically, the key frame image containing the geomagnetic intensity modulus value is stored at the midpoint position of the buffer and the previous and next frame information is accumulated. If the accumulated previous and next frame information meets the set buffer capacity threshold, it constitutes the current buffer visual geomagnetic sequence, wherein the buffer capacity threshold is dynamically adjusted according to the size of the carrier parallax change and whether the geomagnetic environment is stable.

[0030] In the aforementioned visual-inertial SLAM loop detection method based on geomagnetic information optimization, step F selects an area in the closest distance sequence as a visual feature matching area using the visual word bag model according to the obtained optimal matching path and the final matching distance, and then uses the visual word bag model to perform visual feature matching on the selected area and determine whether a loop exists, thereby completing the visual-inertial SLAM loop detection operation. Specifically, the visual word bag model is used to perform visual feature matching on the selected area, as shown in formula (4).

[0031] T new =T origin ×(1-β·(1-dis Norm ) k ) (4)

[0032] Among them, T new is the new word bag model threshold, T origin is the original word bag model threshold, β is the weight factor, dis Normis the normalized form of the geomagnetic sequence matching result, and k is the nonlinear factor.

[0033] A visual inertial SLAM loop detection system based on geomagnetic information optimization includes a coarse matching module, a local optimization module, a key frame introduction module, a visual geomagnetic sequence acquisition module, a cumulative distance matrix calculation module and a loop judgment module, wherein the coarse matching module is used to solve the coarse matching of the collected data at a low resolution level and obtain a first geomagnetic sequence matching result; the local optimization module is used to optimize the first geomagnetic sequence matching result in a local neighborhood and obtain a second geomagnetic sequence matching result; the key frame introduction module is used to introduce a key frame image based on the second geomagnetic sequence matching result and calculate the geomagnetic intensity modulus of the key frame image corresponding to the time stamp, thereby obtaining a key frame image containing the geomagnetic intensity modulus; The visual geomagnetic sequence acquisition module is used to store key frame images containing geomagnetic intensity modulus values ​​into a buffer and form a visual geomagnetic sequence in the current buffer; the cumulative distance matrix calculation module is used to calculate the distance matrix between the visual geomagnetic sequence in the current buffer and the visual geomagnetic sequence in the historical buffer, and then calculate the cumulative distance matrix through dynamic programming to obtain the optimal matching path and the final matching distance; the loop judgment module is used to select an area in the sequence with the closest distance as the visual word bag model for visual feature matching, and then use the visual word bag model to perform visual feature matching on the selected area and determine whether there is a loop, thereby completing the visual inertial SLAM loop detection operation.

[0034] The beneficial effects of the present invention are:

[0035] (1) The present invention first solves the coarse matching of the collected data at a low-resolution level and obtains the first geomagnetic sequence matching result, then optimizes the first geomagnetic sequence matching result in a local neighborhood and obtains the second geomagnetic sequence matching result, then introduces the key frame image based on the second geomagnetic sequence matching result and calculates the geomagnetic intensity modulus of the key frame image corresponding to the time stamp, thereby obtaining the key frame image containing the geomagnetic intensity modulus, then stores the key frame image containing the geomagnetic intensity modulus into the buffer and forms the current buffer visual geomagnetic sequence, then calculates the distance matrix between the current buffer visual geomagnetic sequence and the historical buffer visual geomagnetic sequence, then calculates the cumulative distance matrix through dynamic programming to obtain the optimal matching path and the final matching distance, then selects the area in the sequence with the closest distance as the visual feature matching of the visual word bag model according to the obtained optimal matching path and the final matching distance, and then uses the visual word bag model to select the selected area The visual-inertial SLAM loop detection method and system are effectively realized, which has the function of performing coarse screening and then fine matching in loop detection by using an efficient geomagnetic sequence matching algorithm and deep fusion of visual key frames, and can perform gradient calculation acceleration, edge detection, line segment extraction and verification through the Fast-EDLines algorithm, which not only effectively integrates visual information and inertial data, but also ensures the accuracy and high confidence of line feature extraction, and can perform efficient and robust loop detection by adopting the FastDTW algorithm, geomagnetic sequence matching and dynamic adjustment of the key frame buffer. The present invention makes full use of the geometric information of line features in the image, combines geomagnetic data to improve positioning accuracy, and takes into account real-time and stability, and can provide solid and reliable technical support for SLAM systems in complex environments.

[0036] (2) The present invention deeply integrates the visual, inertial and geomagnetic multi-sensor data built into the smartphone, innovatively introduces the geomagnetic information in the nine-axis IMU to construct a sequence matching mechanism, and significantly improves the loop detection accuracy and positioning stability of the visual inertial SLAM system in complex indoor environments; effectively realizes that the visual inertial SLAM loop detection method and system have the function of solving the problem that the traditional visual word bag model is prone to false detection and missed detection in dark light and low texture scenes with the assistance of geomagnetic sequence matching, and the loop detection success rate can be increased to more than 90% by dynamically adjusting the key frame buffer strategy and the adaptive threshold matching algorithm, and by utilizing the spatial specificity and temporal stability of the geomagnetic field to construct a real-time geomagnetic feature database, low-cost deployment without prior equipment can be achieved, which not only significantly reduces the false detection and missed detection rates, but also improves the overall accuracy and robustness of the SLAM system in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1This is an overall flow chart of a visual-inertial SLAM loop detection method based on geomagnetic information optimization of the present invention;

[0038] Figure 2 This is a schematic diagram of the working principle of a visual-inertial SLAM loop detection system based on geomagnetic information optimization of the present invention;

[0039] Figure 3 Schematic diagram of the accelerated geomagnetic sequence matching double screening strategy of the present invention;

[0040] Figure 4 Schematic diagram of VINS_loop and Mag_loop trajectories and X / Y / Z direction displacements in an embodiment of the present invention;

[0041] Figure 5 It is a newly added loop trajectory diagram detected by the Mag_Loop algorithm in Rviz in an embodiment of the present invention;

[0042] Figure 6 This is a comparison chart of the VECtor Benchmark units-scooter sequence algorithm trajectories in an embodiment of the present invention;

[0043] Figure 7 2 is a schematic diagram of the VECtor Benchmark units-scooter sequence evo_ape detection results in an embodiment of the present invention;

[0044] Figure 8 is a comparison diagram of underground space sequence algorithm trajectories in an embodiment of the present invention;

[0045] Figure 9 3. It is a schematic diagram of the detection results of the underground space sequence evo_ape in an embodiment of the present invention;. DETAILED DESCRIPTION

[0046] The present invention will be further described below with reference to the accompanying drawings.

[0047] like Figure 1-2 As shown, the visual inertial SLAM loop detection method based on geomagnetic information optimization of the present invention includes the following steps:

[0048] like Figure 3 As shown, step A is to solve the coarse matching of the collected data at a low resolution level to obtain the first geomagnetic sequence matching result. The specific steps are as follows:

[0049] Among them, by first solving the coarse matching at a low-resolution level and then performing refinement optimization in the local neighborhood, the global calculation of the entire distance matrix is ​​avoided, thereby significantly reducing the time and space complexity of the algorithm, ensuring efficient calculation when matching time series of different lengths and speeds;

[0050] Step A1: downsample the original time series in the collected data and then average adjacent sampling points to generate a low-resolution version, wherein the downsampling process is to reduce the sequence length by half while preserving the overall shape and trend of the original sequence;

[0051] Step A2: Calculate the initial rough matching path using the DTW algorithm at the lowest resolution level and obtain the first geomagnetic sequence matching result. The specific steps are as follows:

[0052] Step A21, calculate the Euclidean distance between each point in the two sequences point by point and construct a distance matrix, as shown in formula (1),

[0053] D i，j =d(i, j)=|p i -q j | (1)

[0054] Among them, D i,j is the cumulative Euclidean distance, i and j are indexes, d(i, j) is the local cost function, p i For the database sequence P(p1, p2, ..., p i ,...,p m ), q i For the query sequence Q(q1,q2,...,q i ,...,q n ) in the element;

[0055] In step A22, dynamic programming is used to recursively calculate the cumulative minimum distance path, and then the similarity of the two sequences is measured by calculating the cumulative sum of the Euclidean distances of all points on the matching path. Specifically, a path with the minimum Euclidean distance is found from the starting point (1, 1) to the position (m, n) and simultaneously satisfies the boundary constraint, continuity constraint and monotonic constraint. The boundary constraint is that the starting point of the alignment path is the lower left corner (1, 1) of the accumulation matrix and the end point is the upper right corner (m, n) of the accumulation matrix. The continuity constraint is that the points on the alignment path are continuous and the indexes of the two sequences on the time axis can only be gradually increased and can only move one step at a time. The monotonic constraint is that the alignment path is monotonically non-decreasing and the indexes of the two sequences cannot fall back, as shown in formula (2).

[0056]

[0057] Among them, Di-1,j 、D i,j-1 and D i-1,j-1 are the cumulative Euclidean distances of points (i-1, j), (i, j-1), and (i-1, j-1) respectively.

[0058] like Figure 4 As shown, step B is to optimize the first geomagnetic sequence matching result in a local neighborhood to obtain the second geomagnetic sequence matching result. The specific steps are as follows:

[0059] Step B1: Projecting the matching results of the first geomagnetic sequence obtained at low resolution into a higher resolution sequence, and then expanding each low-resolution point mapping to a local neighborhood within a preset radius, thereby limiting the search area for high-resolution matching;

[0060] In step B2, the preliminary alignment path is locally refined using a DTW algorithm with spatial constraints in a local neighborhood within a preset radius to obtain a second geomagnetic sequence matching result, which is then used as the initial path at a higher resolution level.

[0061] like Figure 5 As shown, step C, based on the second geomagnetic sequence matching result, introduces the key frame image and calculates the geomagnetic intensity modulus value of the key frame image corresponding to the time stamp to obtain the key frame image containing the geomagnetic intensity modulus value. The specific steps are as follows:

[0062] Step C1: introducing keyframe images. The keyframe images are selected based on criteria including motion change, parallax change, and time interval. The keyframe images are used to reduce computational redundancy, ensure a balance between real-time performance and accuracy, and enhance stability during intense exercise or in harsh environments.

[0063] Step C2, calculate the geomagnetic intensity modulus of the key frame image corresponding to the timestamp, specifically assuming that the geomagnetic field at a certain point in the room is represented by the components of the three coordinate axes (m x , m y , m z ), then the geomagnetic intensity modulus M is as shown in formula (3),

[0064]

[0065] like Figure 6 As shown, step D, the key frame image containing the geomagnetic intensity modulus value is stored in the buffer and constitutes the visual geomagnetic sequence of the current buffer. Specifically, the key frame image containing the geomagnetic intensity modulus value is stored at the midpoint position of the buffer and the previous and next frame information is accumulated. If the accumulated previous and next frame information meets the set buffer capacity threshold, then the visual geomagnetic sequence of the current buffer is constituted, wherein the buffer capacity threshold is dynamically adjusted according to the size of the carrier parallax change and whether the geomagnetic environment is stable.

[0066] When the carrier parallax changes greatly or the geomagnetic environment is unstable, a larger buffer window (n=25, corresponding to a buffer length of 51) is set to better capture dynamic changes; when the parallax change is moderate, a medium buffer window (n=12, corresponding to a buffer length of 25) is set; and when the parallax change is small and the geomagnetic environment is stable, a smaller buffer window (n=5, corresponding to a buffer length of 11) is used.

[0067] like Figure 7 As shown, step E calculates the distance matrix between the visual geomagnetic sequence of the current buffer zone and the visual geomagnetic sequence of the historical buffer zone, and then calculates the cumulative distance matrix through dynamic programming to obtain the optimal matching path and the final matching distance;

[0068] like Figure 8 As shown in FIG, step F, according to the obtained optimal matching path and the final matching distance, selects the area in the closest distance sequence as the visual feature matching area of ​​the visual word bag model, and then uses the visual word bag model to match the visual features of the selected area and judge whether there is a loop, completing the visual inertial SLAM loop detection operation. Specifically, the visual word bag model is used to match the visual features of the selected area, as shown in formula (4).

[0069] T new =T origin ×(1-β·(1-dis Norm ) k ) (4)

[0070] Among them, T new is the new word bag model threshold, T origin is the original word bag model threshold, β is the weight factor, dis Norm is the normalized form of the geomagnetic sequence matching result, and k is the nonlinear factor.

[0071] Appropriately lowering the matching threshold in the bag-of-words model can compensate for missed loop detections, thereby better aligning the matching results with the geomagnetic sequence. However, an excessively low threshold can introduce more localized false detections, affecting loop detection accuracy. Candidate frames closer to the geomagnetic sequence can be assigned a higher weight, allowing the bag-of-words model to have a lower minimum match threshold, while candidate frames farther away require a higher minimum match threshold.

[0072] like Figure 9As shown, a visual-inertial SLAM loop detection system based on geomagnetic information optimization includes a coarse matching module, a local optimization module, a key frame introduction module, a visual geomagnetic sequence acquisition module, a cumulative distance matrix calculation module and a loop judgment module, wherein the coarse matching module is used to solve the coarse matching of the collected data at a low resolution level and obtain a first geomagnetic sequence matching result; the local optimization module is used to optimize the first geomagnetic sequence matching result in a local neighborhood and obtain a second geomagnetic sequence matching result; the key frame introduction module is used to introduce a key frame image based on the second geomagnetic sequence matching result and calculate the geomagnetic intensity modulus value of the key frame image corresponding to the time stamp, thereby obtaining a key frame image containing the geomagnetic intensity modulus value ; The visual geomagnetic sequence acquisition module is used to store the key frame image containing the geomagnetic intensity modulus value into the buffer and form the visual geomagnetic sequence of the current buffer; the cumulative distance matrix calculation module is used to calculate the distance matrix between the visual geomagnetic sequence of the current buffer and the visual geomagnetic sequence of the historical buffer, and then calculate the cumulative distance matrix through dynamic programming to obtain the optimal matching path and the final matching distance; the loop judgment module is used to select the area in the sequence with the closest distance as the visual word bag model for visual feature matching, and then use the visual word bag model to perform visual feature matching on the selected area and determine whether there is a loop, thereby completing the visual inertial SLAM loop detection operation.

[0073] In summary, the present invention provides a visual-inertial SLAM loop detection method and system based on geomagnetic information optimization. First, the collected data is roughly matched at a low-resolution level to obtain a first geomagnetic sequence matching result. The first geomagnetic sequence matching result is then optimized in a local neighborhood to obtain a second geomagnetic sequence matching result. Then, based on the second geomagnetic sequence matching result, a key frame image is introduced and the geomagnetic intensity modulus of the key frame image corresponding to the timestamp is calculated, thereby obtaining a key frame image containing the geomagnetic intensity modulus. The key frame image containing the geomagnetic intensity modulus is then stored in a buffer and constitutes a visual geomagnetic sequence of the current buffer. Subsequently, the distance matrix between the visual geomagnetic sequence of the current buffer and the visual geomagnetic sequence of the historical buffer is calculated. The cumulative distance matrix is ​​then calculated through dynamic programming to obtain the optimal matching path and the final matching distance. Then, according to the obtained optimal matching path and the final matching distance, the region with the closest distance is selected in the sequence for visual feature matching using the visual word bag model. Domain, and then use the visual word bag model to match the visual features of the selected area and determine whether there is a loop to complete the visual inertial SLAM loop detection operation; it effectively realizes that the visual inertial SLAM loop detection method and system have the function of first coarse screening and then fine matching in loop detection by using an efficient geomagnetic sequence matching algorithm and a deep fusion of visual key frames, and can perform gradient calculation acceleration, edge detection, line segment extraction and verification through the Fast-EDLines algorithm, which not only effectively integrates visual information and inertial data, but also ensures the accuracy and high confidence of line feature extraction, and by adopting the FastDTW algorithm and geomagnetic sequence matching and dynamic adjustment of the key frame buffer, efficient and robust loop detection can be performed. The present invention makes full use of the geometric information of line features in the image, combines geomagnetic data to improve positioning accuracy, and takes into account real-time and stability, which can provide solid and reliable technical support for SLAM systems in complex environments;

[0074] In addition, the present invention deeply integrates the visual, inertial and geomagnetic multi-sensor data built into the smartphone, and innovatively introduces the geomagnetic information in the nine-axis IMU to construct a sequence matching mechanism, which significantly improves the loop detection accuracy and positioning stability of the visual inertial SLAM system in complex indoor environments; it effectively realizes that the visual inertial SLAM loop detection method and system have the function of solving the false detection and missed detection problems of the traditional visual word bag model in dark light and low-texture scenes with the assistance of geomagnetic sequence matching, and the loop detection success rate can be increased to more than 90% by dynamically adjusting the key frame buffer strategy and the adaptive threshold matching algorithm, and by utilizing the spatial specificity and temporal stability of the geomagnetic field to construct a real-time geomagnetic feature database, low-cost deployment without prior equipment can be achieved, which not only significantly reduces the false detection and missed detection rates, but also improves the overall accuracy and robustness of the SLAM system in complex environments.

[0075] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A visual-inertial SLAM loop detection method based on geomagnetic information optimization, characterized by: The following steps are included: Step A, performing a coarse matching on the collected data at a low resolution level to obtain a first geomagnetic sequence matching result; Step B, optimizing the first geomagnetic sequence matching result in a local neighborhood to obtain a second geomagnetic sequence matching result; Step C, based on the second geomagnetic sequence matching result, introducing a key frame image and calculating the geomagnetic intensity modulus value corresponding to the time stamp of the key frame image to obtain a key frame image containing the geomagnetic intensity modulus value; Step D, storing the key frame images containing the geomagnetic intensity modulus value into a buffer and forming a visual geomagnetic sequence in the current buffer; Step E: Calculate the distance matrix between the visual geomagnetic sequence of the current buffer and the visual geomagnetic sequence of the historical buffer, and then calculate the cumulative distance matrix through dynamic programming to obtain the optimal matching path and the final matching distance; In step F, based on the obtained optimal matching path and the final matching distance, the area in the closest distance sequence is selected as the visual feature matching area using the visual word bag model. The visual word bag model is then used to match the visual features of the selected area and determine whether there is a loop, thus completing the visual inertial SLAM loop detection operation.

2. A visual-inertial SLAM loop detection method based on geomagnetic information optimization according to claim 1, characterized in that: Step A: solve the coarse matching of the collected data at a low resolution level to obtain the first geomagnetic sequence matching result. The specific steps are as follows: Step A1: downsample the original time series in the collected data and then average adjacent sampling points to generate a low-resolution version, wherein the downsampling process is to reduce the sequence length by half while preserving the overall shape and trend of the original sequence; Step A2: Calculate the initial rough matching path using the DTW algorithm at the lowest resolution level and obtain the first geomagnetic sequence matching result. The specific steps are as follows: Step A21, calculate the Euclidean distance between each point in the two sequences point by point and construct a distance matrix, as shown in formula (1), D i，j =d(i,j)=|p i -q j | (1) Among them, D i，j is the cumulative Euclidean distance, i and j are indexes, d(i, j) is the local cost function, p i For the database sequence P(p1, p2, ..., p i ,...,p m ), q i For the query sequence Q(q1,q2,...,q j ,...,q n ) in the element; In step A22, dynamic programming is used to recursively calculate the cumulative minimum distance path, and then the similarity of the two sequences is measured by calculating the cumulative sum of the Euclidean distances of all points on the matching path. Specifically, a path with the minimum Euclidean distance is found from the starting point (1, 1) to the position (m, n) and simultaneously satisfies the boundary constraint, continuity constraint and monotonic constraint. The boundary constraint is that the starting point of the alignment path is the lower left corner (1, 1) of the accumulation matrix and the end point is the upper right corner (m, n) of the accumulation matrix. The continuity constraint is that the points on the alignment path are continuous and the indexes of the two sequences on the time axis can only be gradually increased and can only move one step at a time. The monotonic constraint is that the alignment path is monotonically non-decreasing and the indexes of the two sequences cannot fall back, as shown in formula (2). Among them, D i-1，j 、D i，j-1 and D i-1，j-1 are the cumulative Euclidean distances of points (i-1, j), (i, j-1), and (i-1, j-1) respectively.

3. A visual-inertial SLAM loop detection method based on geomagnetic information optimization according to claim 2, characterized in that: Step B, optimizing the first geomagnetic sequence matching result in a local neighborhood to obtain the second geomagnetic sequence matching result, the specific steps are as follows: Step B1: Projecting the matching results of the first geomagnetic sequence obtained at low resolution into a higher resolution sequence, and then expanding each low-resolution point mapping to a local neighborhood within a preset radius, thereby limiting the search area for high-resolution matching; In step B2, the preliminary alignment path is locally refined using a DTW algorithm with spatial constraints in a local neighborhood within a preset radius to obtain a second geomagnetic sequence matching result, which is then used as the initial path at a higher resolution level.

4. The visual-inertial SLAM loop detection method based on geomagnetic information optimization according to claim 3, wherein: Step C, based on the second geomagnetic sequence matching result, introduce the key frame image and calculate the geomagnetic intensity modulus value of the key frame image corresponding to the time stamp, to obtain the key frame image containing the geomagnetic intensity modulus value. The specific steps are as follows: Step C1: introducing keyframe images. The keyframe images are selected based on criteria including motion change, parallax change, and time interval. The keyframe images are used to reduce computational redundancy, ensure a balance between real-time performance and accuracy, and enhance stability during intense exercise or in harsh environments. Step C2, calculate the geomagnetic intensity modulus of the key frame image corresponding to the timestamp. Specifically, assume that the geomagnetic field of a certain point in the room is represented by the components of the three directional coordinate axes as (), then the geomagnetic intensity modulus is as shown in formula (3), 5. A visual-inertial SLAM loop detection method based on geomagnetic information optimization according to claim 4, characterized in that: Step D, storing the key frame image containing the geomagnetic intensity modulus value into the buffer and forming a visual geomagnetic sequence of the current buffer. Specifically, the key frame image containing the geomagnetic intensity modulus value is stored at the midpoint of the buffer and the previous and next frame information is accumulated. If the accumulated previous and next frame information meets the set buffer capacity threshold, it constitutes the visual geomagnetic sequence of the current buffer, wherein the buffer capacity threshold is dynamically adjusted according to the size of the carrier parallax change and whether the geomagnetic environment is stable.

6. The visual-inertial SLAM loop detection method based on geomagnetic information optimization according to claim 5, characterized in that: In step F, based on the obtained optimal matching path and the final matching distance, the area in the closest distance sequence is selected as the visual feature matching area using the visual word bag model. The visual word bag model is then used to match the visual features of the selected area and determine whether there is a loop, thus completing the visual inertial SLAM loop detection operation. Specifically, the visual word bag model is used to match the visual features of the selected area, as shown in formula (4). T new =T origin ×(1-β·(1-dis Norm ) k ) (4) Among them, T new is the new word bag model threshold, T origin is the original word bag model threshold, β is the weight factor, dis Norm is the normalized form of the geomagnetic sequence matching result, and k is the nonlinear factor.

7. A visual-inertial SLAM loop detection system based on geomagnetic information optimization, wherein the visual-inertial SLAM loop detection process is based on the visual-inertial SLAM loop detection method according to any one of claims 1 to 6, characterized in that: It includes a coarse matching module, a local optimization module, a key frame introduction module, a visual geomagnetic sequence acquisition module, a cumulative distance matrix calculation module and a loop judgment module. The coarse matching module is used to solve the coarse matching of the collected data at a low resolution level and obtain the first geomagnetic sequence matching result; The local optimization module is used to optimize the first geomagnetic sequence matching result in a local neighborhood and obtain a second geomagnetic sequence matching result; The key frame introduction module is used to introduce a key frame image based on the second geomagnetic sequence matching result and calculate the geomagnetic intensity modulus value of the key frame image corresponding to the time stamp, thereby obtaining a key frame image containing the geomagnetic intensity modulus value; The visual geomagnetic sequence acquisition module is used to store key frame images containing geomagnetic intensity modulus values ​​into a buffer and form a visual geomagnetic sequence in the current buffer; The cumulative distance matrix calculation module is used to calculate the distance matrix between the visual geomagnetic sequence of the current buffer zone and the visual geomagnetic sequence of the historical buffer zone, and then calculate the cumulative distance matrix through dynamic programming to obtain the optimal matching path and the final matching distance; The loop detection module is used to select an area in the closest distance sequence as the visual feature matching area of ​​the visual word bag model according to the obtained optimal matching path and the final matching distance, and then use the visual word bag model to match the visual features of the selected area and determine whether there is a loop, thereby completing the visual inertial SLAM loop detection task.

Images