Underwater SLAM methods, systems, and devices based on fast hash loop closure detection

Abstract

This invention relates to the field of underwater positioning technology based on image features, specifically to an underwater SLAM method, system, and device based on fast hash loop closure detection. The method first enhances underwater images in real time and accurately extracts, matches, and filters key feature points, providing a reliable feature point foundation for underwater SLAM. Then, based on disparity and the number of key feature point pairs, key frames are determined. For non-key frames, factor graph optimization is performed using a fixed number of historical adjacent frame feature points, improving the efficiency of navigation parameter generation while ensuring accuracy. For key frames, global feature vectors are extracted and hashed to quickly retrieve the most similar key historical frames. These are then optimized with the historical frame underwater images using a pose graph to obtain navigation parameters, effectively reducing long-term positioning drift and comprehensively improving the accuracy and stability of underwater SLAM.

Description

Technical Field

[0001] This invention relates to the field of underwater localization technology based on image features, specifically to underwater SLAM methods, systems, and devices based on fast hash loop closure detection. Background Technology

[0002] Underwater visual SLAM faces numerous challenges from complex environments, such as high dynamic range of illumination, image blurring, and weak textures, leading to low reliability of feature extraction and matching, and poor accuracy in loop closure detection, severely impacting system precision and robustness. Furthermore, underwater platforms typically have limited computing resources, making it difficult to directly deploy complex visual SLAM models.

[0003] Currently, commonly used feature extraction and matching methods can be mainly divided into two categories: one is based on traditional handcrafted features, such as SIFT and ORB; the other is based on deep learning. These methods have the following problems:

[0004] Traditional manual feature methods are poorly adapted to underwater environments with large variations in lighting and weak or blurred textures. Feature repeatability and discriminativeness are significantly reduced, and the matching success rate is low, which seriously affects the tracking and mapping stability of SLAM systems.

[0005] While deep learning-based feature methods have improved matching accuracy, they have a large number of model parameters, slow inference speed, high hardware resource requirements, and are difficult to run in real time on low-computing-power underwater platforms.

[0006] (3) In terms of loop closure detection, existing methods are mostly based on bag-of-words models or linear retrieval strategies. Bag-of-words models need to be pre-trained and have limited generalization ability for underwater scenarios, while the time cost of linear retrieval increases sharply with the increase of the database, making it difficult to meet the needs of real-time SLAM applications. Summary of the Invention

[0007] The purpose of this invention is to provide an underwater SLAM method, system, and device based on fast hash loop closure detection.

[0008] The technical solution of this invention is as follows:

[0009] An underwater SLAM method based on fast hash loop closure detection includes the following operations:

[0010] S1: Real-time acquisition of underwater images, image enhancement processing to obtain underwater enhanced images; acquisition of feature points and descriptors of the current frame of underwater enhanced images, feature point matching and feature point filtering with the previous frame of underwater enhanced images to obtain key feature point pairs;

[0011] S2: Based on the disparity and number of key feature point pairs between the current frame underwater augmented image and the previous frame underwater image, determine whether the current frame underwater augmented image is a key frame; if not, the key feature points of the current frame underwater augmented image and the corresponding first number of historical adjacent frame underwater images are processed by factor graph optimization to obtain navigation parameters.

[0012] If so, execute S3;

[0013] S3: Obtain the global feature vector of the current frame's underwater augmented image, perform a hash operation to obtain a hash signature; based on the hash signature, query the hash table to extract the candidate feature vector with the highest similarity to the current frame's global feature vector and a similarity greater than the similarity threshold, and obtain the corresponding frame's underwater augmented image as the key historical frame image; the key historical frame image, all historical frame underwater augmented images, and the current frame's underwater augmented image are processed by pose graph optimization to obtain navigation parameters.

[0014] It also includes acquiring IMU data of the current frame underwater image, performing pre-integration processing to obtain the initial pose of the current frame, which is used as the initial value for feature point selection and is used to perform the feature point selection operation in S1.

[0015] It also includes acquiring IMU data for each frame in a first number of historical adjacent underwater images, performing pre-integration processing to obtain their respective initial poses, and then performing factor graph optimization processing in S2 with the initial pose of the current frame and the key feature points of each frame to obtain navigation parameters.

[0016] The feature point matching operation in S1 is as follows: Feature points and descriptors from the current frame's underwater augmented image and the previous frame's underwater augmented image are used to form feature maps for the current frame and the previous frame, respectively. After processing by a self-attention mechanism, these feature maps are residually connected to the current frame's feature map and the previous frame's feature map, and then subjected to cross-attention processing to obtain the current frame's attention feature map and the previous frame's attention feature map. The confidence scores of the current frame's attention feature map and the previous frame's attention feature map are obtained. Based on the confidence scores, point pruning is performed on the feature points of the current frame's and the previous frame's underwater augmented images to obtain initial feature point pairs. Based on the similarity score matrix and the matchability score matrix between the initial feature point pairs, a probabilistic assignment matrix is ​​obtained. The initial feature point pairs corresponding to the maximum values ​​of their respective rows and columns in the probabilistic assignment matrix are retained to obtain the matched feature point pairs.

[0017] The point pruning operation is as follows: the feature points of the underwater augmented image are corrected based on the confidence level of the neighborhood space constraint to obtain the corrected confidence level; the corrected confidence level of the feature points of the current frame and the corrected confidence level of the candidate matching feature points of the previous frame are obtained; feature point pairs whose corrected confidence level of the feature points and the corrected confidence level of the corresponding candidate matching feature points are greater than the mutual verification threshold and whose initial similarity is greater than the similarity threshold are retained to obtain the initial feature point pairs; among the initial feature point pairs, feature point pairs whose descriptor information entropy is greater than the information entropy threshold are retained to obtain the initial feature point pairs of the current frame and the previous frame.

[0018] The image enhancement operation in S1 is as follows: After white balance correction, the underwater image undergoes gamma correction and sharpening to obtain a gamma-corrected image and a sharpened image. The gamma-corrected image and the sharpened image are then processed by Gaussian pyramids to obtain a first Gaussian pyramid image and a second Gaussian pyramid image. The Laplacian weight, saliency weight, and saturation weight of the gamma-corrected image and the sharpened image are obtained, fused, and then normalized to obtain a gamma-corrected normalized weight image and a sharpened normalized image. The gamma-corrected normalized weight image and the sharpened normalized image are then processed by Laplacian pyramids to obtain a first Laplacian pyramid image and a second Laplacian pyramid image. The first Gaussian pyramid image is fused with the first Laplacian pyramid image, and the second Gaussian pyramid image is fused with the second Laplacian pyramid image, respectively, to obtain a Gaussian pyramid fused image and a Laplacian pyramid fused image. After fusion processing, the underwater enhanced image is obtained.

[0019] The method for obtaining feature points of the current frame underwater enhancement image in S1 is as follows: after reconstructing the 64-dimensional features of each 8×8 grid region pixel in the current frame underwater enhancement image, multiple convolutions are performed to obtain a convolutional feature map; based on the 64 channels of the convolutional feature map, a 65th channel is added, and a key point distribution probability map is generated by Softmax; points with a probability greater than the probability threshold are taken as feature points.

[0020] An underwater SLAM system based on fast hash loop closure detection, used to implement the above-mentioned underwater SLAM method based on fast hash loop closure detection, includes:

[0021] The key feature point generation module is used to acquire underwater images in real time, perform image enhancement processing, and obtain underwater enhanced images; it acquires the feature points and descriptors of the current frame of underwater enhanced images, performs feature point matching and feature point filtering with the previous frame of underwater enhanced images, and obtains key feature point pairs.

[0022] The first navigation parameter generation module is used to determine whether the current frame of underwater augmented image is a key frame based on the disparity and the number of key feature point pairs between the current frame of underwater augmented image and the previous frame of underwater image; if not, the key feature points of the current frame of underwater augmented image and the corresponding first number of historical adjacent frame underwater images are processed by factor graph optimization to obtain navigation parameters; if so, the second navigation parameter generation module is executed.

[0023] The second navigation parameter generation module is used to obtain the global feature vector of the current frame underwater augmented image, and obtain a hash signature through hash operation; based on the hash signature, it queries the hash table to extract the candidate feature vector with the highest similarity to the current frame global feature vector and the similarity is greater than the similarity threshold, and obtains the corresponding frame underwater augmented image as the key historical frame image; the key historical frame image, all historical frame underwater augmented images, and the current frame underwater augmented image are processed by pose graph optimization to obtain the navigation parameters.

[0024] An underwater SLAM device based on fast hash loop closure detection includes a processor and a memory, wherein the processor implements the above-described underwater SLAM method based on fast hash loop closure detection when executing a computer program stored in the memory.

[0025] A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described underwater SLAM method based on fast hash loop closure detection.

[0026] The beneficial effects of this invention are as follows:

[0027] This invention provides an underwater SLAM method based on fast hash loop closure detection. First, by real-time enhancement of underwater images and accurate extraction, matching, and screening of key feature points, it effectively suppresses pseudo-feature interference caused by underwater red channel attenuation and oversaturation, providing a reliable feature foundation for underwater SLAM and ensuring positioning accuracy from the source. Then, key frames are determined based on disparity and the number of key feature points. For non-key frames, factor graph optimization is performed by combining feature points from historical adjacent frames, which avoids redundant calculations and improves real-time performance, while also using multi-frame information to offset single-frame noise and enhance the stability of navigation parameters. For key frames, global feature vectors are extracted and hashed to quickly retrieve the key historical frames with the highest similarity. These are then compared with all frames of underwater images and processed by pose graph optimization to obtain navigation parameters, comprehensively improving the accuracy and stability of underwater SLAM.

[0028] This invention provides an underwater SLAM method based on fast hash loop closure detection. It designs a hash loop closure detection method based on the current frame of the underwater image, which can quickly complete the preliminary retrieval of a massive keyframe database within constant time complexity. This overcomes the bottleneck problem that the retrieval efficiency of the traditional bag-of-words model decreases linearly with map expansion, and significantly improves the real-time performance of loop closure detection. This mechanism does not require a pre-trained model, has strong generalization ability, and effectively avoids the problem of poor adaptability of the bag-of-words model in specific scenarios such as underwater. Attached Figure Description

[0029] The solutions and advantages of this application will become clear to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0030] In the attached diagram:

[0031] Figure 1 This is a flowchart illustrating the method of this embodiment.

[0032] Figure 2 The above are illustrations showing the underwater image enhancement effects of the method in this embodiment and existing methods.

[0033] Figure 3 As shown in the embodiment, this is a diagram illustrating the feature point matching effect of the method in underwater images under varying perspectives, weak textures, and lighting conditions.

[0034] Figure 4 This is a schematic diagram of the SLAM trajectory and loop position of the method in this embodiment. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the exemplary embodiments of this application clearer, the technical solutions in the exemplary embodiments of this application are described clearly and completely below. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0036] An underwater SLAM method based on fast hash loop closure detection, see [link to relevant documentation]. Figure 1 This includes the following operations:

[0037] S1: Real-time acquisition of underwater images, image enhancement processing to obtain underwater enhanced images; acquisition of feature points and descriptors of the current frame of underwater enhanced images, feature point matching and feature point filtering with the previous frame of underwater enhanced images to obtain key feature point pairs;

[0038] S2: Based on the disparity and number of key feature point pairs between the current frame underwater augmented image and the previous frame underwater image, determine whether the current frame underwater augmented image is a key frame; if not, the key feature points of the current frame underwater augmented image and the corresponding first number of historical adjacent frame underwater images are processed by factor graph optimization to obtain navigation parameters.

[0039] If so, execute S3;

[0040] S3: Obtain the global feature vector of the current frame's underwater augmented image, perform a hash operation to obtain a hash signature; based on the hash signature, query the hash table to extract the candidate feature vector with the highest similarity to the current frame's global feature vector and a similarity greater than the similarity threshold, and obtain the corresponding frame's underwater augmented image as the key historical frame image; the key historical frame image, all historical frame underwater augmented images, and the current frame's underwater augmented image are processed by pose graph optimization to obtain navigation parameters.

[0041] The specific steps are detailed below.

[0042] S1: Acquire underwater images in real time, perform image enhancement processing to obtain underwater enhanced images; acquire feature points and descriptors of the current frame of underwater enhanced images, perform feature point matching and feature point filtering with the previous frame of underwater enhanced images to obtain key feature point pairs.

[0043] By acquiring underwater images in real time and performing image enhancement processing, the defects of underwater environment caused by light scattering and water absorption, such as image blurring and insufficient contrast, are effectively improved, so that weak feature points that were originally affected by noise and had low visibility can be clearly presented. Furthermore, by matching and filtering feature points of the current frame with the previous frame of the enhanced image, redundant feature points affected by instantaneous noise and with poor cross-frame stability are eliminated, and only key feature point pairs with cross-frame consistency and high discriminativeness are retained, thereby significantly improving the robustness and effectiveness of feature points.

[0044] First, the underwater camera acquires underwater images in real time. Since underwater images usually have obvious problems such as blue-green color cast, insufficient brightness, poor contrast and blurred details, in order to obtain high-quality underwater images, this embodiment sets up image enhancement processing on the real-time acquired underwater images to obtain underwater enhanced images.

[0045] The specific details of the image enhancement operation steps are as follows.

[0046] Step 1: After white balance correction, the underwater image is then subjected to gamma correction and sharpening to enhance the contrast and brightness of the underwater image, as well as the edges and details of the image, resulting in the gamma-corrected image and the sharpened image.

[0047] Sharpening can be achieved using the following formula:

[0048] ,

[0049] To sharpen the image, For white balance correction images, Indicates to Perform Gaussian filtering. These are the filter parameters.

[0050] Step 2: Perform Gaussian pyramid processing on the gamma-corrected image and the sharpened image respectively to obtain the first Gaussian pyramid image and the second Gaussian pyramid image.

[0051] Step 3: Obtain the Laplacian weight, saliency weight, and saturation weight of the gamma-corrected image and the sharpened image respectively. After fusion, normalize the results to obtain the gamma-corrected normalized weight and the sharpened normalized image.

[0052] The saliency weight is obtained based on the gradient strength and local information entropy of the image to be processed, and the divergence index of the white balance corrected image. The calculation formula is as follows:

[0053] ,

[0054] ,

[0055] ,

[0056] ,

[0057] For significance weight, The gradient intensity of the image to be processed is either a gamma-corrected image or a sharpened image. , The images to be processed are in x Brightness gradient in the y-direction The brightness of the image to be processed. The local information entropy of the image to be processed is used to enhance the weight values ​​of corner / edge regions while avoiding color shift interference in the brightness domain. For the first k The probability of the brightness histogram of a region of interest (which can be randomly selected). The first compensation parameter is an empirical value. The divergence index is the value of the white balance corrected image, which is obtained by white balance correction of the underwater image. This embodiment is designed based on... Weighting can suppress bright but unstructured water atomization regions, where In pixels Center window The mean and standard deviation, It is the Sigmoid function. As the second compensation parameter, an empirical value, many underwater "bright fog areas" are misjudged as significant by ordinary significance, while the formula is designed... It can suppress it, explicitly suppressing spurious divergence. , , These are the gradient strength coefficient, the local information entropy coefficient, and the divergence indicator coefficient, respectively.

[0058] Underwater images typically present two practical problems: severe red channel attenuation, making it easy to artificially inflate saturation through noise or enhancement; and oversaturation leading to local color inconsistencies, which harms both photometric consistency and feature descriptors. Therefore, in this embodiment, the saturation weight is determined based on the perceived saturation, moderate saturation preference, and channel reliability of the image to be processed. The image to be processed is either a gamma-corrected or sharpened image. The calculation formula is as follows:

[0059] ,

[0060] ,

[0061] ,

[0062] ,

[0063] For saturation weights, To perceive saturation, To perceive the saturation coefficient, For brightness, , These are the chromaticity components in the Lab color space. a Representing the weight from green to red, b Representing the range from blue to yellow, This is the third compensation item, based on experience. For moderate saturation preference, This is an empirical saturation value, with a range of (0.25, 0.35). The standard deviation of empirical saturation is taken in the range of (0.15, 0.25). Colors that are too gray (low saturation) have poor distinguishability, while colors that are too vibrant often come from unreliable enhancement or local clipping, both of which will harm matching consistency. Therefore, the middle range is used. Colors that are too low or too high are weighted less. To ensure channel reliability, this feature is used to suppress red channel noise saturation. When the red channel is unreliable, it automatically reduces the weighting to prevent false saturation. , , These represent the pixel mean values ​​of the local neighborhood window in the red, green, and blue channels, reflecting the local intensity of each channel. , , These represent the pixel standard deviations of the local neighborhood window in the red, green, and blue channels, respectively. , These are the fourth and fifth compensation items, respectively.

[0064] Step 4: Perform Laplacian pyramid processing on the gamma-corrected normalized weight and sharpened normalized images respectively to obtain the first Laplacian pyramid image and the second Laplacian pyramid image.

[0065] Step 5: Perform image fusion on the corresponding layers of the first Gaussian pyramid image and the first Laplacian pyramid image, and the second Gaussian pyramid image and the second Laplacian pyramid image, respectively, to obtain the Gaussian pyramid fused image and the Laplacian pyramid fused image. After fusion processing, an underwater enhanced image is obtained. This process solves the halo effect that may occur in ordinary fusion results, avoids detail loss, and improves image quality. The image enhancement processing results of this embodiment can be found in [reference needed]. Figure 2 Compared with existing advanced image enhancement methods such as CLAHE and MSRCR, the method in this embodiment can effectively remove color deviations in underwater images, improve overall brightness, and enhance detail information.

[0066] Then, the feature points and descriptors of the current frame of underwater augmented image are obtained, and feature point matching is performed with the previous frame of underwater augmented image to obtain matching feature point pairs. After feature point filtering, key feature point pairs are obtained.

[0067] The method for obtaining feature points of the current frame underwater enhancement image is as follows: after reconstructing the 64-dimensional features of each 8×8 grid region pixel in the current frame underwater enhancement image, perform multiple convolutions (preferably 4 convolutions) to obtain a convolution feature map; on the basis of the 64 channels of the convolution feature map, add a 65th channel to distinguish featureless regions, generate a key point distribution probability map through Softmax, and take the points with a probability greater than the probability threshold as feature points.

[0068] The method for obtaining the descriptor corresponding to the feature point of the current frame underwater enhanced image is as follows: 64-dimensional feature reconstruction is performed on each pixel of the 8×8 grid region in the current frame underwater enhanced image to obtain the initial reconstruction map; the initial reconstruction map is upsampled to obtain the first reconstructed upsampled map; the initial reconstruction map is convolved and upsampled to obtain the second reconstructed upsampled map; the initial reconstruction map, the first reconstructed upsampled map, and the second reconstructed upsampled map are added element-wise to obtain the sample fusion feature map; the feature vector of the 8×8 grid region where the feature point is located in the sample fusion feature map is used as the descriptor of the feature point.

[0069] The specific details of feature point matching are as follows.

[0070] Step 1: Take the feature maps formed by the feature points and descriptors of the current frame underwater augmented image and the previous frame underwater augmented image, and use them as the current frame feature map and the previous frame feature map. After processing by the self-attention mechanism, perform residual connections with the current frame feature map and the previous frame feature map respectively, and then perform cross-attention processing to obtain the current frame attention feature map and the previous frame attention feature map.

[0071] Step 2: Obtain the confidence scores of the attention feature map of the current frame and the attention feature map of the previous frame (which can be obtained through a multilayer perceptron). Based on the confidence scores, perform point pruning on the feature points of the underwater augmentation images of the current frame and the previous frame to obtain initial feature point pairs.

[0072] The steps for point pruning are as follows.

[0073] The confidence of feature points in the underwater enhanced image is corrected based on neighborhood space constraints. This achieves local neighborhood space constraints and obtains the corrected confidence, which facilitates the subsequent filtering out of false high-confidence feature points caused by local noise within a single frame.

[0074] The corrected confidence level is calculated using the following formula:

[0075] ,

[0076] ,

[0077] ,

[0078] For the first i Corrected confidence level of each feature point For the first i The (initial) confidence level of each feature point. For the first i Mean confidence score of the neighborhood of each feature point For the first i The neighborhood confidence standard deviation of each feature point For the first i In the neighborhood of the nth feature point l The (initial) confidence level of each point, L For the first i The total number of neighborhood points of each feature point.

[0079] The corrected confidence of feature points in the current frame is obtained. The corrected confidence of candidate matching feature points in the previous frame, the corrected confidence of retaining only feature points, and the corrected confidence of corresponding candidate matching feature points are all greater than the mutual verification threshold. Feature point pairs with an initial similarity greater than the similarity threshold between the feature point and the candidate matching feature point are obtained. This achieves cross-frame adaptive mutual verification of feature point confidence, avoids the one-sidedness of single-frame evaluation, and removes invalid feature points with high confidence in a single frame but no corresponding cross-frame matching, thereby reducing the amount of subsequent matching calculations.

[0080] The mutual verification threshold is calculated using the following formula:

[0081] ,

[0082] For mutual verification threshold, The mean confidence score is corrected for all feature points in the current frame. The mean of the confidence scores is corrected for all (candidate) feature points in the current frame.

[0083] The initial similarity between a feature point and a candidate matching feature point (cross-frame candidate matching similarity) is the cosine similarity of the corresponding descriptor.

[0084] In the initial screening of feature point pairs, feature point pairs with descriptor information entropy greater than the information entropy threshold are retained to solve the problem of high confidence but low discriminative ability caused by repeated textures in underwater images. Redundant feature points with high confidence but poor descriptor discriminative ability are pruned to further improve the matching accuracy and obtain the initial feature point pairs of the current frame and the previous frame.

[0085] Step 3: Based on the similarity score matrix and the matchability score matrix between the initial feature point pairs, obtain the probabilistic assignment matrix. Retain the initial feature point pairs that are the maximum values ​​of their respective rows and columns in the probabilistic assignment matrix to obtain the matching feature point pairs.

[0086] In the similarity score matrix, the similarity score of the initial feature point pairs can be calculated using the following formula:

[0087] ,

[0088] For initial feature point pairs i , j Similarity score, For mutual information score calculation, For initial feature point pairs i , j descriptor, For initial feature point pairs i , j The set of gray values ​​in the neighborhood, Mutual information weights are used to balance the weights of descriptors and neighborhood structures. Since underwater images (before the background) may have lighting variations, the mutual information of neighborhood grayscale can reflect the similarity of local structures and is more robust than a single descriptor.

[0089] In the matchability score matrix, the matchability score of the initial feature point pair can be calculated using the following formula:

[0090] ,

[0091] For initial feature point pairs i , j Matchability score , These are the initial feature point pairs. i , j neighborhood descriptor entropy, , These are the initial feature point pairs. i , j The set of gradient magnitudes under multiple Gaussian blur scales The coefficient of variation function ensures that the local complexity of the matching pairs is similar through the pairwise entropy difference, and filters feature points that are susceptible to noise interference through response stability, thereby reducing the probability of mismatches from the root.

[0092] In the probabilistic assignment matrix, the assignment probability values ​​of the initial feature point pairs are calculated using the following formula:

[0093] ,

[0094] For initial feature point pairs i , j The probability value of the allocation. For action i All scores and, For example j Sum of all scores, minus scores that were counted twice. This ensures that the denominator is the total global score of "row + column". If a feature point has a high score in a row but a low score in the corresponding column, its probability will be significantly reduced (because there is a better match in the column), naturally avoiding one-to-many mismatches.

[0095] For probabilistic assignment matrix Retain initial feature point pairs that satisfy the following conditions : .

[0096] The aforementioned feature point selection method can be implemented using the RANSAC method.

[0097] To demonstrate the effectiveness of feature point matching and feature point selection in this embodiment, feature point matching was performed on underwater images with varying viewpoints, weak textures, and changes in illumination. The results are shown in [link to documentation]. Figure 3 It can be seen that the method of this embodiment can achieve high-precision matching in underwater image perspective changes, weak texture and illumination changes.

[0098] S2: Based on the disparity and number of key feature point pairs between the current frame underwater augmented image and the previous frame underwater image, determine whether the current frame underwater augmented image is a key frame; if not, the key feature points of the current frame underwater augmented image and the corresponding first number of historical adjacent frame underwater images are processed by factor graph optimization to obtain navigation parameters; if so, execute S3.

[0099] Using disparity and the number of key feature point pairs as criteria, the value of the current frame can be quickly distinguished, avoiding redundant storage and processing of non-key frames and reducing computational overhead. Non-key frames can be fused with feature points from adjacent historical frames and the navigation parameters can be optimized using factor graphs. This can help offset the possible false saturation and feature noise interference in underwater enhanced images by leveraging information from multiple frames, thereby improving the stability of parameter estimation. Accurate selection of key frames can preserve core motion and scene information, providing a reliable foundation for subsequent loop closure detection in S3 and balancing navigation real-time performance and accuracy.

[0100] Based on the disparity and the number of key feature point pairs between the current frame underwater augmented image and the previous frame underwater image, determine whether the current frame underwater augmented image is a key frame.

[0101] The disparity between the current frame of the underwater augmented image and the previous frame of the underwater image is the difference between the coordinates of key feature point pairs. When the disparity between the current frame of the underwater augmented image and the previous frame of the underwater image is greater than the disparity threshold, and the number of key feature point pairs is greater than the key feature point pair count threshold, the current frame of the underwater augmented image is a key frame; otherwise, the current frame of the underwater augmented image is not a key frame.

[0102] If not, the current frame underwater augmented image is a key frame. The key feature points of the current frame underwater augmented image and the corresponding first number (9 frames) of historical adjacent frame underwater images are processed by factor graph optimization to obtain navigation parameters, which include position and attitude.

[0103] If so, perform loopback detection in S3.

[0104] S3: Obtain the global feature vector of the current frame's underwater augmented image, perform a hash operation to obtain a hash signature; based on the hash signature, query the hash table to extract the candidate feature vector with the highest similarity to the current frame's global feature vector and a similarity greater than the similarity threshold, and obtain the corresponding frame's underwater augmented image as the key historical frame image; the key historical frame image, all historical frame underwater augmented images, and the current frame's underwater augmented image are processed by pose graph optimization to obtain navigation parameters.

[0105] By generating hash signatures through global feature vector hashing and querying for matching, it can quickly complete the retrieval of massive keyframes with constant time complexity, breaking through the bottleneck of the linear decrease in retrieval efficiency of traditional bag-of-words models as the map expands, ensuring the real-time solution of navigation parameters. Furthermore, because it does not require a pre-trained model and has strong generalization ability, it can effectively adapt to special scenarios such as underwater red attenuation and illumination fluctuations, avoiding the problem of poor adaptability of traditional models. Combining the key historical frames with the highest similarity with all frames for pose graph optimization can introduce high-quality historical scene information to offset the noise and pseudo-feature interference of underwater enhanced images, reduce parameter estimation bias, and further improve the accuracy, stability, and robustness of navigation parameters.

[0106] First, the vector formed by the descriptors corresponding to all key feature points of the current frame underwater augmented image is used as the global feature vector of the current frame underwater augmented image. This feature represents the overall information of the image in the form of a high-dimensional floating-point vector.

[0107] Then, the global feature vector is hashed to obtain a hash signature. Specifically, several hyperplanes are randomly generated, dividing the entire space into multiple sub-regions. Data within the same sub-region are considered spatially adjacent. The global feature vector of the image is projected onto the hyperplanes, and a binary hash signature is generated using projection symbols. The combination of symbols on different hyperplanes forms a hash signature of length n, serving as a low-dimensional index representation of the features.

[0108] Next, based on the hash signature, a query is performed in the hash table to extract candidate feature vectors whose global feature vector has the highest similarity to the current frame's global feature vector and whose similarity is greater than a similarity threshold. The corresponding underwater enhanced image of this frame is then obtained as the key historical frame image. Specifically, using a bit offset m, multiple hash buckets within the offset range m of the hash signature are searched. A set of candidate feature vectors is read from each bucket. The candidate feature vector with the highest cosine similarity to the current frame's global feature vector and whose similarity is greater than a similarity threshold is selected as the old key frame similar to the current frame, and the corresponding underwater enhanced image of this frame is obtained as the key historical frame image.

[0109] Finally, the navigation parameters are obtained by simulating the pose graph of the key historical frame images, all historical frame underwater augmented images, and the current frame underwater augmented image.

[0110] For the hash loop closure detection effect of this embodiment, please refer to [link / reference]. Figure 4 Schematic diagram of SLAM trajectory and loop position. Figure 4 The green area represents the odometry track, the red area represents the closed-loop optimized track, and the enlarged views of the blue and yellow dashed boxes are located in... Figure 4 The lower left part of the middle, through Figure 4It can be clearly observed that the loop closure detection successfully found multiple loops (red line segments), correcting the trajectory divergence of the odometer, indicating that the hash loop closure detection method of this embodiment works well in underwater scenarios.

[0111] In addition, to further improve the accuracy of navigation parameters, this embodiment also includes acquiring IMU data of the underwater image of the current frame, performing pre-integration processing to obtain the initial pose of the current frame, which is used as the initial value for feature point selection and is used to perform the feature point selection operation in S1.

[0112] Furthermore, it also includes acquiring IMU data for each frame in a first number of historical adjacent underwater images, performing pre-integration processing to obtain their respective initial poses, and then performing factor graph optimization processing in S2 with the initial pose of the current frame and the key feature points of each frame to obtain navigation parameters.

[0113] This embodiment also provides an underwater SLAM system based on fast hash loop closure detection, used to implement the above-mentioned underwater SLAM method based on fast hash loop closure detection, including:

[0114] The key feature point generation module is used to acquire underwater images in real time, perform image enhancement processing, and obtain underwater enhanced images; it acquires the feature points and descriptors of the current frame of underwater enhanced images, performs feature point matching and feature point filtering with the previous frame of underwater enhanced images, and obtains key feature point pairs.

[0115] The first navigation parameter generation module is used to determine whether the current frame of underwater augmented image is a key frame based on the disparity and the number of key feature point pairs between the current frame of underwater augmented image and the previous frame of underwater image; if not, the key feature points of the current frame of underwater augmented image and the corresponding first number of historical adjacent frame underwater images are processed by factor graph optimization to obtain navigation parameters; if so, S3 is executed.

[0116] The second navigation parameter generation module is used to obtain the global feature vector of the current frame underwater augmented image, and obtain a hash signature through hash operation; based on the hash signature, it queries the hash table to extract the candidate feature vector with the highest similarity to the current frame global feature vector and the similarity is greater than the similarity threshold, and obtains the corresponding frame underwater augmented image as the key historical frame image; the key historical frame image, all historical frame underwater augmented images, and the current frame underwater augmented image are processed by pose graph optimization to obtain the navigation parameters.

[0117] This embodiment also provides an underwater SLAM device based on fast hash loop closure detection, including a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the above-described underwater SLAM method based on fast hash loop closure detection.

[0118] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described underwater SLAM method based on fast hash loop closure detection.

[0119] This embodiment provides an underwater SLAM method based on fast hash loop closure detection. First, by real-time enhancement of underwater images and accurate extraction, matching, and screening of key feature points, it effectively suppresses pseudo-feature interference caused by underwater red channel attenuation and oversaturation, providing a reliable feature foundation for underwater SLAM and ensuring positioning accuracy from the source. Then, key frames are determined based on disparity and the number of key feature points. For non-key frames, factor graph optimization is performed by combining feature points from historical adjacent frames, which avoids redundant calculations and improves real-time performance, while also using multi-frame information to cancel single-frame noise and enhance the stability of navigation parameters. For key frames, global feature vectors are extracted and hashed to quickly retrieve the key historical frames with the highest similarity. These are then compared with all frames of underwater images and optimized using pose graphs to obtain navigation parameters, comprehensively improving the accuracy and stability of underwater SLAM.

[0120] This embodiment provides an underwater SLAM method based on fast hash loop closure detection. It designs a hash loop closure detection method based on the current frame of the underwater image, which can quickly complete the preliminary retrieval of a massive keyframe database in constant time complexity. It overcomes the bottleneck problem that the retrieval efficiency of the traditional bag-of-words model decreases linearly with map expansion, and significantly improves the real-time performance of loop closure detection. This mechanism does not require a pre-trained model, has strong generalization ability, and effectively avoids the problem of poor adaptability of the bag-of-words model in specific scenarios such as underwater.

[0121] While exemplary embodiments of the invention have been described herein, many other variations or modifications conforming to the principles of the invention can be directly determined or derived from the disclosure of this invention without departing from its spirit and scope. Therefore, the scope of the invention should be understood and recognized to cover all such other variations or modifications.

Claims

1. An underwater SLAM method based on fast hash loop closure detection, characterized in that, This includes the following operations: S1: Real-time acquisition of underwater images, image enhancement processing to obtain an enhanced underwater image; the image enhancement processing operation is as follows: after white balance correction, the underwater image is subjected to gamma correction and sharpening processing respectively to obtain gamma-corrected images and sharpened images; the gamma-corrected images and sharpened images are subjected to Gaussian pyramid processing respectively to obtain a first Gaussian pyramid image and a second Gaussian pyramid image; the Laplacian weights, saliency weights, and saturation weights of the gamma-corrected images and sharpened images are obtained respectively, and after fusion, normalization processing is performed to obtain gamma-corrected normalized weights and sharpened normalized images; the gamma-corrected normalized weights and sharpened normalized images are subjected to Laplacian pyramid processing respectively to obtain a first Laplacian pyramid image and a second Laplacian pyramid image; the first Gaussian pyramid image is fused with the first Laplacian pyramid image, and the second Gaussian pyramid image is fused with the second Laplacian pyramid image, respectively, to obtain a Gaussian pyramid fused image and a Laplacian pyramid fused image, which are then fused to obtain the enhanced underwater image; The feature points and descriptors of the current frame of underwater augmented image are obtained, and feature point matching and filtering are performed with the previous frame of underwater augmented image to obtain key feature point pairs. S2: Based on the disparity between the current frame underwater augmented image and the previous frame underwater image and the number of key feature point pairs, determine whether the current frame underwater augmented image is a key frame; If not, obtain the IMU data of each frame in the first number of historical adjacent underwater images, perform pre-integration processing, and use it as the initial pose of each frame. Combine the initial pose of the current frame, the underwater augmented image of the current frame, and the key feature points of the corresponding first number of historical adjacent underwater images, and perform factor graph optimization processing to obtain the navigation parameters. If so, execute S3; S3: Obtain the global feature vector of the underwater enhanced image in the current frame, perform a hash operation, and obtain the hash signature; Based on the hash signature, a query is performed in the hash table to extract the candidate feature vectors that have the highest similarity to the global feature vector of the current frame and whose similarity is greater than the similarity threshold, and the corresponding underwater enhanced image of the frame is obtained as the key historical frame image. Navigation parameters are obtained by processing key historical frame images, underwater enhanced images of all historical frames, and underwater enhanced images of the current frame through pose graph optimization; the navigation parameters include position and attitude.

2. The underwater SLAM method based on fast hash loop closure detection according to claim 1, characterized in that, It also includes acquiring the IMU data of the underwater image of the current frame, performing pre-integration processing to obtain the initial pose of the current frame, which is used as the initial value for feature point selection and is used to perform the feature point selection operation in S1.

3. The underwater SLAM method based on fast hash loop closure detection according to claim 1, characterized in that, The feature point matching operation in S1 is as follows: The feature maps formed by the feature points and descriptors of the current frame underwater augmented image and the previous frame underwater augmented image are used as the current frame feature map and the previous frame feature map, respectively. After being processed by the self-attention mechanism, they are respectively connected with the current frame feature map and the previous frame feature map, and then processed by cross-attention to obtain the current frame attention feature map and the previous frame attention feature map. The confidence scores of the attention feature map of the current frame and the attention feature map of the previous frame are obtained respectively. Based on the confidence scores, the feature points of the underwater enhancement images of the current frame and the previous frame are pruned to obtain the initial feature point pairs. Based on the similarity score matrix and the matchability score matrix between the initial feature point pairs, a probabilistic assignment matrix is ​​obtained. The initial feature point pairs corresponding to the maximum values ​​of their respective rows and columns in the probabilistic assignment matrix are retained to obtain the matching feature point pairs.

4. The underwater SLAM method based on fast hash loop closure detection according to claim 3, characterized in that, The pruning operation is as follows: The feature points of the underwater enhanced image are subjected to confidence correction based on neighborhood space constraints to obtain the corrected confidence. Obtain the corrected confidence of feature points in the current frame and the corrected confidence of candidate matching feature points in the previous frame. Retain feature point pairs whose corrected confidence of the feature points and the corrected confidence of the corresponding candidate matching feature points are greater than the mutual verification threshold and whose initial similarity is greater than the similarity threshold, and obtain the initial screening feature point pairs. In the initial screening of feature point pairs, the feature point pairs whose descriptor information entropy is greater than the information entropy threshold are retained to obtain the initial feature point pairs of the current frame and the previous frame.

5. The underwater SLAM method based on fast hash loop closure detection according to claim 1, characterized in that, The method for obtaining feature points of the current frame underwater enhancement image in S1 is as follows: after reconstructing the 64-dimensional features of each 8×8 grid region pixel in the current frame underwater enhancement image, multiple convolutions are performed to obtain a convolutional feature map; based on the 64 channels of the convolutional feature map, a 65th channel is added, and a key point distribution probability map is generated by Softmax. Points with a probability greater than the probability threshold are taken as feature points.

6. An underwater SLAM system based on fast hash loop closure detection, characterized in that, The underwater SLAM method based on fast hash loop closure detection as described in claim 1 is characterized by comprising: The key feature point generation module is used to acquire underwater images in real time, perform image enhancement processing, and obtain underwater enhanced images; it acquires the feature points and descriptors of the current frame of underwater enhanced images, performs feature point matching and feature point filtering with the previous frame of underwater enhanced images, and obtains key feature point pairs. The first navigation parameter generation module is used to determine whether the current frame of underwater augmented image is a key frame based on the disparity and the number of key feature point pairs between the current frame of underwater augmented image and the previous frame of underwater image; if not, the key feature points of the current frame of underwater augmented image and the corresponding first number of historical adjacent frame underwater images are processed by factor graph optimization to obtain navigation parameters; if so, the second navigation parameter generation module is executed. The second navigation parameter generation module is used to obtain the global feature vector of the current frame underwater augmented image, and obtain a hash signature through hash operation; based on the hash signature, it queries the hash table to extract the candidate feature vector with the highest similarity to the current frame global feature vector and the similarity is greater than the similarity threshold, and obtains the corresponding frame underwater augmented image as the key historical frame image; the key historical frame image, all historical frame underwater augmented images, and the current frame underwater augmented image are processed by pose graph optimization to obtain the navigation parameters.

7. An underwater SLAM device based on fast hash loop closure detection, characterized in that, It includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the underwater SLAM method based on fast hash loop closure detection as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the underwater SLAM method based on fast hash loop closure detection as described in any one of claims 1-5.

Images