Loop closure detection method based on point cloud elevation truncation and hatch feature

Abstract

This invention discloses a loop closure detection method based on point cloud elevation truncation and hatch features. First, gravity-aligned point clouds are extracted, and the bulk material point cloud interference is truncated using an elevation histogram to obtain initial hatch data. Hatch boundaries are extracted using image semantic segmentation, and refined point clouds and image features are generated. Second, planar patches of the hold wall are extracted and intersected to generate 3D line segments and corner points. Multimodal feature descriptors are then concatenated to construct a local topology graph with corner points as nodes and line segments and their weights as edges. Subsequently, subgraph isomorphic matching is performed between the current graph and historical graphs to calculate node similarity and edge consistency, outputting a comprehensive score. Finally, loop closure frames are identified and pose transformations are calculated through dynamic threshold filtering and random sampling consistency geometric verification. This method effectively eliminates dynamic interference, integrates multimodal features, and improves the accuracy and robustness of loop closure detection in complex port environments.

Description

Technical Field

[0001] This invention relates to the technical field of loop closure detection, and more particularly to a loop closure detection method based on point cloud elevation truncation and hatch features. Background Technology

[0002] With the rapid development of global trade and the continuous improvement of port automation, the demand for intelligent bulk carrier loading and unloading operations is becoming increasingly urgent. In port bulk cargo handling scenarios, large equipment such as ship loaders and unloaders need precise positioning and path planning in the complex and ever-changing ship hold environment to achieve efficient and safe automated operations. Meanwhile, Simultaneous Localization and Mapping (SLAM) technology, as the core support for achieving autonomous navigation of equipment, directly determines the reliability and accuracy of the entire automation system. However, in the unique operating environment of a bulk carrier hold, traditional SLAM systems face severe challenges.

[0003] Loop closure detection, a key module of SLAM systems, aims to identify whether a device has returned to a historical position. By establishing pose constraints, it eliminates long-term accumulated odometry errors and ensures the consistency of the global map. Currently, mainstream loop closure detection methods are mainly divided into appearance-based and geometry-based methods. Appearance-based methods, such as the bag-of-words model, perform similarity matching by extracting global descriptors from images or point clouds. While computationally efficient, they are sensitive to changes in ambient lighting and viewpoint differences, and are prone to mismatches in environments with high structural repetition, such as bulk carrier holds. Geometry-based methods, such as ICP registration and feature point matching, while robust to viewpoint changes, have high computational complexity, struggle to guarantee real-time performance, and experience a sharp performance degradation in dynamic environments.

[0004] In bulk carrier hold operations, existing loop closure detection methods face even greater challenges. First, the dynamic surfaces formed by the accumulation of bulk materials within the hold continuously alter the environment's geometry. Traditional global point cloud matching methods suffer from numerous false matches due to the interference of dynamic bulk materials, leading to loop closure detection failures or incorrect pose corrections. Second, while the hold bulkhead structure is relatively stable, its surface typically lacks significant texture features, making it difficult for visual appearance-based methods to extract reliable feature descriptors. Third, the hatch area, a critical location for ship loader operations, exhibits certain geometric regularities in its coaming structure, but existing methods lack effective utilization of this specific structure, failing to reliably identify hatch features under dynamic bulk material interference.

[0005] Furthermore, existing methods have limitations in handling multimodal data fusion. While some studies have attempted to combine LiDAR point clouds and camera images for loop closure detection, most employ simple feature stitching or weighted fusion strategies, failing to fully leverage the complementary advantages of different modalities. In the environment of a bulk carrier hold, point cloud data provides accurate 3D geometric information, while image data captures rich texture and semantic information. How to effectively fuse the features of these two modalities to construct a loop closure detection framework robust to dynamic disturbances has become a pressing technical challenge.

[0006] In summary, existing loop closure detection methods have significant limitations in the dynamic and structured environment of bulk carrier holds: they are sensitive to dynamic bulk material disturbances, lack effective utilization of hatch features, and have inadequate multimodal fusion strategies. Therefore, there is an urgent need to develop a loop closure detection method that can effectively address dynamic bulk material disturbances, fully utilize hatch structural features, and achieve deep multimodal fusion to improve the positioning accuracy and reliability of automated bulk cargo handling systems in ports. Summary of the Invention

[0007] In view of this, the present invention provides a loop closure detection method based on point cloud elevation truncation and hatch features. The purpose is to effectively remove the interference of dynamic bulk material point cloud through elevation truncation technology, construct a stable topological graph representation using the structural features of the hatch coaming, and achieve multimodal loop closure detection by integrating point cloud geometric features and image semantic features. This enables high-precision and robust position recognition and pose estimation in complex port loading and unloading operation scenarios, providing reliable technical support for the autonomous navigation of port equipment such as ship loaders and ship unloaders.

[0008] To achieve the above objectives, this invention provides a loop closure detection method based on point cloud elevation truncation and hatch features, comprising the following steps: B1: Input raw point cloud data and auxiliary image data, extract the gravity direction of the inertial measurement unit to align the raw point cloud data, project the aligned point cloud data onto the elevation histogram, extract the elevation separation plane corresponding to the bulk material accumulation surface, truncate and remove the dynamic bulk material point cloud data below the elevation separation plane, and obtain the initial upper hatch point cloud data; input the auxiliary image data into the semantic segmentation network to extract the hatch boundary pixels, map the hatch boundary pixels to the initial upper hatch point cloud data, and output the refined upper hatch point cloud data and hatch image feature descriptor; B2: Perform normal vector estimation on the refined upper hatch point cloud data to extract planar patches of the cabin bulkhead, perform intersection calculation on spatially adjacent planar patches to generate three-dimensional spatial line segments of the hatch coaming, extract the intersection points of the three-dimensional spatial line segments as hatch corner points, and perform feature stitching between the hatch image feature descriptor and the hatch corner points to output a multimodal hatch corner point feature descriptor and three-dimensional spatial line segments; B3: Use the multimodal hatch corner feature descriptor as a graph node, use the three-dimensional spatial line segment connecting the hatch corner as a graph edge, extract the spatial length and relative angle of the three-dimensional spatial line segment as edge weights, and output the local hatch topology map of the current frame; B4: Retrieve historical hatch topology maps from the global map database, perform subgraph isomorphic matching between the local hatch topology map and the historical hatch topology map, calculate the node feature similarity and edge weight geometric consistency of the matched subgraphs, and output a comprehensive similarity score and a candidate list of matched historical frames. B5: The comprehensive similarity score is used to perform dynamic threshold filtering on the candidate list of matching historical frames. The selected candidate frames are then subjected to geometric verification on the multimodal hatch corner feature descriptor using the random sampling consensus algorithm. The final identified loop closure frame is output and the relative pose transformation is calculated to achieve loop closure detection.

[0009] As a further improvement of the present invention: Optionally, step B1 further includes: The gravity acceleration measurement vector is read from the inertial measurement unit, the rotation matrix that rotates the gravity acceleration measurement vector to the negative Z-axis direction of the world coordinate system is calculated, and the rotation matrix is ​​applied to the three-dimensional coordinates of each point in the original point cloud data to obtain the point cloud data after gravity direction alignment. The gravity-aligned point cloud data is uniformly discretized along the elevation axis, dividing the elevation range into... Equally spaced elevation intervals are used to count the number of point clouds contained in each interval, resulting in an elevation histogram. The maximum positive jump boundary in the number of point clouds between adjacent elevation intervals is detected within this histogram, and the lower boundary elevation value of the elevation interval corresponding to this maximum positive jump boundary is determined as the elevation threshold of the elevation separation plane. The elevation threshold The elevation boundary corresponds to the area between the bulk material stack surface and the ship's bulkhead structure; the elevation value in the point cloud data after aligning the gravity direction is strictly lower than the elevation threshold. Point cloud removal, retaining elevation values ​​not lower than The point cloud was used to obtain the initial point cloud data of the upper hatch. Auxiliary image data is input into a pre-trained semantic segmentation network to obtain pixel-by-pixel semantic category prediction results. The set of pixel coordinates belonging to the hatch boundary category in these pixel-by-pixel semantic category prediction results is extracted and denoted as the hatch boundary pixel. Using the camera intrinsic parameter matrix and the extrinsic parameter transformation matrix from the LiDAR to the camera coordinate system, the image ray direction corresponding to each pixel coordinate in the hatch boundary pixel is transformed to the three-dimensional space of the initial upper hatch point cloud data, preserving the spatial perpendicular distance between the initial upper hatch point cloud data and the image ray direction within the distance tolerance. The point cloud within the range is used to obtain refined point cloud data of the upper hatch; the intermediate feature layer output tensor corresponding to the hatch boundary region of the semantic segmentation network is extracted, and a global average pooling operation is performed on the intermediate feature layer output tensor to compress the output of the global average pooling operation to a dimension of [dimensional value missing]. A fixed-length feature vector is used as a feature descriptor for the hatch image. .

[0010] Optionally, step B2 further includes: For each point in the refined upper hatch point cloud data , with the point Select the neighborhood radius of the center The set of neighboring points within the range; if the number of points in the set of neighboring points is less than the minimum threshold of the number of neighboring points. Then the neighborhood radius is expanded to ,in The neighborhood radius expansion coefficient and A new neighborhood point set is selected until the number of points in the neighborhood point set is not less than [number missing]. Until then; perform singular value decomposition on the coordinate matrix formed by the finally determined neighborhood point set, and determine the singular vector with the smallest singular value as the point. normal vector estimate In the refined point cloud data of the upper hatch, the deviation of the normal vector direction is within the angular tolerance. Points within a given range that are spatially connected are grouped together to form a single planar patch, resulting in a set of planar patches. , Indicates the total number of planar patches; For the planar patches in the set of planar patches that are spatially adjacent and The combined statement and The equations of the planes are used to solve for the spatial intersection line of the two planar patches. This spatial intersection line is then constrained within... and Within the public space coverage area, a set of three-dimensional spatial line segments representing the edges of the hatch coaming is generated; in the set of three-dimensional spatial line segments, the two endpoints of each three-dimensional spatial line segment are compared with the endpoints of all other three-dimensional spatial line segments, and the segments whose mutual spatial distance is less than the corner point merging distance threshold are merged. The endpoints are merged into the same hatch corner point to obtain the set of hatch corner points. , Indicates the total number of hatch corner points; For each hatch corner point in the set of hatch corner points Extract the hatch corner points The three-dimensional coordinate vector is used as the geometric feature vector. The geometric feature vector It is a three-dimensional vector; the geometric feature vector The hatch image feature descriptor output in step B1 Perform vector concatenation to obtain a vector with dimension . The spliced ​​vector is used as the hatch corner point. Multimodal hatch corner feature descriptor ,in Feature descriptor for hatch image Dimensions.

[0011] Optionally, step B3 further includes: Constructing an undirected weighted graph As a partial hatch topology diagram, in which For a set of graph nodes, For the graph edge set, The set of edge weights; the set of hatch corner points output in step B2. Each hatch corner Corresponding multimodal hatch corner feature descriptor Store the graph node set ; The three-dimensional space line segments output in step B2 whose endpoints have been merged into hatch corner points. Store the graph edge set Three-dimensional space line segment Connect graph node set hatch corner Corner of the hatch ; For graph edge sets Each three-dimensional space line segment Extract the three-dimensional space line segments Spatial length With the three-dimensional space line segment Angle relative to the horizontal reference direction Calculate the three-dimensional space line segment Corresponding edge weights The formula for calculating edge weights is: ; in, Represents a line segment in three-dimensional space Edge weights, To connect the hatch corner points Corner of the hatch Three-dimensional space line segments; Represents a line segment in three-dimensional space The spatial length, in meters; Represents the graph edge set The mean of the spatial lengths of all three-dimensional line segments is used to normalize the length components. Represents a line segment in three-dimensional space The angle between the horizontal reference direction and the horizontal reference direction. The range of values ​​is The unit is radians. Pi; This represents the length component weighting coefficient. The range of values ​​for is (0, 1); (1- () represents the directional component weighting coefficient; it calculates all three-dimensional space line segments. Edge weights Store in edge weight set Output an undirected weighted graph. This is a local hatch topology map of the current frame.

[0012] Optionally, step B4 further includes: Local hatch topology map of the current frame To query the map, all historical hatch topology maps corresponding to stored historical frames are extracted from the global map database to form a historical topology map set. The historical topology map set contains the first... The frame history hatch topology diagram is recorded as follows: , , This indicates the total number of historical frames stored in the global map database. , , The first The historical hatch topology graph of a frame contains a set of graph nodes, a set of graph edges, and a set of edge weights; the historical topology graph set is pre-filtered by the number of nodes, and the first frame is selected as the first frame. Frame history hatch topology diagram Number of graph nodes With query graph Number of graph nodes The absolute value of the difference exceeds the node number tolerance. Historical frames are excluded from the historical topology map set to obtain a pre-filtered historical topology map set; For each frame of the historical hatch topology map in the pre-screened historical topology map set With query graph Execute the subgraph isomorphic matching algorithm, in Searching for and Subgraphs isomorphic to the graph structure ,Will graph node set Each node in the middle and The pairing relationships between corresponding nodes in the graph node set are stored in the matching node pair set. ,Will graph edge set The edges of each figure in the middle The pairing relationships between corresponding graph edges in the graph edge set are stored in the matching edge pair set. ; Based on the set of matching node pairs Calculate node feature similarity The node feature similarity For matching the set of node pairs The mean cosine similarity of the multimodal hatch corner feature descriptors for all matching node pairs is calculated using the following formula: ; in, Represents the set of matching node pairs The total number of node pairs; Represents the set of matching node pairs A pair of matching nodes in the middle, For the pair of matching nodes belonging to the query graph Side node index, For the pair of matching nodes belonging to the first Frame history hatch topology diagram The node index on the side; Represents a query graph The middle node index is The multimodal hatch corner feature descriptor corresponding to the node, Indicates the first Frame history hatch topology diagram The middle node index is The multimodal hatch corner feature descriptor corresponding to the node; Based on the set of matching edge pairs Calculate edge weights geometric consistency The calculation formula is: ; in, Represents the set of matching edge pairs The total number of edge pairs in the middle; Represents the set of matching edge pairs A pair of matching edges in the middle, For querying the image Middle Connector Node With nodes Three-dimensional space line segments, For the first Frame history hatch topology diagram Middle Connector Node With nodes Three-dimensional space line segments; Represents a line segment in three-dimensional space Edge weights, Represents a line segment in three-dimensional space Edge weights; Node feature similarity Geometric consistency with edge weights A weighted fusion is performed to obtain a comprehensive similarity score. The calculation formula is: ; in, Represents a query graph With the Frame history hatch topology diagram A comprehensive similarity score between them; The fusion weight coefficients represent the similarity of node features. The range of values ​​for is (0,1); (1- ) represents the fusion weight coefficient for geometric consistency of edge weights; after calculating the comprehensive similarity score for all historical frames in the pre-screened historical topology map set, sort them in descending order of comprehensive similarity score, and output the comprehensive similarity score set and the corresponding candidate list of matching historical frames.

[0013] Optionally, step B5 further includes: The average comprehensive similarity score of all completed matching sessions in the global map database is calculated. with standard deviation Calculate the dynamic filtering threshold using the following formula. : ; in, Indicates the dynamic filtering threshold. This represents the average overall similarity score of all completed matching sessions in the global map database. This represents the standard deviation of the overall similarity score across all completed matching sessions in the global map database. This represents the sensitivity coefficient for filtering; a comprehensive similarity score is retained in the candidate list of matching historical frames. satisfy From the historical frames, we obtain a set of filtered candidate frames; For each candidate frame in the filtered candidate frame set, nearest neighbor feature matching is performed between the multimodal hatch corner feature descriptor subset of the current frame and the multimodal hatch corner feature descriptor subset of the candidate frames to obtain an initial hatch corner matching pair set. A random sampling consensus algorithm is then performed on the initial hatch corner matching pair set. This algorithm randomly samples three hatch corner matching pairs from the initial set and uses these three pairs to solve for the candidate relative pose transformation matrix between the current frame and the candidate frames. Finally, the reprojection error of the initial hatch corner matching pair set under the candidate relative pose transformation matrix is ​​less than the reprojection tolerance. The hatch corner point matching pair is counted as the interior point, after... After several random sampling iterations, the candidate relative pose transformation matrix with the largest number of inliers is selected as the geometric verification result; the inlier ratio is calculated. The interior point ratio The number of interior points corresponding to the geometric verification result Total number of matching pairs in the initial hatch corner point matching pair set The ratio, i.e. ; Number of interior points Exceeding the threshold for the number of interior points And the interior point ratio Exceeding the minimum interior point ratio threshold The candidate frames are determined as loop closure frames that pass geometric verification. The candidate relative pose transformation matrix corresponding to the loop closure frame is used as the relative pose transformation. The final identified loop closure frame and relative pose transformation are output to realize loop closure detection.

[0014] Compared with the prior art, the present invention has at least the following beneficial effects: Traditional loop closure detection methods often fail to match features or generate numerous false matches when dealing with dynamic surfaces formed by bulk material accumulation due to continuous changes in environmental geometry. This invention innovatively utilizes gravity direction information provided by an inertial measurement unit (IMU) to project point cloud data along the elevation axis onto an elevation histogram. The elevation separation plane is determined by detecting the maximum positive jump boundary in point cloud data between adjacent elevation intervals. This method accurately distinguishes between the bulk material accumulation surface and the ship's bulkhead structure. By truncating and eliminating the dynamic bulk material point cloud data below the elevation separation plane, only the stable upper hatch point cloud data is retained. Experiments show that this technology maintains a stable truncation effect under different bulk material accumulation heights and distribution patterns, effectively eliminating interference from the dynamic environment on loop closure detection, enabling the system to maintain reliable positioning performance during continuous bulk material loading and unloading operations.

[0015] Traditional methods often treat the entire point cloud scene as a whole, lacking effective utilization of specific structural features. This invention innovatively extracts planar patches of the ship's bulkhead from refined upper hatch point cloud data. It generates three-dimensional spatial line segments of the hatch coaming by calculating the intersection of adjacent planar patches, and further extracts the intersection points of these line segments as hatch corner points. Based on these geometric elements, this invention constructs a local hatch topology map with multimodal hatch corner point feature descriptors as nodes and three-dimensional spatial line segments as edges, where edge weights comprehensively consider the spatial length and relative angle of the line segments. This topology map representation method not only preserves the geometric topological relationships of the hatch structure but also enhances the matching discrimination ability through the dual constraints of node features and edge weights. In practical applications, this method can effectively distinguish hatch structures of different shapes and sizes, maintaining high recognition accuracy even under partial occlusion or changing viewing angles.

[0016] This invention concatenates the hatch image feature descriptors extracted by a semantic segmentation network with the 3D geometric features of hatch corner points to form a multimodal hatch corner feature descriptor, fully leveraging the complementary advantages of texture semantic information in image data and precise geometric information in point cloud data. In the loop closure detection stage, this invention employs a subgraph isomorphic matching algorithm to perform structural matching between the local hatch topology and historical hatch topology, and calculates a comprehensive similarity score through a weighted fusion of node feature similarity and edge weight geometric consistency. More importantly, this invention introduces a geometric verification mechanism based on a random sampling consensus algorithm, further filtering out truly reliable loop closure frames by calculating the relative pose transformation and inlier ratio of candidate frames. This multi-level verification mechanism effectively reduces the false matching rate and maintains stable detection performance even under conditions of partial environmental changes or noise interference, providing a reliable guarantee for the precise positioning of port automation equipment. Attached Figure Description

[0017] Figure 1This is a flowchart illustrating a loop closure detection method based on point cloud elevation truncation and hatch features according to an embodiment of the present invention. Figure 2 This is a schematic diagram of a multimodal data preprocessing and feature extraction process according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the hatch geometry extraction and feature fusion process according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the process for constructing a partial hatch topology diagram according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the subgraph isomorphic matching and similarity scoring process according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the dynamic threshold screening and geometric verification process according to an embodiment of the present invention. Detailed Implementation

[0018] The present invention will be further described below with reference to the accompanying drawings, but this is not intended to limit the present invention in any way. Any modifications or substitutions made based on the teachings of the present invention shall fall within the protection scope of the present invention.

[0019] Example 1: A loop closure detection method based on point cloud elevation truncation and hatch features, such as... Figure 1 As shown, it includes the following steps: B1: Input raw point cloud data and auxiliary image data, extract the gravity direction aligned raw point cloud data from the inertial measurement unit, project the aligned point cloud data onto the elevation histogram, extract the elevation separation plane corresponding to the bulk material accumulation surface, truncate and remove the dynamic bulk material point cloud data below the elevation separation plane, and obtain the initial upper hatch point cloud data; input the auxiliary image data into the semantic segmentation network to extract hatch boundary pixels, map the hatch boundary pixels to the initial upper hatch point cloud data, and output refined upper hatch point cloud data and hatch image feature descriptors, such as... Figure 2 As shown, it includes: B11: Read the gravity acceleration measurement vector from the inertial measurement unit, calculate the rotation matrix that rotates the gravity acceleration measurement vector to the negative Z-axis direction of the world coordinate system, and apply the rotation matrix to the three-dimensional coordinates of each point in the original point cloud data to obtain the point cloud data after gravity direction alignment; B12: The point cloud data aligned with the gravity direction is uniformly discretized along the elevation axis, dividing the elevation range into... In this embodiment, an elevation histogram is obtained by dividing the elevation into equally spaced elevation intervals and counting the number of point clouds contained in each interval. The elevation range is set to 200, and is automatically determined by the global minimum and global maximum values ​​of the elevation components in the point cloud data aligned with the gravity direction of the current frame. The maximum positive jump boundary of the number of point clouds between adjacent elevation intervals is detected in the elevation histogram. Specifically, let the first... The number of point clouds within each elevation interval is Calculate the first-order difference sequence of adjacent intervals. , take Interval index for obtaining the maximum value The elevation value of the lower boundary of the elevation interval corresponding to the maximum positive jump boundary is determined as the elevation threshold of the elevation separation plane. The elevation threshold The elevation boundary corresponds to the area between the bulk material stack surface and the ship's bulkhead structure; the elevation value in the point cloud data after aligning the gravity direction is strictly lower than the elevation threshold. Point cloud removal, retaining elevation values ​​not lower than The point cloud was used to obtain the initial point cloud data of the upper hatch. B13: Input the auxiliary image data into the pre-trained semantic segmentation network to obtain pixel-by-pixel semantic category prediction results. Extract the set of pixel coordinates belonging to the hatch boundary category from the pixel-by-pixel semantic category prediction results, and denote the set of pixel coordinates as the hatch boundary pixels. In this embodiment, the semantic segmentation network adopts the DeepLabV3+ architecture, and the backbone network is ResNet-50. Pre-training is completed on a dataset of labeled images of ship hatch scenes with different bulk material stacking states. The pre-training dataset contains 5000 labeled images. The labeled range of the hatch boundary category is the area within 5 pixels from the center line of the visible edge of the hatch coaming. The Adam optimizer is used for training, and the initial learning rate is set to... The image ray direction is decayed using a cosine annealing strategy, with 50 training rounds and a batch size of 8. Using the camera intrinsic parameter matrix and the extrinsic parameter transformation matrix from the LiDAR to the camera coordinate system, the image ray direction corresponding to each pixel coordinate in the hatch boundary pixels is transformed to the 3D space of the initial upper hatch point cloud data, preserving the spatial perpendicular distance between the initial upper hatch point cloud data and the image ray direction within the distance tolerance. Point cloud data within the specified range is used to obtain refined point cloud data for the upper hatch; in this embodiment, distance tolerance... Set to 0.15 meters; extract the intermediate feature layer output tensor corresponding to the hatch boundary region of the semantic segmentation network, perform a global average pooling operation on the intermediate feature layer output tensor, and compress the output of the global average pooling operation to a dimension of 0.15 meters. A fixed-length feature vector is used as a feature descriptor for the hatch image. In this embodiment, the intermediate feature layer selects the output feature map of the fourth residual block of the DeepLabV3+ backbone network, which has 2048 channels and a spatial resolution of 1 / 32 of the input image. When set to 256, the global average pooling operation averages the output feature map across all positions in the spatial dimension, outputting a 256-dimensional fixed-length feature vector that is independent of the input image resolution.

[0020] B2: Normal vector estimation is performed on the refined upper hatch point cloud data to extract planar patches of the cabin bulkhead. Intersection calculations are performed on spatially adjacent planar patches to generate three-dimensional spatial line segments of the hatch coaming. The intersection points of these three-dimensional spatial line segments are extracted as hatch corner points. The hatch image feature descriptor and the hatch corner points are then concatenated to output a multimodal hatch corner point feature descriptor and three-dimensional spatial line segments, such as... Figure 3 As shown, it includes: B21: For each point in the refined upper hatch point cloud data , with the point Select the neighborhood radius of the center The neighborhood point set within the range; perform singular value decomposition on the coordinate matrix formed by the finally determined neighborhood point set, and determine the singular vector with the smallest singular value as the point. normal vector estimate In this embodiment, the initial neighborhood radius Set to 0.3 meters; in the refined upper hatch point cloud data, the deviation of the normal vector direction is within the angular tolerance. Points within a given range that are spatially connected are grouped together to form a single planar patch, resulting in a set of planar patches. , This represents the total number of planar patches; in this embodiment, the angle tolerance... The degree is set to 10 degrees, the Euclidean distance threshold of 0.2 meters is used to determine the spatial connectivity, and the region growing algorithm is used for merging planar patches. Optionally, to improve the robustness of normal vector estimation under non-uniform point cloud density conditions, for each point in the refined upper hatch point cloud data... When selecting a neighborhood, if the number of points in the neighborhood set is less than the minimum neighborhood point threshold... Then the neighborhood radius is expanded to ,in The neighborhood radius expansion coefficient and A new neighborhood point set is selected until the number of points in the neighborhood point set is not less than [number missing]. Up to this point; in this embodiment, the minimum neighborhood point threshold. Set to 10, neighborhood radius expansion factor Set to 1.5, with a maximum allowed number of expansions of 5; if the number of points in the neighborhood set is still less than [a certain value] after 5 expansions... Then the point Points marked as sparse and unreliable are excluded from subsequent planar patch merging; when the number of sparse and unreliable points accounts for more than 30% of the total number of points in the refined upper hatch point cloud data of the current frame, a sparse alarm is triggered and the initial confidence weight of the local hatch topology map output in the current frame is set to 0.5 instead of the default value of 1.0. B22: For planar patches that are spatially adjacent in the set of planar patches. and The combined statement and The equations of the planes are used to solve for the spatial intersection line of the two planar patches. This spatial intersection line is then constrained within... and Within the public space coverage area, a set of three-dimensional spatial line segments representing the edges of the hatch coaming is generated; in this embodiment, the planar patches are combined. and After obtaining the plane equation, the spatial intersection direction vector is calculated as follows: ,like Therefore, the two planes are determined to be approximately coplanar, and no three-dimensional space line segments are generated. and They are planar patches and The normal vector; in this embodiment Set to 0.05; in the set of three-dimensional spatial line segments, perform a spatial distance comparison between the two endpoints of each three-dimensional spatial line segment and the endpoints of all other three-dimensional spatial line segments, and merge segments whose mutual spatial distance is less than the corner point merging distance threshold. The endpoints are merged into the same hatch corner point to obtain the set of hatch corner points. , This represents the total number of hatch corner points; in this embodiment, the corner point merging distance threshold is... Set to 0.1 meters, the three-dimensional coordinates of the hatch corner point after merging are taken as the arithmetic mean of the three-dimensional coordinates of all endpoints involved in the merging; B23: For each hatch corner point in the set of hatch corner points Extract the hatch corner points The three-dimensional coordinate vector is used as the geometric feature vector. The geometric feature vector It is a three-dimensional vector; the geometric feature vector The hatch image feature descriptor output in step B1 Perform vector concatenation to obtain a vector with dimension . The spliced ​​vector is used as the hatch corner point. Multimodal hatch corner feature descriptor ,in Feature descriptor for hatch image Dimensions.

[0021] B3: Using the multimodal hatch corner feature descriptors as graph nodes, and the three-dimensional spatial line segments connecting the hatch corners as graph edges, the spatial length and relative angle of the three-dimensional spatial line segments are extracted as edge weights, and the local hatch topology map of the current frame is output, such as... Figure 4 As shown, it includes: B31: Constructing an Undirected Weighted Graph As a partial hatch topology diagram, in which For a set of graph nodes, For the graph edge set, The set of edge weights; the set of hatch corner points output in step B2. Each hatch corner Corresponding multimodal hatch corner feature descriptor Store the graph node set ; The three-dimensional space line segments output in step B2 whose endpoints have been merged into hatch corner points. Store the graph edge set Three-dimensional space line segment Connect graph node set hatch corner Corner of the hatch ; B32: Set of graph edges Each three-dimensional space line segment Extract the three-dimensional space line segments Spatial length With the three-dimensional space line segment Angle relative to the horizontal reference direction Calculate the three-dimensional space line segment Corresponding edge weights The formula for calculating edge weights is: ; in, Represents a line segment in three-dimensional space Edge weights, To connect the hatch corner points Corner of the hatch Three-dimensional space line segments; Represents a line segment in three-dimensional space The spatial length, in meters; Represents the graph edge set The mean of the spatial lengths of all three-dimensional line segments is used to normalize the length components. Represents a line segment in three-dimensional space The angle between the horizontal reference direction and the horizontal reference direction. The range of values ​​is The unit is radians. Pi; This represents the length component weighting coefficient. The range of values ​​for is (0, 1); (1- () represents the directional component weighting coefficient; in this embodiment, Set to 0.6, the horizontal reference direction is defined as the positive X-axis direction in the horizontal plane of the coordinate system aligned with the gravity direction; calculate all three-dimensional space line segments. Edge weights Store in edge weight set Output an undirected weighted graph. This is a local hatch topology map of the current frame.

[0022] B4: Retrieve historical hatch topology maps from the global map database, perform subgraph isomorphic matching between the local hatch topology map and the historical hatch topology map, calculate the node feature similarity and edge weight geometric consistency of the matched subgraphs, and output a comprehensive similarity score and a candidate list of matched historical frames, such as... Figure 5 As shown, it includes: B41: Local hatch topology diagram of the current frame To query the map, all historical hatch topology maps corresponding to stored historical frames are extracted from the global map database to form a historical topology map set. The historical topology map set contains the first... The frame history hatch topology diagram is recorded as follows: , , This indicates the total number of historical frames stored in the global map database. , , The first The historical hatch topology graph of a frame contains a set of graph nodes, a set of graph edges, and a set of edge weights; the historical topology graph set is pre-filtered by the number of nodes, and the first frame is selected as the first frame. Frame history hatch topology diagram Number of graph nodes With query graph Number of graph nodes The absolute value of the difference exceeds the node number tolerance. Historical frames are excluded from the historical topology graph set to obtain a pre-filtered historical topology graph set; in this embodiment, the number of nodes is tolerant. Set to 2; B42: For each frame of historical hatch topology map in the pre-screened historical topology map set... With query graph Execute the subgraph isomorphic matching algorithm, in Searching for and Subgraphs isomorphic to the graph structure ,Will graph node set Each node in the middle and The pairing relationships between corresponding nodes in the graph node set are stored in the matching node pair set. ,Will graph edge set The edges of each figure in the middle The pairing relationships between corresponding graph edges in the graph edge set are stored in the matching edge pair set. In this embodiment, the subgraph isomorphic matching algorithm adopts the VF2 algorithm. When performing graph structure isomorphic search, the search space is pruned with the equality of node degree as a necessary condition for node candidate matching, and the absolute value of the edge weight difference of the matching edge pair does not exceed the edge weight tolerance. Pruning is performed as a necessary condition for edge candidate matching; in this embodiment Set to 0.2; B43: Based on the set of matching node pairs Calculate node feature similarity The node feature similarity For matching the set of node pairs The mean cosine similarity of the multimodal hatch corner feature descriptors for all matching node pairs is calculated using the following formula: ; in, Represents the set of matching node pairs The total number of node pairs; Represents the set of matching node pairs A pair of matching nodes in the middle, For the pair of matching nodes belonging to the query graph Side node index, For the pair of matching nodes belonging to the first Frame history hatch topology diagram The node index on the side; Represents a query graph The middle node index is The multimodal hatch corner feature descriptor corresponding to the node, Indicates the first Frame history hatch topology diagram The middle node index is The multimodal hatch corner feature descriptor corresponding to the node; Based on the set of matching edge pairs Calculate edge weights geometric consistency The calculation formula is: ; in, Represents the set of matching edge pairs The total number of edge pairs in the middle; Represents the set of matching edge pairs A pair of matching edges in the middle, For querying the image Middle Connector Node With nodes Three-dimensional space line segments, For the first Frame history hatch topology diagram Middle Connector Node With nodes Three-dimensional space line segments; Represents a line segment in three-dimensional space Edge weights, Represents a line segment in three-dimensional space Edge weights; B44: Node feature similarity Geometric consistency with edge weights A weighted fusion is performed to obtain a comprehensive similarity score. The calculation formula is: ; in, Represents a query graph With the Frame history hatch topology diagram A comprehensive similarity score between them; The fusion weight coefficients represent the similarity of node features. The range of values ​​for is (0,1); (1- ) represents the fusion weight coefficient for geometric consistency of edge weights; in this embodiment, Set to 0.6; calculate the comprehensive similarity score for all historical frames in the pre-screened historical topology map set, sort them in descending order of comprehensive similarity score, and output the comprehensive similarity score set and the corresponding candidate list of matching historical frames.

[0023] B5: The comprehensive similarity score is used to dynamically threshold the candidate list of matching historical frames. The selected candidate frames are then geometrically verified using a random sampling consensus algorithm on the multimodal hatch corner feature descriptor. The final identified loop closure frame is output, and the relative pose transformation is calculated to achieve loop closure detection. Figure 6 As shown, it includes: B51: Calculate the average comprehensive similarity score of all completed matching sessions in the global map database. with standard deviation Calculate the dynamic filtering threshold using the following formula. : ; in, Indicates the dynamic filtering threshold. This represents the average overall similarity score of all completed matching sessions in the global map database. This represents the standard deviation of the overall similarity score across all completed matching sessions in the global map database. This represents the screening sensitivity coefficient; in this embodiment, Set to 1.5; retain the comprehensive similarity score in the candidate list of matching historical frames. satisfy From the historical frames, we obtain a set of filtered candidate frames; Optionally, to address the issue that the number of completed matching sessions in the global map database is insufficient for reliable statistical analysis during the initial stage of this method's operation, and To address the issue of unstable dynamic filtering thresholds, a fixed threshold mechanism and a robust statistical correction mechanism are introduced during the warm-up phase: when the total number of completed matching sessions in the global map database has not reached the warm-up threshold... When using a fixed initial threshold Replace dynamic filtering threshold Perform filtering; when the total number of matched sessions reaches Then, switch to dynamic filtering threshold. Perform filtering; in this embodiment, the preheating quantity threshold. Set to 50, fix the initial threshold. Set to 0.65; Furthermore, to enhance the robustness of the dynamic screening threshold to outliers in historical scores, the calculation... and First, samples with a comprehensive similarity score of completed matched sessions in the global map database that are below the 5th percentile or above the 95th percentile are removed. A robust mean is then calculated for the remaining samples. With robust standard deviation And correct the dynamic filtering threshold to In this embodiment, the following is adopted: replace Perform the screening; the fixed threshold mechanism in this warm-up phase is combined with robust statistical correction, so that the dynamic screening threshold can maintain reasonable screening sensitivity in the early stage of the task and when there are outliers in the historical scores, reducing missed detections and false detections caused by threshold instability. B52: For each candidate frame in the filtered candidate frame set, perform nearest neighbor feature matching between the multimodal hatch corner feature descriptor subset of the current frame and the multimodal hatch corner feature descriptor subset of the candidate frames to obtain an initial hatch corner matching pair set; in this embodiment, nearest neighbor feature matching is performed based on cosine distance, and a nearest neighbor distance ratio test is applied to filter unreliable matches: let the cosine distance between the current frame feature descriptor and the first nearest neighbor in the candidate frame feature descriptor subset be... The cosine distance of the second nearest neighbor is ,when The matching pair is accepted if it is found, otherwise it is rejected; in this embodiment, the ratio threshold is used. Set to 0.8; B53: Execute a random sampling consensus algorithm on the initial set of hatch corner point matching pairs. The random sampling consensus algorithm randomly samples three pairs of hatch corner point matching pairs from the initial set of hatch corner point matching pairs, and uses the three pairs of hatch corner point matching pairs to solve for the candidate relative pose transformation matrix between the current frame and the candidate frame. Ensure that the reprojection error of the initial set of hatch corner point matching pairs under the candidate relative pose transformation matrix is ​​less than the reprojection tolerance. The hatch corner point matching pair is counted as the interior point, after... After several random sampling iterations, the candidate relative pose transformation matrix with the largest number of inliers is selected as the geometric verification result. B54: Calculate the interior point ratio The interior point ratio The number of interior points corresponding to the geometric verification result Total number of matching pairs in the initial hatch corner point matching pair set The ratio, i.e. ; Number of interior points Exceeding the threshold for the number of interior points And the interior point ratio Exceeding the minimum interior point ratio threshold Candidate frames are determined as loop closure frames that pass geometric verification. The candidate relative pose transformation matrix corresponding to the loop closure frame is used as the relative pose transformation, and the finally identified loop closure frame and relative pose transformation are output to achieve loop closure detection. In this embodiment, the inlier number threshold is used. Set to 6, minimum interior point ratio threshold Set it to 0.5.

[0024] It should be noted that the sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0025] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0026] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A loop closure detection method based on point cloud elevation truncation and hatch features, characterized in that, Includes the following steps: B1: Input raw point cloud data and auxiliary image data, extract the gravity direction of the inertial measurement unit to align the raw point cloud data, project the aligned point cloud data onto the elevation histogram, extract the elevation separation plane corresponding to the bulk material accumulation surface, truncate and remove the dynamic bulk material point cloud data below the elevation separation plane, and obtain the initial upper hatch point cloud data; input the auxiliary image data into the semantic segmentation network to extract the hatch boundary pixels, map the hatch boundary pixels to the initial upper hatch point cloud data, and output the refined upper hatch point cloud data and hatch image feature descriptor; B2: Perform normal vector estimation on the refined upper hatch point cloud data to extract planar patches of the cabin bulkhead, perform intersection calculation on spatially adjacent planar patches to generate three-dimensional spatial line segments of the hatch coaming, extract the intersection points of the three-dimensional spatial line segments as hatch corner points, and perform feature stitching between the hatch image feature descriptor and the hatch corner points to output a multimodal hatch corner point feature descriptor and three-dimensional spatial line segments; B3: Use the multimodal hatch corner feature descriptor as a graph node, use the three-dimensional spatial line segment connecting the hatch corner as a graph edge, extract the spatial length and relative angle of the three-dimensional spatial line segment as edge weights, and output the local hatch topology map of the current frame; B4: Retrieve historical hatch topology maps from the global map database, perform subgraph isomorphic matching between the local hatch topology map and the historical hatch topology map, calculate the node feature similarity and edge weight geometric consistency of the matched subgraphs, and output a comprehensive similarity score and a candidate list of matched historical frames. B5: The comprehensive similarity score is used to perform dynamic threshold filtering on the candidate list of matching historical frames. The selected candidate frames are then subjected to geometric verification on the multimodal hatch corner feature descriptor using the random sampling consensus algorithm. The final identified loop closure frame is output and the relative pose transformation is calculated to achieve loop closure detection.

2. The loop detection method based on point cloud elevation cut-off and hatch feature of claim 1, wherein, Step B1 further includes: The gravity acceleration measurement vector is read from the inertial measurement unit, the rotation matrix that rotates the gravity acceleration measurement vector to the negative Z-axis direction of the world coordinate system is calculated, and the rotation matrix is ​​applied to the three-dimensional coordinates of each point in the original point cloud data to obtain the point cloud data after gravity direction alignment. The gravity-aligned point cloud data is uniformly discretized along the elevation axis, dividing the elevation range into... Equally spaced elevation intervals are used to count the number of point clouds contained in each interval, resulting in an elevation histogram. The maximum positive jump boundary in the number of point clouds between adjacent elevation intervals is detected within this histogram, and the lower boundary elevation value of the elevation interval corresponding to this maximum positive jump boundary is determined as the elevation threshold of the elevation separation plane. The elevation threshold The elevation boundary between the bulk material stack surface and the ship's bulkhead structure; the elevation value in the point cloud data after aligning the gravity direction is lower than the elevation threshold. Point cloud removal, retaining elevation values ​​not lower than The point cloud was obtained, and the initial upper hatch point cloud data was obtained.

3. The loop detection method based on point cloud elevation cut-off and hatch feature of claim 2, wherein, Step B1 further includes: Auxiliary image data is input into a pre-trained semantic segmentation network to obtain pixel-by-pixel semantic category prediction results. The set of pixel coordinates belonging to the hatch boundary category in these pixel-by-pixel semantic category prediction results is extracted and denoted as the hatch boundary pixel. Using the camera intrinsic parameter matrix and the extrinsic parameter transformation matrix from the LiDAR to the camera coordinate system, the image ray direction corresponding to each pixel coordinate in the hatch boundary pixel is transformed to the three-dimensional space of the initial upper hatch point cloud data, preserving the spatial perpendicular distance between the initial upper hatch point cloud data and the image ray direction within the distance tolerance. The point cloud within the range is used to obtain refined point cloud data of the upper hatch; the intermediate feature layer output tensor corresponding to the hatch boundary region of the semantic segmentation network is extracted, and a global average pooling operation is performed on the intermediate feature layer output tensor to compress the output of the global average pooling operation to a dimension of [dimensional value missing]. A fixed-length feature vector is used as a feature descriptor for the hatch image. .

4. The loop detection method based on point cloud elevation cut-off and hatch feature of claim 3, wherein, Step B2 further includes: For each point in the refined upper hatch point cloud data , with the point Select the neighborhood radius of the center The set of neighboring points within the range; if the number of points in the set of neighboring points is less than the minimum threshold of the number of neighboring points. Then the neighborhood radius is expanded to ,in The neighborhood radius expansion coefficient and A new neighborhood point set is selected until the number of points in the neighborhood point set is not less than [number missing]. Until then; perform singular value decomposition on the coordinate matrix formed by the finally determined neighborhood point set, and determine the singular vector with the smallest singular value as the point. normal vector estimate In the refined point cloud data of the upper hatch, the deviation of the normal vector direction is within the angular tolerance. Points within a given range that are spatially connected are grouped together to form a single planar patch, resulting in a set of planar patches. , This indicates the total number of planar facets.

5. The loop closure detection method based on point cloud elevation truncation and hatch features according to claim 4, characterized in that, Step B2 further includes: For the planar patches in the set of planar patches that are spatially adjacent and The combined statement and The equations of the planes are used to solve for the spatial intersection line of the two planar patches. This spatial intersection line is then constrained within... and Within the public space coverage area, a set of three-dimensional spatial line segments representing the edges of the hatch coaming is generated; in the set of three-dimensional spatial line segments, the two endpoints of each three-dimensional spatial line segment are compared with the endpoints of all other three-dimensional spatial line segments, and the segments whose mutual spatial distance is less than the corner point merging distance threshold are merged. The endpoints are merged into the same hatch corner point to obtain the set of hatch corner points. , Indicates the total number of hatch corner points; For each hatch corner point in the set of hatch corner points Extract the hatch corner points The three-dimensional coordinate vector is used as the geometric feature vector. The geometric feature vector It is a three-dimensional vector; the geometric feature vector The hatch image feature descriptor output in step B1 Perform vector concatenation to obtain a vector with dimension . The spliced ​​vector is used as the hatch corner point. Multimodal hatch corner feature descriptor ,in Feature descriptor for hatch image Dimensions.

6. The loop detection method based on point cloud elevation cut-off and hatch feature of claim 5, wherein, Step B3 further includes: Constructing an undirected weighted graph As a partial hatch topology diagram, in which For a set of graph nodes, For the graph edge set, The set of edge weights; the set of hatch corner points output in step B2. Each hatch corner Corresponding multimodal hatch corner feature descriptor Store the graph node set ; The three-dimensional space line segments output in step B2 whose endpoints have been merged into hatch corner points. Store the graph edge set Three-dimensional space line segment Connect graph node set hatch corner Corner of the hatch ; For graph edge sets Each three-dimensional space line segment Extract the three-dimensional spatial line segments Spatial length With the three-dimensional space line segment Angle relative to the horizontal reference direction Calculate the three-dimensional space line segment Corresponding edge weights The formula for calculating edge weights is: ； in, Represents a line segment in three-dimensional space Edge weights, To connect the hatch corner points Corner of the hatch Three-dimensional space line segments; Represents a line segment in three-dimensional space The spatial length, in meters; Represents the graph edge set The mean of the spatial lengths of all three-dimensional line segments is used to normalize the length components. Represents a line segment in three-dimensional space The angle between the horizontal reference direction and the horizontal reference direction. The range of values ​​is The unit is radians. Pi; This represents the length component weighting coefficient. The range of values ​​for is (0, 1); (1- () represents the directional component weighting coefficient; All calculated three-dimensional space line segments Edge weights Store in edge weight set Output an undirected weighted graph. This is a local hatch topology map of the current frame.

7. The loop detection method based on point cloud elevation cut-off and hatch feature of claim 6, wherein, Step B4 further includes: Local hatch topology map of the current frame To query the map, all historical hatch topology maps corresponding to stored historical frames are extracted from the global map database to form a historical topology map set. The historical topology map set contains the first... The frame history hatch topology diagram is recorded as follows: , , This indicates the total number of historical frames stored in the global map database. , , The first The historical hatch topology graph of a frame contains a set of graph nodes, a set of graph edges, and a set of edge weights; the historical topology graph set is pre-filtered by the number of nodes, and the first frame is selected as the first frame. Frame history hatch topology diagram Number of graph nodes With query graph Number of graph nodes The absolute value of the difference exceeds the node number tolerance. Historical frames are excluded from the historical topology map set to obtain a pre-filtered historical topology map set; For each frame of the historical hatch topology map in the pre-screened historical topology map set With query graph Execute the subgraph isomorphic matching algorithm, in Searching for and Subgraphs isomorphic to the graph structure ,Will graph node set Each node in the middle and The pairing relationships between corresponding nodes in the graph node set are stored in the matching node pair set. ,Will graph edge set The edges of each figure in the middle The pairing relationships between corresponding graph edges in the graph edge set are stored in the matching edge pair set. .

8. The loop closure detection method based on point cloud elevation truncation and hatch features according to claim 7, characterized in that, Step B4 further includes: Based on the set of matching node pairs Calculate node feature similarity The node feature similarity For matching the set of node pairs The mean cosine similarity of the multimodal hatch corner feature descriptors for all matching node pairs is calculated using the following formula: ； in, Represents the set of matching node pairs The total number of node pairs; Represents the set of matching node pairs A pair of matching nodes in the middle, For the pair of matching nodes belonging to the query graph Side node index, For the pair of matching nodes belonging to the first Frame history hatch topology diagram The node index on the side; Represents a query graph The middle node index is The multimodal hatch corner feature descriptor corresponding to the node, Indicates the first Frame history hatch topology diagram The middle node index is The multimodal hatch corner feature descriptor corresponding to the node, Represents the L2 norm; Based on matching edge pair sets Computing edge weight geometric consistency , the calculation formula is: ； in, Represents the set of matching edge pairs The total number of edge pairs in the middle; Represents the set of matching edge pairs A pair of matching edges in the middle, For querying the image Middle Connector Node With nodes Three-dimensional space line segments, For the first Frame history hatch topology diagram Middle Connector Node With nodes Three-dimensional space line segments; Represents a line segment in three-dimensional space Edge weights, Represents a line segment in three-dimensional space Edge weights; The node feature similarity is calculated The edge weight geometry consistency is calculated The comprehensive similarity score is obtained by weighted fusion The calculation formula is: ； in, Represents a query graph With the Frame history hatch topology diagram A comprehensive similarity score between them; The fusion weight coefficients represent the similarity of node features. The range of values ​​for is (0,1); (1- ) represents the fusion weight coefficient for geometric consistency of edge weights; after calculating the comprehensive similarity score for all historical frames in the pre-screened historical topology map set, sort them in descending order of comprehensive similarity score, and output the comprehensive similarity score set and the corresponding candidate list of matching historical frames.

9. The loop closure detection method based on point cloud elevation truncation and hatch features according to claim 8, characterized in that, Step B5 further includes: The average comprehensive similarity score of all completed matching sessions in the global map database is calculated. with standard deviation Calculate the dynamic filtering threshold using the following formula. : ； in, Indicates the dynamic filtering threshold. This represents the average overall similarity score of all completed matching sessions in the global map database. This represents the standard deviation of the overall similarity score across all completed matching sessions in the global map database. This represents the filtering sensitivity coefficient; a comprehensive similarity score is retained in the candidate list of matching historical frames. satisfy From the historical frames, we obtain a set of candidate frames after filtering; For each candidate frame in the filtered candidate frame set, nearest neighbor feature matching is performed between the multimodal hatch corner feature descriptor subset of the current frame and the multimodal hatch corner feature descriptor subset of the candidate frames to obtain an initial hatch corner matching pair set. A random sampling consensus algorithm is then performed on the initial hatch corner matching pair set. This algorithm randomly samples three hatch corner matching pairs from the initial set and uses these three pairs to solve for the candidate relative pose transformation matrix between the current frame and the candidate frames. Finally, the reprojection error of the initial hatch corner matching pair set under the candidate relative pose transformation matrix is ​​less than the reprojection tolerance. The hatch corner point matching pair is counted as the interior point, after... After several random sampling iterations, the candidate relative pose transformation matrix with the largest number of inliers is selected as the geometric verification result; the inlier ratio is calculated. The interior point ratio The number of interior points corresponding to the geometric verification result Total number of matching pairs in the initial hatch corner point matching pair set The ratio, i.e. ; Number of interior points Exceeding the threshold for the number of interior points And the interior point ratio Exceeding the minimum interior point ratio threshold The candidate frames are determined as loop closure frames that pass geometric verification. The candidate relative pose transformation matrix corresponding to the loop closure frame is used as the relative pose transformation. The final identified loop closure frame and relative pose transformation are output to realize loop closure detection.

Images