Real-time closed loop detection framework based on FPGA and ultrafast descriptor generator

Abstract

The technical scheme of the present application discloses a real-time closed-loop detection framework and an ultra-fast descriptor generator based on FPGA, characterized by comprising a descriptor generation module and a retrieval verification module; the descriptor generation module comprises a preprocessing submodule, a clustering submodule and a descriptor calculation submodule which are sequentially cascaded; the retrieval verification module comprises a retrieval submodule based on a bag-of-words method and a verification submodule based on global descriptor cross-validation. Compared with the prior art, the clustering search space of the present application is reduced by about 2000 times. The present application eliminates the blocking problem caused by data dependence through a non-blocking architecture, and reduces the descriptor calculation complexity through a cylindrical filter and fast direction calculation. During retrieval, the technical scheme disclosed by the present application only needs 5 local descriptors to maintain high accuracy, realizes a performance improvement of 128.8 times, a single-frame processing time of 16.5 milliseconds, meets the real-time requirement of 50 milliseconds, and the energy consumption is only 21.6 mJ / frame.

Description

Technical Field

[0001] This invention relates to a real-time loop closure detection (LCD) framework in a Light Detection and Ranging (LiDAR) Simultaneous Localization and Mapping (SLAM) system based on a Field Programmable Gate Array (FPGA), and particularly to a highly parallel descriptor generation accelerator and a dual descriptor cross-validation mechanism for intelligent vehicles, belonging to the field of autonomous driving perception and localization technology. Background Technology

[0002] Loop closure detection plays a crucial role in LiDAR-based SLAM systems, correcting accumulated errors in the SLAM algorithm by identifying previously visited locations by intelligent vehicles, thereby improving localization accuracy and map quality in large-scale environments. In autonomous driving applications, considering that advanced high-precision LiDAR scanning frequencies can reach 20Hz, the LCD algorithm needs to simultaneously meet the requirements of high accuracy, high recall, and real-time processing speed of 50 milliseconds per frame. While the state-of-the-art Bag of Words 3D (BoW3D) algorithm achieves high accuracy through unique 3D edge descriptors and efficient bag-of-words retrieval, processing a single frame of 64-line LiDAR data on a desktop processor requires 1620 milliseconds, which cannot meet the demands of real-time applications. Existing technologies suffer from three main problems:

[0003] First, the clustering module is inefficient. Traditional k-means-based FPGA clustering accelerators are inefficient when processing large-scale point cloud data. The square window scanning of the Connected-Component Labeling (CCL) method does not handle vertical features well, and the clustering merging process has data dependency issues, which causes processing bottlenecks. Second, the computational complexity of descriptors is high. Existing methods retain low-quality key points after clustering, and the calculation of the relative direction between key points requires frequent use of inverse trigonometric functions and division operations, which consumes a lot of computing resources. The two-dimensional image acceleration technology based on Oriented Fast and Rotated BRIEF (ORB) is difficult to apply to point cloud descriptor acceleration due to the inherent complexity of three-dimensional spatial relationships. Third, the retrieval speed is slow. Existing retrieval algorithms only use a single type of descriptor, and each descriptor can only provide limited information. Therefore, a large number of descriptors are needed to maintain accuracy. Some algorithms reduce retrieval time by extracting structured information to generate a single global descriptor, but the accuracy decreases due to the loss of detailed information. Summary of the Invention

[0004] The purpose of this invention is to solve the technical problems of existing lidar-based closed-loop detection algorithms, such as large clustering search space, high descriptor computation complexity, and slow retrieval speed.

[0005] To achieve the above objectives, the technical solution of the present invention discloses a real-time closed-loop detection framework and an ultra-fast descriptor generator based on FPGA, characterized in that it includes a descriptor generation module and a retrieval and verification module, wherein the descriptor generation module is deployed on the FPGA programmable logic side and the retrieval and verification module is deployed on the FPGA processing system side. The descriptor generation module consists of a preprocessing submodule, a clustering submodule, and a descriptor computation submodule, which are cascaded together. The preprocessing submodule is used to convert disordered point cloud data into ordered two-dimensional distance images, extract and retain high curvature edge feature points in the two-dimensional distance images, remove the remaining points, and define the retained high curvature edge feature points as feature points. The clustering submodule is used to perform Euclidean distance clustering on feature points to form point clusters; The descriptor calculation submodule is used to filter high-quality clusters from all point clusters, obtain key points, and generate local descriptors, where: If the number of feature points contained in the current point cluster is greater than or equal to the preset number of points threshold, and the shape of the current point cluster satisfies that the aspect ratio rshape is greater than the preset ratio threshold, then the current point cluster is a high-quality cluster. The geometric center of all feature points within a high-quality cluster is extracted as the key point; Based on key points, a fast relative direction calculation method based on lookup tables is used to generate local descriptors; The retrieval and validation module includes a retrieval submodule based on the bag-of-words method and a validation submodule based on global descriptor cross-validation. The retrieval submodule receives the local descriptors generated by the descriptor computation submodule. It adopts the bag-of-words model, converts the local descriptors into word vectors, and retrieves the historical frame with the highest similarity to the current frame from the database as a loop closure candidate frame. The retrieval submodule outputs the ID of the loop closure candidate frame and the relative pose between the current frame and the loop closure candidate frame. The verification submodule is used to generate a global descriptor matrix based on closed-loop candidate frames and relative poses, and to verify the global descriptor matrix using a dual-descriptor cross-validation mechanism.

[0006] Preferably, the preprocessing submodule processes the disordered point cloud data using the following steps: The 3D point cloud data is projected onto a 2D distance image matrix by using a lookup table. Each pixel of the 2D distance image matrix stores the 3D coordinates and distance information of the corresponding point. For each point in the two-dimensional distance image, select several consecutive points that are adjacent to it in the same row of the two-dimensional distance image matrix as a reference set S. Calculate the sum of the distance differences between the current point and all points in set S, and then divide by the number of points in set S to obtain the curvature value c of the current point. All points in the two-dimensional distance image whose curvature value c is within the preset curvature threshold range are marked as high curvature edge feature points. Only the high curvature edge feature points are retained, and the remaining points are discarded.

[0007] Preferably, the clustering submodule adopts a non-blocking vertical-horizontal two-step clustering architecture based on the two-dimensional distance image to achieve clustering and form the point clusters, including a vertical clustering stage and a horizontal clustering and non-blocking label merging stage; The vertical clustering phase includes the following steps: Traverse the feature points in each column of the two-dimensional distance image, and compare the approximate Euclidean distance of each feature point with the k preceding feature points in the same column: If the approximate Euclidean distance between the current feature point and any of the previous k feature points is less than the preset vertical clustering threshold, then the current feature point is merged into the point cluster where the corresponding previous feature point is located, that is, the current feature point is assigned the same label value as the corresponding previous feature point. If the approximate Euclidean distance between the current feature point and all the previous k feature points is greater than the preset vertical clustering threshold, then the current feature point is determined to be the starting point of a new point cluster, and a new globally unique label value is assigned to the current feature point. The horizontal clustering and non-blocking label merging stage employs an input column buffer, a lookup column buffer, and an update column buffer to complete label merging and updating within three consecutive clock cycles, including the following steps: Clock processing cycle one: The input column buffer reads a new column of feature points from the two-dimensional distance image. For each feature point, the approximate Euclidean distance between it and the feature points in the same row and the preceding m columns in the two-dimensional distance image is calculated. If the calculated approximate Euclidean distance is less than the preset horizontal clustering threshold, the equivalence relationship between the current feature point cluster and the corresponding previous feature point cluster is recorded in the merge table. Clock processing cycle two: Perform reverse traversal on the feature points processed in clock processing cycle one stored in the lookup column buffer, and find the minimum label value among all associated labels of the current point cluster according to the equivalence relationship recorded in the merge table. Clock processing cycle three: The feature points processed by clock processing cycle two stored in the update column buffer are updated in a forward direction, and the label values ​​of all feature points are updated to the minimum label value found in clock processing cycle two, thereby completing the merging of point clusters.

[0008] Preferably, for two adjacent feature points in the same column or row of the two-dimensional distance image, the approximate Euclidean distance is adopted. Calculate, where, and These are the horizontal projection distances from the two feature points to the origin of the lidar, respectively, which are the distance information stored in the two-dimensional distance image matrix.

[0009] Preferably, the aspect ratio rshape of the current point cluster is calculated using the following formula:

[0010] Where maxZ, minZ, maxX, minX, maxY, and minY are the maximum and minimum coordinates of all points in the cluster along the Z-axis, X-axis, and Y-axis, respectively.

[0011] Preferably, the descriptor calculation submodule generates the local descriptor using the following steps: The horizontal azimuth angle of any point in the two-dimensional distance image relative to the origin is calculated in advance, and the obtained angle value is stored in a lookup table composed of read-only memory; Based directly on the coordinate indices of any two key points in the two-dimensional distance image, the absolute angle values ​​of the two key points relative to the X-axis are read from the lookup table, the difference between the two absolute angle values ​​is calculated, and the relative angle of the two key points is obtained. If the relative angle is less than 0 degrees or greater than 360 degrees, periodic correction is performed, thereby quickly obtaining the relative direction angle of one key point relative to the other key point. Centered on the current keypoint, the surrounding neighborhood is divided into several sectors based on the calculated relative direction angle. Within each sector, the nearest neighbor keypoint is searched. The distance between the current keypoint and its corresponding neighbor keypoint is used as the descriptor dimension value of the current sector. The values ​​of all sectors are concatenated to generate a local descriptor vector with rotation invariance. If there is no keypoint in the current sector, the descriptor dimension value of the current sector is zero.

[0012] Preferably, the verification submodule generates the global descriptor using the following steps: Using the relative pose output by the retrieval submodule, the point cloud of the current frame is transformed into the coordinate system of the closed-loop candidate frame; The scanning context algorithm is used to divide the transformed point cloud into grids in the horizontal and vertical directions, calculate the maximum height of points in each grid, and construct the global descriptor matrix of the current frame.

[0013] Preferably, the dual-descriptor cross-validation mechanism used in the verification submodule includes preliminary cross-validation and global re-detection and secondary validation, wherein: Preliminary cross-validation: Retrieve the global descriptor of the loop closure candidate frame from the database, calculate the cosine distance between the global descriptor of the loop closure candidate frame and the global descriptor of the current frame. If the distance is less than the preset threshold, the loop closure is determined to be valid and the result is output directly. Otherwise, global re-detection and secondary validation are performed. Global re-detection and secondary verification: The global descriptor of the current frame is used to re-search the entire database to find the most similar historical frame ID as a new candidate frame. If no new candidate frame is found, it is determined that there is no closed loop. If a new candidate frame is found, the second verification step is initiated: the local descriptor of the current frame is used to perform geometric consistency verification with the new candidate frame. If the verification passes, the loop closure result is corrected and output; otherwise, it is determined that there is no loop closure.

[0014] Compared to existing technologies, the clustering search space of this invention is reduced by approximately 2000 times. This invention eliminates the blocking problem caused by data dependencies through a non-blocking architecture and reduces descriptor computational complexity through cylindrical filters and fast direction calculations. During retrieval, the technical solution disclosed in this invention requires only 5 local descriptors to maintain high accuracy, achieving a 128.8-fold performance improvement, a single-frame processing time of 16.5 milliseconds, meeting the 50-millisecond real-time requirement, and consuming only 21.6 mJ / frame. Attached Figure Description

[0015] Figure 1 The following is a schematic diagram illustrating the application scenarios and process of the closed-loop detection framework proposed in this invention: (a) shows the role of closed-loop detection in the SLAM system, which constructs constraints by identifying the location of vehicle revisits (Loop) to correct the accumulated localization error; (b) shows the three main processing steps of this invention: Step 1, descriptor generation (including point cloud preprocessing, clustering, and descriptor calculation), Step 2, retrieval (based on the bag-of-words model), and Step 3, database update.

[0016] Figure 2This diagram illustrates the rotating scanning imaging principle of a multi-line lidar (LiDAR), where: (a) is a side view showing N vertically arranged laser emitters inside the lidar, simultaneously emitting laser beams to measure distance; (b) is a top view showing the lidar rotating at a constant angular velocity, completing one vertical scan with each angular increment, ultimately forming a frame of 360-degree point cloud data. The clustering algorithm of this invention utilizes this physical scanning characteristic.

[0017] Figure 3 This is a diagram of the overall system architecture of the FPGA-based real-time loop closure detection framework of the present invention. In the diagram: the left side is the FPGA programmable logic (PL) end, which deploys a descriptor generation accelerator responsible for converting the raw point cloud data stream into compact local descriptors in real time; the right side is the FPGA processing system (PS) end, which deploys a retrieval and verification module responsible for receiving descriptors for loop closure retrieval, using global descriptors for cross-validation, and finally outputting the loop closure detection result (ID and pose). Figure 4 The diagram shows the internal hardware architecture of the descriptor generation accelerator, which consists of three cascaded sub-modules: 1. Preprocessing accelerator: containing a range image projection unit and an edge selector, which extracts features using a curvature buffer; 2. Clustering accelerator: containing a vertical clusterer and a horizontal clusterer, which are connected by a non-blocking architecture; 3. Descriptor computation accelerator: containing a key point calculator and a descriptor organizer, which generates the final descriptor using a lookup table (LUT).

[0018] Figure 5 This diagram illustrates the label initialization logic for the vertical clustering stage, showing a column of data representing distances to the image. The algorithm iterates from top to bottom, comparing the distance of the current point with the point above it in the same column. If the distance is short, the label of the point above it is inherited; if the distance is long, a new label (Init Label) is initialized.

[0019] Figure 6 This diagram illustrates the label merging logic during the horizontal clustering stage, showing the row scanning process of the image. When a point in the current column is found to be very close to a point in the previous column, it indicates that they belong to the same object. At this point, the two different labels (e.g., Label A and Label B) need to be merged and recorded, indicating that they actually represent the same cluster.

[0020] Figure 7 The diagram illustrates the comparison of clustering search spaces for different algorithms. (a) shows the search space of the BoW3D algorithm, which requires a fully connected search across the entire sector, resulting in a huge computational burden. (b) shows the search space of existing FPGA accelerators, which typically use a rectangular window (e.g., 7×7) for searching. (c) shows the search space of the method of this invention, which only requires searching the predecessor points in the same column and the predecessor points in the same row. The search space is reduced by about 2000 times compared to BoW3D, greatly reducing the consumption of computational resources.

[0021] Figure 8 This diagram illustrates the data flow of a non-blocking label merging architecture, showing the pipelined operation of three column buffers: Column 1 (Input Column): Receives input data and determines whether merging is needed; Column 2 (Finding Column): Performs a reverse scan to find the root label (minimum label) of the cluster; Column 3 (Update Column): Performs a forward scan to update the labels of all points to the root label. This design allows the label merging operation to be completed over three cycles, avoiding pipeline blockage in traditional connected component labeling algorithms.

[0022] Figure 9 The diagram illustrates the principles of keypoint filtering and fast descriptor calculation. (a) shows the working principle of the column-shape filter, which eliminates flat or irregular clusters by calculating the aspect ratio of the clusters, retaining only high-quality columnar features such as tree trunks and lampposts as keypoints; (b) demonstrates a fast relative orientation calculation method, utilizing pre-stored angle values... Subtraction directly yields the relative direction, replacing complex trigonometric function operations. Detailed Implementation

[0023] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0024] This invention discloses a real-time closed-loop detection framework and an ultra-fast descriptor generator based on a field-programmable gate array (FPGA), including a descriptor generation module and a retrieval and verification module.

[0025] The descriptor generation module is deployed on the FPGA programmable logic side and includes a preprocessing submodule, a clustering submodule, and a descriptor calculation submodule that are cascaded in sequence.

[0026] The preprocessing submodule is used to transform unordered point cloud data into an ordered data structure and extract features, specifically including the following steps: Range Image Projection: Utilizing the physical characteristics of LiDAR (vertical and horizontal angular resolution), the 3D point cloud data is projected into a 2D range image matrix through a lookup table (LUT). Each pixel in the matrix stores the 3D coordinates and distance information of the corresponding point.

[0027] Curvature calculation: For each point in the 2D distance image, select a series of consecutive points that are adjacent to it in the same row of the 2D distance image matrix as a reference set S. For example, select 5 points before and after it, for a total of 10 points as the reference set S. Calculate the sum of the distance differences between the current point and all points in set S, and divide the sum by the number of points in set S to obtain the curvature value c of the current point.

[0028] Edge feature point extraction: Set a curvature threshold range (e.g., greater than 10 and less than 20000), mark all points with curvature values ​​within this range as high curvature edge feature points, retain only these high curvature edge feature points for subsequent processing, and discard the remaining points as planar points or noise.

[0029] The clustering submodule is used to perform Euclidean distance clustering on edge feature points to form point clusters.

[0030] In this embodiment of the invention, the clustering submodule adopts a non-blocking vertical-horizontal two-step clustering architecture based on distance images to achieve clustering. This architecture solves the data dependency problem through pipelined parallel processing, specifically including the following steps: Approximate Euclidean Distance Calculation: During clustering, the distribution characteristics of LiDAR point clouds on the distance image are utilized to replace complex square root calculations with an approximation method. For two adjacent points in the same column or row of the distance image, the simplified formula for calculating their 3D spatial distance is: ,in, and These are the horizontal projection distances from the two points to the origin of the lidar, which are the distance information stored when projecting the distance image.

[0031] Vertical clustering stage (intra-column clustering): This stage iterates through each column of edge feature points in the image. Assuming we are currently processing the nth edge feature point, we compare its distance to the k preceding edge feature points in the same column (e.g., the first 3 points, i.e., n-1, n-2, n-3). If the approximate Euclidean distance between the current edge feature point and any previous edge feature point is less than the preset vertical clustering threshold, then the current edge feature point is merged into the cluster where the previous edge feature point is located, that is, the current point is assigned the same label value Label ID as the previous edge feature point. If the approximate Euclidean distance between the current edge feature point and all preceding edge feature points is greater than the threshold, then the current edge feature point is determined to be the starting point of a new cluster, and a new globally unique label value is assigned to it.

[0032] Horizontal clustering and non-blocking tag merging stage (inter-row merging): To achieve streaming processing in hardware and eliminate waiting latency, a three-column buffer structure (Input Column, Finding Column, and Update Column) is used to complete tag merging and updating within three consecutive clock cycles. Cycle 1 (Input and Initial Association): Read in a new column of data (input column). For each edge feature point in this input column, search for edge feature points within the same row and the preceding m columns (e.g., the first 3 columns) of the image. If the calculated approximate Euclidean distance is less than the preset horizontal clustering threshold, it indicates that the cluster to which the current edge feature point belongs is spatially connected to the cluster to which the preceding edge feature point belongs, and they need to be merged. In this case, the equivalence relationship between the different label values ​​corresponding to the two clusters is recorded in the Merge Table. Cycle 2 (Minimum Label Search): The data processed in Cycle 1 (search column) is traversed in reverse. Using the Union-Find concept, the minimum label value (Root Label) among all associated labels of the current cluster is found based on the equivalence relationship of the records in the merged table. This minimum label value is used as the final unified identifier of the current cluster.

[0033] Cycle 3 (Merge Table Update): Perform a forward update on the data (update column). Update the label values ​​of all edge feature points in this column to the minimum label value found in Cycle 2, thus completing the cluster merging.

[0034] Through the above steps, the originally discrete edge feature points are clustered into point clusters with independent labels, and the time complexity of the algorithm is reduced from... Reduce to .

[0035] The descriptor calculation submodule is used to filter high-quality clusters from all point clusters, calculate key points, and generate local descriptors.

[0036] In this embodiment of the invention, the descriptor calculation submodule uses a cylindrical filter to screen high-quality clusters and calculates key points. The screening criteria are: the number of feature points contained in the cluster is greater than or equal to a preset point threshold (e.g., 10), and the shape of the cluster satisfies that the aspect ratio rshape is greater than a preset ratio threshold (e.g., 1.5).

[0037] The formula for calculating the aspect ratio is rshape:

[0038] Where maxZ, minZ, maxX, minX, maxY, and minY are the maximum and minimum coordinates of all points in the cluster along the Z-axis, X-axis, and Y-axis, respectively. Clusters that meet the above conditions are identified as column-like objects (such as tree trunks or lampposts), and the geometric centers of all their points are extracted as keypoints.

[0039] In this embodiment of the invention, the descriptor calculation submodule generates local descriptors using a fast relative direction calculation method based on lookup tables, specifically including the following steps: Pre-calculated angle lookup table: During the system initialization phase, the horizontal azimuth angle of any point in the image relative to the origin is pre-calculated, and these angle values ​​are stored in a lookup table (LUT) composed of read-only memory (ROM).

[0040] Relative Angle Acquisition: When calculating the relative direction between two key points (let's call them points A and B), instead of performing complex arctangent function (atan2) calculations, the absolute angle values ​​of points A and B relative to the X-axis are directly read from the lookup table above, based on their coordinate indices in the distance image. and .

[0041] Angle difference calculation: Calculate the relative angle between two points using simple subtraction. If the result is less than 0 degrees or greater than 360 degrees, a periodic correction (adding or subtracting 360 degrees) is performed to quickly obtain the direction angle of point B relative to point A.

[0042] Descriptor generation: Centered on the current keypoint, the surrounding neighborhood is divided into several sectors (e.g., 120 sectors) based on the calculated relative direction angles. Within each sector, the nearest neighbor keypoint is searched, and the distance between the current point and that neighbor is used as the descriptor dimension value for that sector. If there are no keypoints in a sector, that dimension is set to zero. Finally, the values ​​from all sectors are concatenated to generate a rotation-invariant local descriptor vector.

[0043] The retrieval and verification module is deployed on the FPGA processing system and includes a retrieval submodule based on the bag-of-words method and a verification submodule based on global descriptor cross-validation.

[0044] The retrieval submodule receives the local descriptors generated by the descriptor computation submodule. It employs a Bag of Words (BoW) model, converting the local descriptors into word vectors and retrieving the historical frame with the highest similarity to the current frame from the database—the loop closure candidate frame. The retrieval submodule outputs the loop closure candidate frame's ID (LoopID) and the relative pose between the current frame and the loop closure candidate frame.

[0045] The verification submodule is used to eliminate false positives, and it adopts a dual-descriptor cross-validation mechanism. The specific steps are as follows: Global descriptor generation: First, using the relative pose output by the retrieval submodule, the current frame point cloud is transformed into the coordinate system of the closed-loop candidate frame. Then, using the Scan Context algorithm, the transformed point cloud is divided into grids in the horizontal and vertical directions, the maximum height of points within each grid is calculated, and the global descriptor matrix of the current frame is constructed.

[0046] Preliminary cross-validation: Retrieve the global descriptors of candidate frames from the database and calculate the cosine distance between them and the global descriptor of the current frame. If the distance is less than a preset threshold, the loop closure is considered valid, and the result is output directly.

[0047] Global Re-detection and Secondary Verification: If the initial verification fails (distance greater than the threshold), it indicates that the local descriptor retrieval result is incorrect. In this case, the global descriptor of the current frame is directly used to perform a re-detection across the entire database to find the most similar historical frame ID (NewID). If a new candidate frame is found, the secondary verification stage begins: the local descriptor of the current frame is used to perform geometric consistency verification with the new candidate frame (NewID) (e.g., calculating the Euclidean distance between relative poses). If the verification passes (e.g., pose offset is within a reasonable range), the loop closure result is corrected and output; otherwise, it is determined that there is no loop closure.

[0048] Example 1: Hardware platform configuration scheme for a real-time closed-loop detection system based on FPGA The AMD Xilinx ZCU102 development board was used as the hardware platform, equipped with an XCZU9EG-2FFVB1156 FPGA chip. The descriptor generation accelerator was deployed on the FPGA programmable logic side, operating at a frequency of 150MHz. The retrieval and verification module was deployed on the FPGA processing system side, utilizing a quad-core ARM Cortex-A53 processor and 2GB of DDR4 memory. Raw point cloud data was transferred between the programmable logic side and the processing system side via direct memory access. In terms of resource utilization, the accelerator occupied 636 BRAM blocks (69.7% utilization), 77 DSPs (3.1% utilization), 163,958 triggers (29.9% utilization), and 129,480 lookup tables (47.2% utilization).

[0049] Example 2: Parameter Configuration Scheme for a Closed-Loop Detection Framework Clustering submodule parameters: Vertical clustering distance threshold set to 0.06 meters, horizontal clustering distance threshold set to 0.1 meters, clustering window size set to 4. Descriptor calculation submodule parameters: Cylindrical filter point threshold set to 10 points, aspect ratio threshold set to 1.5, curvature calculation using 5 consecutive points before and after, points with curvature values ​​between 10 and 20000 are identified as edge feature points. Retrieval and verification module parameters: Number of retrieval descriptors set to 5, local descriptor validity judgment threshold set to 10, secondary verification pose Euclidean norm threshold set to 3 meters.

[0050] Example 3: Validation scheme of a loop closure detection framework on a public dataset This embodiment was validated on sequences 00, 02, 05, 06, 07, and 08 of the KITTI dataset (acquired by a Velodyne HDL-64E 64-line LiDAR) and sequences DCC 01 and KAIST 01 of the MulRan dataset (acquired by an OS1 64-line LiDAR). In terms of detection accuracy, the invention achieved a maximum F1 score of 98.6% on the KITTI 00 sequence, with an average maximum F1 score of 89.4% across the eight test sequences. Regarding processing speed, the descriptor generation time averaged 3.6 milliseconds, and the total processing time per frame averaged 16.5 milliseconds, meeting the 50-millisecond real-time requirement, representing a 128.8-fold performance improvement compared to the traditional BoW3D descriptor generation method. In terms of energy consumption, the FPGA platform's descriptor generation energy consumption was only 21.6 mJ / frame, a 99.2% reduction compared to desktop platforms and a 98.6% reduction compared to embedded ARM platforms.

Claims

1. A real-time closed-loop detection framework and ultra-fast descriptor generator based on FPGA, characterized in that, It includes a descriptor generation module and a retrieval and verification module, wherein the descriptor generation module is deployed on the FPGA programmable logic side and the retrieval and verification module is deployed on the FPGA processing system side; The descriptor generation module consists of a preprocessing submodule, a clustering submodule, and a descriptor computation submodule, which are cascaded together. The preprocessing submodule is used to convert disordered point cloud data into ordered two-dimensional distance images, extract and retain high curvature edge feature points in the two-dimensional distance images, remove the remaining points, and define the retained high curvature edge feature points as feature points. The clustering submodule is used to perform Euclidean distance clustering on feature points to form point clusters; The descriptor calculation submodule is used to filter high-quality clusters from all point clusters, obtain key points, and generate local descriptors, where: If the number of feature points contained in the current point cluster is greater than or equal to the preset number of points threshold, and the shape of the current point cluster satisfies that the aspect ratio rshape is greater than the preset ratio threshold, then the current point cluster is a high-quality cluster. The geometric center of all feature points within a high-quality cluster is extracted as the key point; Based on key points, a fast relative direction calculation method based on lookup tables is used to generate local descriptors; The retrieval and validation module includes a retrieval submodule based on the bag-of-words method and a validation submodule based on global descriptor cross-validation. The retrieval submodule receives the local descriptors generated by the descriptor computation submodule. It adopts the bag-of-words model, converts the local descriptors into word vectors, and retrieves the historical frame with the highest similarity to the current frame from the database as a loop closure candidate frame. The retrieval submodule outputs the ID of the loop closure candidate frame and the relative pose between the current frame and the loop closure candidate frame. The verification submodule is used to generate a global descriptor matrix based on closed-loop candidate frames and relative poses, and to verify the global descriptor matrix using a dual-descriptor cross-validation mechanism.

2. The FPGA-based real-time closed-loop detection framework and ultra-fast descriptor generator as described in claim 1, characterized in that, The preprocessing submodule processes the unordered point cloud data using the following steps: The 3D point cloud data is projected onto a 2D distance image matrix by using a lookup table. Each pixel of the 2D distance image matrix stores the 3D coordinates and distance information of the corresponding point. For each point in the two-dimensional distance image, select several consecutive points that are adjacent to it in the same row of the two-dimensional distance image matrix as a reference set S. Calculate the sum of the distance differences between the current point and all points in set S, and then divide by the number of points in set S to obtain the curvature value c of the current point. All points in the two-dimensional distance image whose curvature value c is within the preset curvature threshold range are marked as high curvature edge feature points. Only the high curvature edge feature points are retained, and the remaining points are discarded.

3. The FPGA-based real-time closed-loop detection framework and ultra-fast descriptor generator as described in claim 1, characterized in that, The clustering submodule uses a non-blocking vertical-horizontal two-step clustering architecture based on the two-dimensional distance image to achieve clustering and form the point clusters, including a vertical clustering stage and a horizontal clustering and non-blocking label merging stage. The vertical clustering phase includes the following steps: Traverse the feature points in each column of the two-dimensional distance image, and compare the approximate Euclidean distance of each feature point with the k preceding feature points in the same column: If the approximate Euclidean distance between the current feature point and any of the previous k feature points is less than the preset vertical clustering threshold, then the current feature point is merged into the point cluster where the corresponding previous feature point is located, that is, the current feature point is assigned the same label value as the corresponding previous feature point. If the approximate Euclidean distance between the current feature point and all the previous k feature points is greater than the preset vertical clustering threshold, then the current feature point is determined to be the starting point of a new point cluster, and a new globally unique label value is assigned to the current feature point. The horizontal clustering and non-blocking label merging stage employs an input column buffer, a lookup column buffer, and an update column buffer to complete label merging and updating within three consecutive clock cycles, including the following steps: Clock processing cycle one: The input column buffer reads a new column of feature points from the two-dimensional distance image. For each feature point, the approximate Euclidean distance between it and the feature points in the same row and the preceding m columns in the two-dimensional distance image is calculated. If the calculated approximate Euclidean distance is less than the preset horizontal clustering threshold, the equivalence relationship between the current feature point cluster and the corresponding previous feature point cluster is recorded in the merge table. Clock processing cycle two: Perform reverse traversal on the feature points processed in clock processing cycle one stored in the lookup column buffer, and find the minimum label value among all associated labels of the current point cluster according to the equivalence relationship recorded in the merge table. Clock processing cycle three: The feature points processed by clock processing cycle two stored in the update column buffer are updated in a forward direction, and the label values ​​of all feature points are updated to the minimum label value found in clock processing cycle two, thereby completing the merging of point clusters.

4. The FPGA-based real-time closed-loop detection framework and ultra-fast descriptor generator as described in claim 3, characterized in that, For two adjacent feature points in the same column or row of the two-dimensional distance image, the approximate Euclidean distance is adopted. Calculate, where, and These are the horizontal projection distances from the two feature points to the origin of the lidar, respectively, which are the distance information stored in the two-dimensional distance image matrix.

5. The FPGA-based real-time closed-loop detection framework and ultra-fast descriptor generator as described in claim 1, characterized in that, The aspect ratio rshape of the current point cluster is calculated using the following formula: Where maxZ, minZ, maxX, minX, maxY, and minY are the maximum and minimum coordinates of all points in the cluster along the Z-axis, X-axis, and Y-axis, respectively.

6. The FPGA-based real-time closed-loop detection framework and ultra-fast descriptor generator as described in claim 1, characterized in that, The descriptor calculation submodule generates the local descriptor using the following steps: The horizontal azimuth angle of any point in the two-dimensional distance image relative to the origin is calculated in advance, and the obtained angle value is stored in a lookup table composed of read-only memory; Based directly on the coordinate indices of any two key points in the two-dimensional distance image, the absolute angle values ​​of the two key points relative to the X-axis are read from the lookup table, the difference between the two absolute angle values ​​is calculated, and the relative angle of the two key points is obtained. If the relative angle is less than 0 degrees or greater than 360 degrees, periodic correction is performed, thereby quickly obtaining the relative direction angle of one key point relative to the other key point. Centered on the current keypoint, the surrounding neighborhood is divided into several sectors based on the calculated relative direction angle. Within each sector, the nearest neighbor keypoint is searched. The distance between the current keypoint and its corresponding neighbor keypoint is used as the descriptor dimension value of the current sector. The values ​​of all sectors are concatenated to generate a local descriptor vector with rotation invariance. If there is no keypoint in the current sector, the descriptor dimension value of the current sector is zero.

7. The FPGA-based real-time closed-loop detection framework and ultra-fast descriptor generator as described in claim 1, characterized in that, The verification submodule generates the global descriptor using the following steps: Using the relative pose output by the retrieval submodule, the point cloud of the current frame is transformed into the coordinate system of the closed-loop candidate frame; The scanning context algorithm is used to divide the transformed point cloud into grids in the horizontal and vertical directions, calculate the maximum height of points in each grid, and construct the global descriptor matrix of the current frame.

8. The FPGA-based real-time closed-loop detection framework and ultra-fast descriptor generator as described in claim 1, characterized in that, The dual-descriptor cross-validation mechanism employed by the verification submodule includes preliminary cross-validation and global re-detection and secondary validation, wherein: Preliminary cross-validation: Retrieve the global descriptor of the loop closure candidate frame from the database, calculate the cosine distance between the global descriptor of the loop closure candidate frame and the global descriptor of the current frame. If the distance is less than the preset threshold, the loop closure is determined to be valid and the result is output directly. Otherwise, global re-detection and secondary validation are performed. Global re-detection and secondary verification: The global descriptor of the current frame is used to re-search the entire database to find the most similar historical frame ID as a new candidate frame. If no new candidate frame is found, it is determined that there is no closed loop. If a new candidate frame is found, the second verification step is initiated: the local descriptor of the current frame is used to perform geometric consistency verification with the new candidate frame. If the verification passes, the loop closure result is corrected and output; otherwise, it is determined that there is no loop closure.

Images