A SLAM method based on multi-sensor fusion accelerated by GPU
By adopting GPU-accelerated lidar/IMU tightly coupled front-end, factor graph optimization and visual loopback detection module in SLAM technology, the contradiction between calculation speed and real-time performance of multi-sensor fusion SLAM technology is solved, and the SLAM technology is improved Real-time and accuracy, especially loopback detection capabilities in scenarios where features are not obvious or repeated.
Patent Information
- Application Number
- CN202210459576.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-28
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-04-28
AI Technical Summary
The existing multi-sensor fusion SLAM technology has a contradiction between calculation speed and real-time performance when processing large data volumes, and cannot accelerate front-end data matching and back-end pose optimization at the same time, especially in cases where the characteristics are not obvious or repeated. insufficient.
Using the GPU acceleration-based lidar/IMU tightly coupled front-end odometer module, the factor graph-based back-end optimization module and the GPU acceleration-based visual loop detection module, the point cloud matching, factor graph optimization and visual feature extraction are implemented. Efficient processing and fusion of sensor data.
Improve the real-time and accuracy of SLAM technology, enhance loopback detection capabilities, especially in scenarios with inconspicuous features or repeated features, realize the effective combination of lidar, IMU and vision sensors, and improve point cloud matching accuracy and Overall performance of the system.
Smart Images

Figure CN115143958B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of robot real-time positioning and map construction, and more specifically to a SLAM method based on GPU acceleration and multi-sensor fusion. Background Art
[0002] Simultaneous Localization and Mapping (SLAM) is a key technology for robots to achieve autonomous navigation. In recent years, the SLAM method of multi-sensor fusion has been continuously concerned and studied, and has been widely used. Although multi-sensor fusion improves the stability, accuracy and robustness of SLAM technology, it also means that with the increase in data volume, more powerful computer computing power is required. However, the fact is that there is a disconnect between the sensor function and the actual information processing function of the system. In recent years, the further development of graphics and image processing technology has put higher requirements on computing speed. Graphics processors (GPUs) have gradually attracted attention from various technical fields with their powerful computing power. The contradiction between the real-time nature of SLAM technology and the huge amount of data needs to be resolved urgently, and the development of GPUs provides a point of convergence for this.
[0003] SLAM technology is very dependent on the accuracy and efficiency of state estimation, so front-end data matching is crucial. If this part can be completed accurately and efficiently, it will provide important guarantees for the accuracy of the map and the overall real-time performance. In addition to the front-end, the pose optimization and loop detection of the back-end also play a very important role, especially in scenes with unclear features or high repetition. Not only is there a high requirement for the completeness and richness of sensor-captured information, but the accuracy and efficiency of loop detection are also worth further improving. The ability to capture loop information in a timely manner and update accurate state information is of great significance to SLAM. Existing technologies can only achieve acceleration in one aspect, but fail to accelerate both parts at the same time, and the fusion method also has shortcomings. The fusion method based on factor graph optimization has a more accurate robot state and is the current research trend of multi-sensor fusion. Therefore, how to accelerate the running speed of these two aspects of SLAM technology based on multi-sensor fusion of factor graph optimization to meet higher-level application requirements is a direction worth studying. Summary of the invention
[0004] In order to enable the SLAM technology to be completed accurately and in real time, the present invention provides a multi-sensor fusion SLAM method based on GPU acceleration, which is characterized by comprising a GPU-accelerated lidar / IMU tightly coupled front-end odometer module, a factor graph-based back-end optimization module, a point cloud map creation module, and a GPU-accelerated visual loop detection module:
[0005] The GPU-accelerated laser radar / IMU tightly coupled front-end odometer module, wherein the GPU-accelerated point cloud matching steps are as follows:
[0006] Step 1: Downsampling: The host (CPU) reads the lidar data and performs voxel grid downsampling on the lidar point cloud data, dividing all point clouds into different voxels and removing points that are far from the grid and obviously wrong.
[0007] Step 2: Motion compensation: The host (CPU) uses IMU motion prediction to perform motion compensation on the point cloud distortion caused by the motion in the radar frame, and projects each point cloud to the start time of the lidar frame;
[0008] Step 3: Voxelize the target point: The host (CPU) sets the point cloud data of the previous frame as the target point cloud and stores it in the container. Use KD-tree to find the point cloud within the radius r of the target point and store it in the container. Copy the target point cloud from the host (CPU) to the device (GPU). Each target point cloud corresponds to a voxel index, initialize the mean and covariance (skip this step if not initialized).
[0009] Step 4: Voxelize source points: The host (CPU) sets the current frame point cloud data as the source point cloud and stores it in a container. Use KD-tree to find the point cloud within the radius r of the target point and store it in the container. Copy the source point cloud from the host (CPU) to the device (GPU). Each source point cloud corresponds to a voxel index, and initializes the mean and covariance.
[0010] Step 5: Calculate the mean and covariance: The host side (CPU) designs the kernel function according to the device conditions and the number of point clouds, sets the kernel width, and the number of threads equals the number of voxels. The driver device side (GPU) models the point cloud into a Gaussian distribution according to the index and calculates the mean and covariance of the source point cloud and the target point cloud.
[0011] Step 6: Calculate relative transformation: The device side (GPU) calculates the correspondence between voxels, uses maximum likelihood estimation to obtain the maximum a posteriori probability, uses Gauss-Newton method to iteratively optimize, uses the number of points in the voxel to weight the error term, and obtains the information matrix according to the covariance matrix of the voxel. Iterate until the target cost function converges to obtain the corresponding transformation matrix T, and transmit the transformation matrix T to the host side (CPU);
[0012] Step 7: Update point cloud: Set the current frame point cloud as the target point cloud and wait for the next calculation.
[0013] The factor graph-based backend optimization module constructs a sliding window including IMU pre-integration factor, lidar matching cost factor, and visual loopback factor. When a new sensor state is added, the factor graph is optimized using a fixed lag smoothing method and the updated pose state is fed back to the GPU-accelerated lidar / IMU tightly coupled front-end odometer module for correction, and the marginalized keyframes are used to build a point cloud map.
[0014] The GPU-accelerated visual loop detection module first performs GPU-accelerated FAST corner feature extraction, followed by dictionary query, similarity scoring, group matching, time rationality check, and structural consistency check. After the loop is determined, the loop factor is added to the factor graph-based backend optimization module to perform a global optimization.
[0015] Optionally, in the factor graph-based backend optimization module, the size of the sensor state nodes in the constructed sliding window factor graph is fixed to 20. The constraints between the optimization factors are: the constraint between the marginalized key frame and the latest frame is a unary edge, and other factors such as IMU pre-integration factor, lidar matching cost factor, and visual loop factor are binary edges.
[0016] Optionally, in the GPU-accelerated visual loop detection module, the steps of GPU-accelerated FAST corner feature extraction are as follows:
[0017] Step a: Obtain image pyramid: The host (CPU) performs subsampling on the camera input image to obtain the image pyramid;
[0018] Step b: Divide into rectangular units: The host side (CPU) divides each resolution image into rectangular units with fixed width and height, and copies them to the device side (GPU). Each rectangular block corresponds to a thread block.
[0019] Step c: Determine the region of interest: The device (GPU) calculates the FAST corner points in the warp and compares the brightness of the selected pixel with enough pixels in its surrounding area to determine the region of interest (ROI).
[0020] Step d: Corner response function evaluation: For each pixel in the region of interest (ROI) in each rectangular block, the device (GPU) first performs a coarse corner response function (CCRF) evaluation and then a corner response function (CRF) evaluation;
[0021] Step e: Non-maximum suppression: The device (GPU) divides the corner response map into a regular cell grid, assigns a thread block to each cell, uses multiple warps to process one cell, and performs non-maximum suppression;
[0022] Step f: CPU reads FAST corner points: All warps in the thread block write the maximum result (CRF score and pixel position) to the specified shared memory. The first thread in the block selects the maximum result for each unit and writes it to the global memory. This completes the FAST corner point detection of the entire image and transfers the result to the host (CPU).
[0023] Optionally, the GPU-accelerated FAST corner feature extraction step c in the GPU-accelerated visual loop detection module calculates the FAST corner in the warp of the device (GPU), first selecting 16 pixels with a radius of 3 for the pixel, and then storing the result of the comparison of the 16 pixels as a bit array as an index to the lookup table. All possible results are calculated as All are binary and stored in bits (8KB), using 4-byte integers to store these results, each integer storing 32 combinations.
[0024] Optionally, the GPU-accelerated FAST corner feature extraction step e in the GPU-accelerated visual loop detection module uses multiple warps to perform non-maximum suppression, and the steps are as follows:
[0025] Step (1): The warp reads the first row of cells, and each thread gets the response of 1 pixel;
[0026] Step (2): The entire warp starts Moore neighborhood suppression. Each thread verifies whether its response reaches the maximum value in Moore neighborhood. Non-maximum threads will suppress their responses.
[0027] Step (3): The warp moves to the next row and repeats steps 1 and 2 to process the entire grid of cells. The first thread in the block selects the maximum result for the cell and writes it to global memory, completing the processing of the cell.
[0028] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:
[0029] (1) The present invention proposes a SLAM method based on GPU-accelerated 3D laser radar, IMU, and visual multi-sensor fusion. The laser radar and IMU are tightly coupled at the front and back ends, and visual sensors are used in the loop detection part. The visual image texture is richer, and the loop closure method is simpler and faster than that of the laser radar. In addition, the loop detection capability is stronger for scenes with unclear or repeated features. The effective combination of these three sensors not only avoids the complexity of the tight coupling of the three sensor data, but also achieves a considerable effect;
[0030] (2) A voxel-based GICP (VGICP) matching cost factor is used. Compared with the commonly used feature-based line and plane point matching methods, this method uses more points to calculate the registration error, resulting in higher matching accuracy. It also uses the parallel structure of the GPU to accelerate the implementation, thus improving the real-time performance of point cloud matching.
[0031] (3) Visual sensors are used for loop detection. The feature detection part is deployed on the GPU for acceleration. After the loop is identified, it is reconfirmed by the lidar key frame, achieving accurate and fast loop detection requirements. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is an overall block diagram of a SLAM method based on GPU acceleration and multi-sensor fusion of the present invention.
[0033] Figure 2 This is a laser radar point cloud matching acceleration strategy diagram of a GPU-accelerated multi-sensor fusion SLAM method of the present invention.
[0034] Figure 3 The invention discloses a sliding window factor graph of a SLAM method based on GPU acceleration and multi-sensor fusion.
[0035] Figure 4 This is a visual loop detection acceleration strategy diagram of a multi-sensor fusion SLAM method based on GPU acceleration in the present invention.
[0036] Figure 5 This is a visual loop detection flow chart of a SLAM method based on GPU acceleration and multi-sensor fusion in the present invention. DETAILED DESCRIPTION
[0037] In order to make the purpose, advantages and technical solutions of the present invention clearer and more specific, further detailed description is given below in conjunction with specific embodiments of the present invention and corresponding drawings.
[0038] Figure 1 It is an overall block diagram of an embodiment of a SLAM method based on GPU acceleration of multi-sensor fusion of the present invention, wherein the sensors include a 3D laser radar, an IMU, and a monocular camera, and the method includes a front-end odometer module based on GPU-accelerated laser radar / IMU tight coupling, a back-end optimization module based on factor graph, a point cloud map creation module, and a visual loop detection module based on GPU acceleration.
[0039] A SLAM method based on GPU-accelerated multi-sensor fusion takes IMU as the body coordinate system and defines the sensor state as follows:
[0040] (1)
[0041] in and is the IMU linear acceleration and angular acceleration noise.
[0042] Figure 2 The present invention is a laser radar point cloud matching acceleration strategy diagram of a multi-sensor fusion SLAM method based on GPU acceleration, which includes the following steps:
[0043] Step 1: Downsampling: Read the LiDAR data and perform voxel grid downsampling on the LiDAR point cloud data. Specifically, a fixed-size voxel container is set (in this embodiment, 0.1m*0.1m*0.1m is selected), all point clouds are divided into different voxels, and points that are far from the grid and obviously wrong are removed;
[0044] Step 2: Motion compensation: Use IMU motion prediction to compensate for point cloud distortion caused by motion within the radar frame. First, integrate the IMU measurements between the start and end of a laser frame scan. Then use the timestamp of each laser measurement point to linearly interpolate the IMU integration results to project each point to the start time of the laser frame. This completes the motion compensation. The IMU will be in the future. The speed, position, and rotation at the moment are calculated as follows:
[0045] (2)
[0046] in and is the white noise in the IMU measurement
[0047] IMU pre-integration obtains the relative motion between two radar time stamps, and the speed measurement, position measurement and rotation measurement between the i-th moment and the j-th moment are calculated as follows:
[0048] (3)
[0049] in , and All of them are white noise in the pre-integration process
[0050] Step 3: Voxelize the target point: The host (CPU) sets the point cloud data of the previous frame as the target point cloud and stores it in the container. Use KD-tree to find the point cloud within the radius r of the target point and store it in the container. Copy the target point cloud from the host (CPU) to the device (GPU). Each target point cloud corresponds to a voxel index, initialize the mean and covariance (skip this step if not initialized).
[0051] Step 4: Voxelize source points: The host (CPU) sets the current frame point cloud data as the source point cloud and stores it in a container. Use KD-tree to find the point cloud within the radius r of the target point and store it in the container. Copy the source point cloud from the host (CPU) to the device (GPU). Each source point cloud corresponds to a voxel index, and initializes the mean and covariance.
[0052] Step 5: Calculate mean and covariance: The host side (CPU) designs the kernel function according to the device conditions and the number of point clouds, sets the kernel width, and the number of threads equals the number of voxels. The driver side (GPU) models the point cloud into a Gaussian distribution according to the index, and models each input point cloud into a Gaussian distribution. , and calculate the mean and covariance , where the covariance is calculated from k neighboring points within the radius r of the point cloud;
[0053] Step 6: Calculate relative transformation: The device (GPU) calculates the correspondence between voxels and the distance between the source point and the neighboring points within the radius r of multiple target points. The distance residual is shown in formula (4). The maximum likelihood estimation is used to obtain the maximum posterior probability. The Gauss-Newton method is used for iterative optimization. The error term is weighted by the number of points in the voxel. The information matrix is obtained according to the covariance matrix of the voxel. It is iterated until the target cost function converges and the transformation matrix with the highest confidence is found. , its cost function is shown in formula (5), and finally the transformation matrix Transmit to the host (CPU);
[0054] (4)
[0055] Where s represents the origin, t represents the target point, and j represents the number of multiple target points.
[0056] (5)
[0057] Where i represents the number of neighboring points.
[0058] FIG3 is a back-end sliding window factor graph of a SLAM method based on GPU acceleration of multi-sensor fusion of the present invention, including an IMU pre-integration factor, a laser radar matching cost factor, and a visual loopback factor. The IMU pre-integration factor is given by formula (3), the laser radar matching cost factor is given by formula (5), and the visual loopback factor is given by formula (6). Specifically, a factor graph with a fixed window size of 20 state nodes is constructed, and the constraints between the optimization factors are: the constraint between the marginalized key frame and the latest frame is a unary edge, and other factors such as the IMU pre-integration factor, the laser radar matching cost factor, and the visual loopback factor are binary edges. When a new sensor state is added, the fixed lag smoothing method is used to optimize the factor graph and marginalize the old frame, and the marginalized frame is managed by the key frame mechanism. The key frame has a good spatial distribution and has sufficient overlap with the latest frame. An overlap rate threshold is set to manage the key frame. If the overlap rate is less than the threshold (90% is selected in this embodiment), it is set as a key frame. Otherwise, the frame is discarded. The key frames marginalized by the factor graph are sent to the point cloud map creation module as the initial estimate of the sensor state. The point cloud map creation module continuously accepts the marginalized key frames, obtains the transformation relationship between the key frames, and splices them into a globally consistent point cloud map.
[0059] Figure 4 This is a visual loop detection acceleration strategy diagram for a multi-sensor fusion SLAM method based on GPU acceleration. Based on the GPU-accelerated visual loop detection module, the acceleration part is FAST corner feature detection. The specific steps are:
[0060] Step a: Obtain image pyramid: The host (CPU) performs subsampling on the camera input image to obtain the image pyramid;
[0061] Step b: Divide into rectangular units: The host side (CPU) divides each resolution image into rectangular units with fixed width and height, and copies them to the device side (GPU). Each rectangular block corresponds to a thread block.
[0062] Step c: Determine the region of interest: The device (GPU) calculates the FAST corner points in the warp, compares the brightness of the selected pixel with the 16 pixels around it with a radius of 3, and then stores the brightness comparison results of the 16 pixels as a bit array as the index of the lookup table. Calculate all possible results as types and are all binary, store them in The results are stored in 4-byte integers in the 8KB memory, with each integer storing 32 combinations. The index hits in the cache table to determine the region of interest (ROI);
[0063] Step d: Corner response function evaluation: For each pixel in the region of interest (ROI) in each rectangular block, the device (GPU) first performs a coarse corner response function (CCRF) evaluation and then a corner response function (CRF) evaluation;
[0064] Step e: Non-maximum suppression: The device (GPU) divides the corner response map into a regular cell grid, assigns a thread block to each cell, and uses multiple warps to process a cell. For non-maximum suppression, the warp first reads the first row of cells, and each thread obtains the response of 1 pixel. Then the entire warp starts Moore neighborhood suppression. Each thread verifies whether its response reaches the maximum value in the Moore neighborhood. The non-maximum thread will suppress its response. Then the warp moves to the next row and repeats steps 1 and 2 to process the entire cell grid.
[0065] Step f: CPU reads FAST corner points: All warps in the thread block write the maximum result (CRF score and pixel position) to the specified shared memory. The first thread in the block selects the maximum result for each unit and writes it to the global memory. This completes the FAST corner point detection of the entire image and transfers the result to the host (CPU).
[0066] Figure 5 This is a visual loop detection flow chart of a multi-sensor fusion SLAM method based on GPU acceleration. To achieve visual loop detection, a visual odometer is first constructed. The visual odometer estimates the transformation matrix through the feature point method, and the scale is corrected every 20 frames by the lidar. The loop detection module is detected by the visual bag-of-words model. The specific steps of visual loop detection based on GPU acceleration are:
[0067] Step 1: Feature extraction: Feature extraction is as follows Figure 4 A visual loop detection acceleration strategy for a multi-sensor fusion SLAM method based on GPU acceleration is described in the steps. After the FAST feature is extracted, the result is transmitted to the host (CPU), the BRIEF descriptor of the feature point is calculated and converted into words (the bag-of-words dictionary tree has been created), the words of the current frame are determined, and the bag-of-words vector of the current frame is formed;
[0068] Step 2: Dictionary query: find the key frames with the same words as the current frame;
[0069] Step 3: Similarity score: Compare the differences between the bag-of-words vectors. The calculation formula is shown in formula (6). Is the difference between the bag-of-words vectors of the two frames less than a threshold (90% is selected in this embodiment)? If so, jump to step 7.
[0070] (6)
[0071] in, represents the bag-of-words vector of the current frame (source), represents the target word bag vector, and N represents the number of features
[0072] Step 4: Group matching: Combine frames with similar time into one group, and only match one of them during matching. If the group matching fails, jump to step 7;
[0073] Step 5: Time rationality check: For a detected group match, calculate the k consecutive group matches that are valid before this, and regard the group match with the highest score as a loop. If the time rationality check fails, jump to step 7;
[0074] Step 6: Structural consistency check: For the current frame and loop frame with the same word bag vector, further confirm the consistency of the word bag vector in terms of structural order. If the structural consistency check fails, jump to step 7;
[0075] Step 7: End the loop. If a loop is detected, output the loop frame. Otherwise, end the loop.
[0076] After the loop is detected, it is further confirmed by the lidar keyframe with the same timestamp. After the loop is confirmed, the loop factor is added to the backend factor graph to perform a global optimization, optimizing the sensor pose state and point cloud map at the same time.
Claims
1. A multi-sensor fusion slam method based on GPU acceleration, It is characterized in that Including GPU-accelerated 3D LiDAR / IMU tightly coupled front-end odometer module, factor graph-based back-end optimization module, point cloud map creation module, and GPU-accelerated visual loop detection module: The GPU-accelerated 3D laser radar / IMU tightly coupled front-end odometer module, wherein the GPU-accelerated point cloud matching steps are as follows: Step 1: Downsampling: The host CPU reads the lidar data and performs voxel grid downsampling on the lidar point cloud data, dividing all point clouds into different voxels and removing points that are far from the grid and obviously wrong. Step 2: Motion compensation: The host CPU uses IMU motion prediction to perform motion compensation on the point cloud distortion caused by the motion within the radar frame, and projects each point cloud to the start time of the lidar frame; Step 3: Voxelize the target point: The host CPU sets the point cloud data of the previous frame as the target point cloud and stores it in the container. Use KD-tree to find the point cloud within the radius r of the target point and store it in the container. Copy the target point cloud from the host CPU to the device GPU. Each target point cloud corresponds to a voxel index, initialize the mean and covariance, and skip this step if it is not initialized. Step 4: Voxelize the source point: The host CPU sets the current frame point cloud data as the source point cloud and stores it in the container. Use KD-tree to find the point cloud within the radius r of the target point and store it in the container. Copy the source point cloud from the host CPU to the device GPU. Each source point cloud corresponds to a voxel index, and initializes the mean and covariance. Step 5: Calculate mean and covariance: The host CPU designs the kernel function according to the device conditions and the number of point clouds, sets the kernel width, and the number of threads equal to the number of voxels. It drives the device GPU to model the point cloud into a Gaussian distribution according to the index and calculates the mean and covariance of the source point cloud and the target point cloud. Step 6: Calculate relative transformation: The GPU on the device calculates the correspondence between voxels, calculates the distance between the source point and the neighboring points within the radius r of multiple target points, uses maximum likelihood estimation to obtain the maximum a posteriori probability, uses Gauss-Newton method to iteratively optimize, uses the number of points in the voxel to weight the error term, and obtains the information matrix based on the covariance matrix of the voxel. Iterate until the target cost function converges to find the transformation matrix with the highest degree of accuracy. , and finally transform the matrix Transmit to the host CPU; Step 7: Update point cloud: Set the current frame point cloud as the target point cloud and wait for the next calculation; The factor graph-based backend optimization module constructs a sliding window including an IMU pre-integration factor, a lidar matching cost factor, and a visual loopback factor; when a new sensor state is added, the factor graph is optimized using a fixed lag smoothing method and the updated pose state is fed back to the GPU-accelerated 3D lidar / IMU tightly coupled front-end odometer module for correction, and the marginalized key frames are used to construct a point cloud map; The GPU-accelerated visual loop detection module first performs GPU-accelerated FAST corner feature extraction, and then performs dictionary query, similarity scoring, group matching, time rationality check, and structural consistency check; when the loop is determined, the loop factor is added to the factor graph-based backend optimization module to perform a global optimization.
2. According to claim 1, a SLAM method based on GPU acceleration of multi-sensor fusion, It is characterized in that In the factor graph-based backend optimization module, the size of the sensor state nodes in the constructed sliding window factor graph is fixed to 20; the constraints between the optimization factors are: the constraint between the marginalized key frame and the latest frame is a unary edge, and the constraints of the IMU pre-integration factor, the lidar matching cost factor, and the visual loop factor are binary edges.
3. The SLAM method of multi-sensor fusion based on GPU acceleration according to claim 1, It is characterized in that The GPU-accelerated visual loop detection module, wherein the steps of GPU-accelerated FAST corner feature extraction are as follows: Step a: Obtaining the image pyramid: The host CPU performs subsampling on the camera input image to obtain the image pyramid; Step b: Divide into rectangular units: The host CPU divides each resolution image into rectangular units of fixed width and height, and copies them to the device GPU. Each rectangular block corresponds to a thread block. Step c: Determine the region of interest: The GPU on the device calculates the FAST corner points in the warp and compares the brightness of the selected pixel with enough pixels in its surrounding area to determine the region of interest (ROI). Step d: Corner response function evaluation: For each pixel of the region of interest ROI in each rectangular block, the device-side GPU first performs a coarse corner response function CCRF evaluation, and then performs a corner response function CRF evaluation; Step e: Non-maximum suppression: The GPU on the device divides the corner response map into a regular grid of cells, assigns a thread block to each cell, uses multiple warps to process one cell, and performs non-maximum suppression; Step f: CPU reads FAST corner points: All warps in the thread block write the maximum result, i.e., CRF score and pixel position, into the specified shared memory. The first thread in the block selects the maximum result for each unit and writes it into the global memory, thus completing the FAST corner point detection for the entire image and transferring the result to the host CPU.
4. The SLAM method of multi-sensor fusion based on GPU acceleration according to claim 3, It is characterized in that In step c, the FAST corner point is calculated in the warp of the GPU on the device side. First, 16 pixels with a radius of 3 are selected, and then the brightness comparison results of the 16 pixels are stored as a bit array as an index of the lookup table; all possible results are calculated as types and are all binary, store them in These results are stored in 4-byte integers, with each integer storing 32 combinations.
5. The SLAM method of multi-sensor fusion based on GPU acceleration according to claim 3, It is characterized in that The step e uses multiple warps to perform non-maximum suppression, and the steps are as follows: Step (1): The warp reads the first row of cells, and each thread gets the response of 1 pixel; Step (2): The entire warp starts Moore neighborhood suppression. Each thread verifies whether its response reaches the maximum value in Moore neighborhood. Non-maximum threads will suppress their responses. Step (3): The warp moves to the next row and repeats steps 1 and 2 to process the entire cell grid; the first thread in the block selects the maximum result for the cell and writes it to the global memory, completing the processing of the cell.
