A single-line laser radar and depth camera fusion mapping method and system

Abstract

This invention discloses a mapping method and system based on the fusion of single-line LiDAR and depth camera, belonging to the field of SLAM mapping technology. The method includes the following steps: S1: joint calibration; S2: ORB descriptor processing; S3: pseudo-LiDAR data conversion; S4: mapping completion. This invention ensures data consistency in time and space dimensions through joint calibration, improves the accuracy and efficiency of feature extraction and matching through ORB descriptor processing, and achieves effective fusion of the two types of data through pseudo-LiDAR data conversion. The resulting 2D grid map, integrated with the Cartographer algorithm framework, significantly improves obstacle detection rate, provides clearer details, accurately displays obstacle locations, avoids collisions during robot navigation, and effectively suppresses map drift. This greatly improves the accuracy and robustness of the SLAM system, enhances the robot's obstacle avoidance ability in complex environments, and reduces costs compared to 3D LiDAR, making it of significant practical application value.

Description

Technical Field

[0001] This invention relates to the field of SLAM mapping technology, specifically to a mapping method and system based on the fusion of single-line lidar and depth camera. Background Technology

[0002] Robotics technology has permeated various fields such as home, industry, and services, with Simultaneous Localization and Mapping (SLAM) and autonomous robot navigation becoming hot research topics. Single-sensor mapping has its limitations. Single-line LiDAR acquires limited two-dimensional planar information, cannot detect objects above the radar scanning plane, and suffers from insufficient mapping accuracy. Furthermore, single-sensor failure may occur in complex indoor environments, while 3D LiDAR is prohibitively expensive. Depth cameras can capture color and texture details of the environment; however, they are sensitive to lighting conditions and perform poorly in low-light or blurred areas.

[0003] Therefore, SLAM mapping research that integrates depth cameras and single-line LiDAR has significant practical application value. To this end, this invention proposes a mapping method and system based on the fusion of single-line LiDAR and a depth camera. Summary of the Invention

[0004] The technical problem to be solved by this invention is: how to improve the accuracy and robustness of SLAM system by combining the advantages of single-line LiDAR and depth camera, and provides a mapping method based on the fusion of single-line LiDAR and depth camera.

[0005] The present invention solves the above-mentioned technical problems through the following technical solution, and the present invention includes the following steps:

[0006] S1: Joint Calibration

[0007] The depth camera and single-line lidar are jointly calibrated to determine the spatial transformation relationship between the camera coordinate system and the lidar coordinate system, that is, to solve the rotation matrix and translation vector between the camera coordinate system and the lidar coordinate system.

[0008] S2: ORB descriptor processing

[0009] The ORB-SLAM2 system is used to acquire depth image data collected by a depth camera, and the Tracking main thread is used to process the depth image data using ORB descriptors.

[0010] S3: Pseudo-laser data conversion

[0011] The depth image data processed by the ORB descriptor is projected and converted into two-dimensional pseudo-laser data.

[0012] S4: Mapping complete

[0013] By integrating 2D pseudo-laser data and single-line lidar data into the Cartographer algorithm framework, a 2D grid map for navigation and obstacle avoidance is generated.

[0014] Furthermore, in step S1, the joint calibration includes time synchronization and spatial synchronization, so that the point cloud data acquired by the depth camera and the single-line lidar data are consistent in time and space dimensions.

[0015] Furthermore, the time synchronization adopts a sliding window matching method, which aligns the data frames of the depth camera and the single-line LiDAR based on timestamps in ROS, wherein the sampling frequency of the depth camera is 30Hz and the sampling frequency of the single-line LiDAR is 10Hz.

[0016] Furthermore, the spatial synchronization is achieved using a QR code calibration board. During the calibration process, the depth camera and single-line LiDAR simultaneously scan the calibration board to extract feature points of the calibration board in their respective coordinate systems. The rotation matrix and translation vector are determined by solving the correspondence between the feature points. The feature point extraction is achieved by the depth camera identifying the corner points of the QR code on the calibration board and the single-line LiDAR scanning the edge feature points of the calibration board. The correspondence between the feature points is fitted based on the least squares method, and the coordinate system transformation parameters are calculated to obtain the rotation matrix and translation vector.

[0017] Furthermore, in step S2, the ORB-SLAM2 system extracts scale-invariant and rotation-invariant feature points from the image acquired by the depth camera using the ORB feature extraction algorithm, and performs feature matching using the BRIEF descriptor to improve the accuracy and efficiency of data matching.

[0018] Furthermore, in step S2, the specific processing procedure includes:

[0019] S21: Start the depth camera to acquire RGB and depth images, and initialize image frames through the Tracking main thread of the ORB-SLAM2 system;

[0020] S22: The FAST corner detection algorithm is used to extract corner features from the image. Direction information is added to each corner to achieve rotation invariance. A scale pyramid is constructed to achieve scale invariance, and ORB feature points are obtained.

[0021] S23: Perform binary encoding on the pixel blocks surrounding each ORB feature point to generate a BRIEF descriptor;

[0022] S24: Use Hamming distance to match the BRIEF descriptors of adjacent frames, filter out successfully matched feature point pairs, and output depth image data with ORB descriptors.

[0023] Furthermore, in step S3, the projection transformation is based on the camera pinhole model. The camera intrinsic parameter matrix is ​​used to project 3D spatial points onto a 2D image plane, and the camera motion pose is estimated using the PnP algorithm in OpenCV. The PnP algorithm solves for the camera's rotation matrix R and translation matrix T based on the 3D spatial points and their corresponding 2D plane projection points. The 3D spatial points are determined by the depth image and pixel coordinates. The transformation formula is as follows:

[0024] ;

[0025] Where s is the scale factor, K is the camera intrinsic parameter matrix, and R and T are the rotation and translation matrices, respectively. Let u and v be the coordinates of a point in three-dimensional space [X,Y,Z], and let u and v be the pixel coordinates of the two-dimensional image plane. This is the camera motion pose matrix.

[0026] Furthermore, in step S4, the specific processing procedure includes:

[0027] S41: Data Preprocessing

[0028] The raw data of the single-line lidar is denoised and filtered, the coordinate transformation of the two-dimensional pseudo-liquid data is performed, and the transformation matrix obtained by joint calibration is used to unify them into the single-line lidar coordinate system.

[0029] S42: Data Fusion

[0030] The aligned two-dimensional pseudo-laser data and single-line lidar data are fused according to timestamps to generate time-synchronized, fused two-dimensional laser scanning data;

[0031] S43: Cartographer Algorithm Iteration

[0032] The fused 2D laser scan data is input into the Cartographer framework. The robot's real-time pose is calculated through front-end scanning and matching. The back-end uses nonlinear optimization algorithms (such as Sparse Pose Adjustment (SPA) or nonlinear least squares optimization implemented in Ceres Solver) to optimize the pose trajectory and suppress map drift.

[0033] S44: Raster Map Generation

[0034] Based on the optimized pose trajectory and fused point cloud data, a two-dimensional grid map is constructed, marking obstacle areas and free areas, and outputting a two-dimensional grid map that can be used for navigation and obstacle avoidance.

[0035] This invention also provides a mapping system based on the fusion of single-line lidar and depth camera, applied to the above-mentioned method, comprising:

[0036] The joint calibration module is used to perform joint calibration of the depth camera and the single-line lidar, and to determine the spatial transformation relationship between the camera coordinate system and the radar coordinate system, that is, to solve the rotation matrix and translation vector between the camera coordinate system and the radar coordinate system.

[0037] The ORB descriptor processing module is used to acquire depth image data from a depth camera using the ORB-SLAM2 system and perform ORB descriptor processing on the depth image data.

[0038] The data conversion module is used to project and convert the depth image data processed by the ORB descriptor into two-dimensional pseudo-laser data;

[0039] The mapping module integrates 2D pseudo-laser data and single-line lidar data into the Cartographer algorithm framework to generate a 2D raster map for navigation and obstacle avoidance.

[0040] Compared with existing technologies, this invention has the following advantages: This mapping method based on the fusion of single-line LiDAR and depth camera overcomes the limitations of a single sensor by combining the advantages of single-line LiDAR and depth camera (single-line LiDAR has limited two-dimensional planar information and insufficient mapping accuracy, while depth camera is sensitive to lighting conditions and performs poorly in low-light or blurred areas). Joint calibration ensures data consistency in time and space dimensions, ORB descriptor processing improves the accuracy and efficiency of feature extraction and matching, and pseudo-LiDAR data conversion achieves effective fusion of the two types of data. The resulting two-dimensional grid map, integrated with the Cartographer algorithm framework, significantly improves obstacle detection rate, provides clearer details, accurately displays obstacle positions, avoids collisions during robot navigation, effectively suppresses map drift, greatly improves the accuracy and robustness of the SLAM system, enhances the robot's obstacle avoidance ability in complex environments, and reduces costs compared to 3D LiDAR, making it of significant practical application value. Attached Figure Description

[0041] Figure 1 This is a flowchart illustrating the mapping method based on the fusion of single-line lidar and depth camera in an embodiment of the present invention.

[0042] Figure 2 This is a schematic diagram of the radar and camera positions in an embodiment of the present invention, wherein RPLIDAR A2 is a single-line lidar;

[0043] Figure 3 This is a schematic diagram of the QR code calibration board in an embodiment of the present invention;

[0044] Figure 4 These are depth point cloud maps and converted two-dimensional point cloud maps in embodiments of the present invention, wherein (a) is a depth point cloud map and (b) is a two-dimensional point cloud map;

[0045] Figure 5 This is a depth information effect diagram of the current frame in an embodiment of the present invention, wherein (a) is an RGB image, (b) is the original camera point cloud, and (c) is a point cloud with ORB descriptor;

[0046] Figure 6 This is a dense point cloud map constructed with ORB descriptors obtained in an embodiment of the present invention;

[0047] Figure 7 This is a schematic diagram illustrating the mapping process performed by controlling the robot's movement via a keyboard in an embodiment of the present invention.

[0048] Figure 8 This is a diagram illustrating the mapping effect of using a keyboard-driven mobile cart to navigate through every corner in an embodiment of the present invention.

[0049] Figure 9 This is a schematic diagram of mapping using fused depth camera data in an embodiment of the present invention. Detailed Implementation

[0050] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0051] This embodiment provides a technical solution: This invention focuses on SLAM mapping in indoor environments, aiming to improve the accuracy and robustness of the SLAM system by integrating the advantages of single-line LiDAR and depth cameras. This invention provides a fusion strategy for LiDAR and depth cameras. This strategy mainly integrates point cloud data acquired by the depth camera (RGB-D camera) into a two-dimensional grid map, improving the accuracy of environmental and obstacle information acquisition and greatly enhancing the obstacle avoidance capability of the intelligent vehicle in complex environments. Specifically, the method uses the Cartographer algorithm as the underlying algorithm framework, transforming the depth data layer with ORB descriptors into a pseudo-LiDAR data layer through projection transformation and fusing it with single-line LiDAR data for mapping. The specific steps are as follows: Figure 1 As shown.

[0052] 1. Joint calibration of depth camera and radar

[0053] To ensure that the point cloud data acquired by the depth camera and the single-line lidar data are in the same latitude, sensor extrinsic parameter calibration experiments are required in the early stages. The purpose of joint calibration of the depth camera and radar is to determine the spatial transformation relationship between the camera coordinate system and the radar coordinate system, that is, to solve for the rotation matrix and translation vector between the camera coordinate system and the radar coordinate system. Joint calibration mainly consists of time synchronization and spatial synchronization.

[0054] Time synchronization: Due to the different sampling frequencies of the same sensors (30Hz for cameras, 10Hz for radar), the internal clocks of the sensors are independent, and drift will accumulate over time. Therefore, time synchronization is a critical step to ensure that data from different sensors are aligned on the time axis. Asynchrony will lead to data fusion errors (such as trajectory drift and map misalignment). This invention utilizes a sliding window matching method (message filters), which selects the closest data frames within a time window for fusion. In ROS (Robot Operating System), message filters is a function package for handling the synchronization of messages from multiple topics. It allows users to align data from different sensors based on timestamps, and is suitable for scenarios that require simultaneous processing of multi-source data, such as vision-LiDAR fusion and multi-camera synchronization.

[0055] Spatial synchronization: As shown in the camera model, the rotation matrix R is obtained by transforming the world coordinate system to the camera coordinate system. c Translation vector T c Similarly, the transformation from the camera coordinate system to the single-line lidar coordinate system also requires the rotation matrix R. lc Translation vector T lc The conversion formula is shown below:

[0056] ;

[0057] in, The coordinates of the point in the radar coordinate system. These are the coordinates of a point in the camera coordinate system.

[0058] Radar and camera positions as follows Figure 2 As shown, unlike the camera intrinsic parameter calibration, the joint calibration changed the checkerboard calibration board to a QR code calibration board. The checkerboard calibration effect is very poor because the joint calibration of the camera and radar requires recording video, which requires constantly shaking the calibration board. Corner information is easily lost when recording with the checkerboard.

[0059] QR code calibration board, such as Figure 3 As shown, the dimensions are both 36.6 cm. The main principle is to have the depth camera and LiDAR simultaneously scan the calibration board and extract its feature points in their respective coordinate systems. By solving the correspondence between these feature points in different coordinate systems, the transformation matrix and translation vector between the depth camera and LiDAR are determined.

[0060] In this embodiment, the specific steps for extracting feature points of the calibration plates in their respective coordinate systems are as follows:

[0061] S101: Fix the QR code calibration board, start the depth camera and single-line LiDAR, and ensure that both scan the calibration board clearly at the same time;

[0062] S102: At the depth camera end, the calibration board is located using the ArUco QR code detection algorithm (QR code recognition algorithm), and the pixel coordinates of the four corner points of the QR code are extracted. Combined with the depth image, the three-dimensional coordinates of the corner points are obtained, thus obtaining the point coordinates P in the camera coordinate system. c ;

[0063] S103: At the single-line lidar end, scan the edge contour of the calibration board, extract contour feature points using a vertex extraction algorithm based on line fitting, and select the laser points corresponding to the four vertices of the calibration board to obtain the three-dimensional coordinates P in the radar coordinate system. l ;

[0064] S104: A one-to-one correspondence between the four vertices of the calibration board extracted by the depth camera and the single-line LiDAR, forming feature point pairs {(P c 1,P l 1),(P c 2,P l 2),(P c 3,P l 3),(P c 4,P l 4)}.

[0065] In this embodiment, the specific steps for solving the correspondence between feature points in different coordinate systems are as follows:

[0066] S111: Initialize rotation matrix R cl Let T be the identity matrix and the translation vector. cl It is a zero vector;

[0067] S112: Construct an objective function based on the least squares method, with the objective of minimizing the coordinate transformation error of all feature point pairs: Σ||P l -(R cl ×P c +T cl )||²;

[0068] S113: The SVD (Singular Value Decomposition) algorithm is used to solve the optimal solution of the objective function, and the initial rotation matrix and translation vector are obtained;

[0069] S114: Remove abnormal feature point pairs whose errors exceed the threshold, repeat steps S112 and S113 for optimization, and output the final rotation matrix R. cl Translation vector Tcl .

[0070] 2. Depth image data to laser point cloud data

[0071] Projecting depth data onto two-dimensional coordinates essentially involves using the camera's intrinsic parameter model to project points in three-dimensional space (determined by the depth map and pixel coordinates) back onto the two-dimensional image plane. The camera's pinhole model is the core of this projection process. Let's assume points in three-dimensional space... Its projection formula is as follows:

[0072]

[0073] Wherein, the K-camera intrinsic parameter matrix, (u,v) is the coordinate of the three-dimensional point P on the two-dimensional image plane.

[0074] Before initiating the transformation, ensure the depth camera node is running and has published the depth image topic and information topic. Then, start the `depth_image_proc` node and the `pointcloud_to_laserscan` node in a new terminal. Although the depth camera can directly acquire environmental depth data, depth data processed by the ORB descriptor is more beneficial for environmental localization and initialization. After acquiring 2D pseudo-laser data, to better obtain the initial pose of the camera projected onto 2D coordinates, the Perspective-n-Point (PnP) algorithm in OpenCV is used to estimate camera motion. PnP describes how to estimate the camera pose given a 3D spatial point and a 2D planar projection point. Projecting depth image data onto 2D image data, more precisely, involves transforming 3D points in the world coordinate system to 2D points in the image coordinate system after a series of transformations, including the transformation from world coordinates to camera coordinates and the projection normalized to the image coordinate system. The transformation formula is shown below:

[0075]

[0076] In the formula, s is the scale factor, K is the camera intrinsic parameter matrix, and R and T are the rotation and translation matrices, respectively. Let u and v be the coordinates of a point in three-dimensional space [X,Y,Z], and let u and v be the pixel coordinates of the two-dimensional image plane. The camera motion pose matrix; depth point cloud map and the transformed 2D point cloud map are shown below. Figure 4 As shown.

[0077] 3. LiDAR and Depth Camera Fusion Strategy

[0078] This invention utilizes the Cartographer algorithm as the underlying algorithm framework for fusing single-line LiDAR and depth cameras. The ORB-SLAM2 system is used to acquire depth data from the depth camera. However, the Cartographer algorithm itself does not support SLAM mapping based on depth data, so the depth data acquired by the camera needs to be converted into pseudo-LiDAR data before being integrated into the Cartographer algorithm for SLAM mapping. ROS itself provides depth nodes for the depth camera; that is, after running the depth camera driver, depth data can be obtained in Rviz through the ` / camera / depth / image` node. However, ORB-SLAM2 utilizes the ORB feature extraction algorithm, which can quickly extract a large number of scale-invariant and rotation-invariant feature points from the images acquired by the depth camera and perform feature matching using BRIEF descriptors. This process effectively utilizes the information in the depth data, improving the accuracy and efficiency of matching. In indoor environments, facing different angles and scale changes of objects such as furniture and walls, ORB can stably extract and match feature points, while the depth data acquired by the depth camera itself is somewhat inferior. In the Tracking thread, the effect of acquiring the depth information of the current frame is as follows: Figure 5 As shown.

[0079] In this embodiment, the specific process of extracting a large number of scale-invariant and rotation-invariant feature points from the image acquired by the depth camera and performing feature matching using the BRIEF descriptor is as follows:

[0080] S201: Image frame preprocessing, grayscale processing of RGB images acquired by depth camera, construction of pyramid model, and processing of acquired corner information;

[0081] S202: Extract ORB feature points, use the FAST corner detection algorithm to detect corners at each scale pyramid layer, remove low-response corners, and calculate the principal direction for each corner point using the gray-scale centroid method to achieve rotation invariance;

[0082] S203: Improved BRIEF descriptor. Within the image patch after feature point main direction correction, a binary comparison is performed based on a 512-pair pixel comparison pattern obtained through offline training, which has low correlation and high variance, to generate a 256-bit binary descriptor.

[0083] S204: Feature point matching: First, calculate the Hamming distance between the feature point descriptors in the current frame and the reference frame (or local map) to perform preliminary matching; then, use the nearest neighbor distance ratio method to filter matching pairs, and finally output the successfully matched feature point pairs and their corresponding depth information.

[0084] from Figure 5As can be seen, the depth point cloud acquired using ORB-SLAM2 is significantly better than the depth point cloud acquired using a depth camera alone, both in terms of detail density and the number of depth points; the number of depth points is increased by 19,805. A dense point cloud constructed using the acquired point cloud with ORB descriptors is shown below. Figure 6 As shown.

[0085] 4. Comparison of Indoor Environmental Experiments

[0086] Before testing the mapping effect of the Cartographer algorithm after fusing depth image data, environmental mapping (living room + dining room) was first performed using a single-line LiDAR for easy comparison. Before mapping, the living room was set as the starting point and the dining room as the ending point. Keyboard control was initiated, with the robot moving 0.5 meters per keystroke to prevent collisions due to improper operation. Upon starting the mapping node, the robot was moved using keyboard control to perform mapping. The mapping process is as follows: Figure 7 As shown. The effect of using the keyboard to move the car to every corner to create a map is as follows. Figure 8 As shown. From Figure 8 As can be seen from the image, when using a single-line LiDAR for mapping, only the supporting legs of the table, chair, and coffee table can be scanned, so only a few points can be displayed in the image. Mapping using fused depth camera data is as follows... Figure 9 As shown.

[0087] from Figure 9 The results show that the Cartographer algorithm, which integrates depth camera data, is significantly better than single-line LiDAR mapping in terms of obstacle detail. The obstacle detection rate is improved by 60%, the obstacle edges are clear, and the position of obstacles on the map can be fully displayed, avoiding collisions during robot navigation. At the same time, there is no map drift, proving that the fusion algorithm is far superior to single-line LiDAR mapping in terms of mapping effect.

[0088] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A mapping method based on the fusion of single-line lidar and depth camera, characterized in that, Includes the following steps: S1: Joint Calibration The depth camera and single-line lidar are jointly calibrated to determine the spatial transformation relationship between the camera coordinate system and the lidar coordinate system, that is, to solve the rotation matrix and translation vector between the camera coordinate system and the lidar coordinate system. S2: ORB descriptor processing The ORB-SLAM2 system is used to acquire depth image data collected by a depth camera, and the Tracking main thread is used to process the depth image data using ORB descriptors. S3: Pseudo-laser data conversion The depth image data processed by the ORB descriptor is projected and converted into two-dimensional pseudo-laser data. S4: Mapping complete By integrating 2D pseudo-laser data and single-line lidar data into the Cartographer algorithm framework, a 2D grid map for navigation and obstacle avoidance is generated.

2. The mapping method based on the fusion of single-line lidar and depth camera according to claim 1, characterized in that, In step S1, the joint calibration includes time synchronization and spatial synchronization, so that the point cloud data acquired by the depth camera and the single-line lidar data are consistent in time and space dimensions.

3. The mapping method based on the fusion of single-line lidar and depth camera according to claim 2, characterized in that, The time synchronization adopts a sliding window matching method, which aligns the data frames of the depth camera and the single-line LiDAR based on the timestamp in ROS. The sampling frequency of the depth camera is 30Hz and the sampling frequency of the single-line LiDAR is 10Hz.

4. The mapping method based on the fusion of single-line lidar and depth camera according to claim 3, characterized in that, The spatial synchronization is achieved using a QR code calibration board. During the calibration process, the depth camera and single-line LiDAR simultaneously scan the calibration board to extract feature points of the calibration board in their respective coordinate systems. The rotation matrix and translation vector are determined by solving the correspondence between the feature points. The feature point extraction is achieved by the depth camera identifying the corner points of the QR code on the calibration board and the single-line LiDAR scanning the edge feature points of the calibration board. The correspondence between the feature points is fitted based on the least squares method, and the coordinate system transformation parameters are calculated to obtain the rotation matrix and translation vector.

5. The mapping method based on the fusion of single-line lidar and depth camera according to claim 4, characterized in that, In step S2, the ORB-SLAM2 system extracts scale-invariant and rotation-invariant feature points from the image acquired by the depth camera using the ORB feature extraction algorithm, and performs feature matching using the BRIEF descriptor to improve the accuracy and efficiency of data matching.

6. The mapping method based on the fusion of single-line lidar and depth camera according to claim 5, characterized in that, In step S2, the specific processing procedure includes: S21: Start the depth camera to acquire RGB and depth images, and initialize image frames through the Tracking main thread of the ORB-SLAM2 system; S22: The FAST corner detection algorithm is used to extract corner features from the image. Direction information is added to each corner to achieve rotation invariance. A scale pyramid is constructed to achieve scale invariance, and ORB feature points are obtained. S23: Perform binary encoding on the pixel blocks surrounding each ORB feature point to generate a BRIEF descriptor; S24: Use Hamming distance to match the BRIEF descriptors of adjacent frames, filter out successfully matched feature point pairs, and output depth image data with ORB descriptors.

7. A mapping method based on the fusion of single-line lidar and depth camera according to claim 5, characterized in that, In step S3, the projection transformation is based on the camera pinhole model. The camera intrinsic parameter matrix is ​​used to project 3D spatial points onto a 2D image plane. The camera motion pose is estimated using the PnP algorithm in OpenCV. The PnP algorithm solves for the camera's rotation matrix R and translation matrix T based on the 3D spatial points and their corresponding 2D projection points. The 3D spatial points are determined by the depth image and pixel coordinates. The transformation formula is as follows: ； Where s is the scale factor, K is the camera intrinsic parameter matrix, and R and T are the rotation and translation matrices, respectively. Let u and v be the coordinates of a point in three-dimensional space [X,Y,Z], and let u and v be the pixel coordinates of the two-dimensional image plane. This is the camera motion pose matrix.

8. The mapping method based on the fusion of single-line lidar and depth camera according to claim 7, characterized in that, In step S4, the specific processing procedure includes: S41: Data Preprocessing The raw data of the single-line lidar is denoised and filtered, the coordinate transformation of the two-dimensional pseudo-liquid data is performed, and the transformation matrix obtained by joint calibration is used to unify them into the single-line lidar coordinate system. S42: Data Fusion The aligned two-dimensional pseudo-laser data and single-line lidar data are fused according to timestamps to generate time-synchronized, fused two-dimensional laser scanning data; S43: Cartographer Algorithm Iteration The fused 2D laser scan data is input into the Cartographer framework. The robot’s real-time pose is calculated by front-end scanning and matching, and the pose trajectory is optimized by nonlinear optimization algorithm in the back-end to suppress map drift. S44: Raster Map Generation Based on the optimized pose trajectory and fused point cloud data, a two-dimensional grid map is constructed, marking obstacle areas and free areas, and outputting a two-dimensional grid map that can be used for navigation and obstacle avoidance.

9. A mapping system based on the fusion of single-line lidar and depth camera, characterized in that, The method applied to any one of claims 1 to 8 includes: The joint calibration module is used to perform joint calibration of the depth camera and the single-line lidar, and to determine the spatial transformation relationship between the camera coordinate system and the radar coordinate system, that is, to solve the rotation matrix and translation vector between the camera coordinate system and the radar coordinate system. The ORB descriptor processing module is used to acquire depth image data from a depth camera using the ORB-SLAM2 system and perform ORB descriptor processing on the depth image data. The data conversion module is used to project and convert the depth image data processed by the ORB descriptor into two-dimensional pseudo-laser data; The mapping module integrates 2D pseudo-laser data and single-line lidar data into the Cartographer algorithm framework to generate a 2D raster map for navigation and obstacle avoidance.

Images