A 3D target detection method integrating 3D lidar and monocular camera

Abstract

The present invention discloses a 3D target detection method that integrates a 3D laser radar and a monocular camera. The method includes the following steps: 1) calibrating the monocular camera; 2) acquiring a camera image and detecting 2D target objects and target categories; 3) acquiring and preprocessing a point cloud from the 3D laser radar; 4) performing ground culling on the preprocessed point cloud; 5) clustering the ground-culled point cloud to obtain the clustered point cloud and 3D bounding boxes corresponding to each type of point cloud; 6) classifying the clustered point cloud and performing non-target culling; 7) obtaining a 2D projection box based on the non-target-culled point cloud and the 3D bounding box; and 8) outputting a fused 3D target point cloud based on the 2D target object and the 2D projection box. The method can improve the accuracy of 3D point cloud detection of target objects and enhance the precision of fusion.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving, and specifically relates to a 3D target detection method that integrates a three-dimensional laser radar and a monocular camera. Background Art

[0002] In the field of autonomous driving, multiple sensors are usually used to obtain a large amount of surrounding environment information, including the location and speed of pedestrians and vehicles, as well as possible behaviors at the next moment, drivable areas, and understanding of traffic rules.

[0003] Currently, autonomous driving platforms commonly use three-dimensional lidar (LiDAR) and optical cameras for perception. Three-dimensional lidar is a type of sensor that uses multi-angle laser beams for detection and ranging. It can send millions of laser pulses per second. Due to its high reliability and accuracy, it excels at estimating the distance, coordinates, and velocity of objects. However, the resulting point cloud has low resolution, making point cloud-based classification methods inaccurate, especially for objects with sparse reflective points. Cameras, a type of image sensor, provide rich information content, boast high resolution and low price. They demonstrate relatively accurate object detection and classification performance, but are susceptible to weather and strong sunlight, and have limitations in estimating the distance, coordinates, and velocity of objects. Due to the inherent limitations of individual sensors, obtaining effective data is difficult. Therefore, the fusion of 3D lidar and optical cameras can leverage the high resolution of camera images for object detection and classification, while also leveraging the reliability of lidar point clouds for obstacle detection and ranging. This combined advantage allows for complete environmental perception and improves algorithm robustness.

[0004] Patent application number 202110447403.0 discloses a 3D target detection method and system based on the fusion of a monocular camera and a lidar. The method first uses an instance segmentation network to calculate the instance segmentation number of each pixel in the image captured by the monocular camera, then fuses the obtained instance segmentation number with the 3D point cloud of the lidar to obtain a fused 3D point cloud, and finally uses a point cloud convolution network to perform 3D target detection on the fused 3D point cloud to obtain a 3D bounding box. This method uses a checkerboard calibration plate to calibrate the external parameters of the three-dimensional lidar and camera. Although it solves the problem of inconsistency between the camera and lidar perspectives, changes in the size of the checkerboard grid, the placement of the calibration plate, and the placement of the sensor will affect the accuracy and stability of the algorithm. In addition, when the radar beam is relatively low, it is difficult to accurately obtain the checkerboard information. This method uses a point cloud convolution network for 3D target detection. Although it improves the accuracy of 3D target detection, the computational complexity of the convolution network is very high and the real-time performance is poor. Summary of the Invention

[0005] In order to solve the problem that the accuracy and stability of the target detection algorithm are reduced due to the easy loss of checkerboard information due to the low radar beam, and the problem that the algorithm computational complexity is high due to the use of point cloud convolutional network to extract object features, the present invention provides a 3D target detection method that integrates 3D lidar and monocular camera.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A 3D target detection method that integrates a 3D laser radar and a monocular camera, the method comprising the following steps: step 1, calibrating the monocular camera; step 2, acquiring a camera image and detecting 2D target objects and target categories; step 3, acquiring a point cloud of the 3D laser radar and preprocessing it; step 4, performing ground culling on the preprocessed point cloud; step 5, clustering the point cloud after ground culling to obtain the clustered point cloud and 3D bounding boxes corresponding to each type of point cloud; step 6, classifying the clustered point cloud and performing non-target culling; step 7, obtaining a 2D projection box based on the point cloud after non-target culling and the 3D bounding box; and step 8, outputting a fused 3D target point cloud based on the 2D target object and the 2D projection box.

[0008] Preferably, the monocular camera is calibrated using Zhang Zhengyou's checkerboard calibration method.

[0009] Preferably, step 2 includes the following steps: step 2.1, using a calibrated monocular camera to obtain a camera image; step 2.2, selecting a target on the camera image through a target detection network to obtain a 2D target object and target category.

[0010] Preferably, step 3 comprises the following steps:

[0011] Step 3.1: Use the installation location of the 3D LiDAR as the coordinate origin and use the right-handed coordinate system to establish the 3D LiDAR coordinate system.

[0012] Step 3.2: Set the 3D LiDAR region of interest (ROI) and extract the scan points within the ROI to form a point cloud P = {p(x, y, z) | p∈ROI}, where p(x, y, z) is the coordinate of the scan point in the 3D LiDAR coordinate system.

[0013] Step 3.3: perform data denoising on the coordinate information of the point cloud in the 3D lidar coordinate system.

[0014] Preferably, the region of interest ROI is -R max <x<R max , and -R min <y<R min , R maxR is the radius of the front and rear sides of the 3D laser radar. min is the radius of the left and right sides of the 3D laser radar, the x-axis is the forward direction of the 3D laser radar, and the x-axis and the y-axis are 90° in the horizontal plane.

[0015] Preferably, the data denoising comprises the following steps:

[0016] Set the neighborhood radius R and search for each scan point p j (x j ,y j , z j ) in the neighborhood of point set P j ={(x, y, z)|(xx j ) 2 +(yy j ) 2 ≤R 2}, where j is the index of the point cloud, j = 1, 2, 3..., P j ∈P;

[0017] Calculate each point set P separately through the median filtering algorithm j The median value Z on the Z axis m and the median Z m Replace the corresponding scanning point p j (x j ,y j , z j ) coordinate value z on the Z axis j .

[0018] Preferably, the step 4 comprises the following steps:

[0019] Step 4.1, initialize i = 0, randomly select three scanning points in the preprocessed point cloud as the initial plane subset points, and construct the plane;

[0020] Step 4.2, determining whether the remaining scan points belong to the plane based on the distances between the remaining scan points and the plane;

[0021] Step 4.3: When the number of scan points belonging to a plane is greater than a preset threshold, the plane is considered a ground point cloud plane and step 4.4 is executed; otherwise, step 4.4 is executed directly.

[0022] Step 4.4: Let i = i + 1 and determine whether i > c, where c is the maximum number of iterations. If so, remove the scan points contained in the ground point cloud plane from the point cloud. Otherwise, proceed to step 4.5.

[0023] In step 4.5, randomly select three different scanning points in the preprocessed point cloud, construct a new plane, and return to step 4.2.

[0024] Preferably, step 6 comprises the following steps:

[0025] Step 6.1, extract m features of n dimensions from the clustered point cloud;

[0026] Step 6.2, constructing a feature vector based on the set of feature values ​​extracted from each scanning point;

[0027] Step 6.3: Based on the feature vector, the scanned points are classified using a point cloud classification model based on a support vector machine to obtain the target category of the point cloud;

[0028] In step 6.4, the scanning points that do not belong to the target are removed from the clustered point cloud.

[0029] Preferably, the step 7 comprises the following steps:

[0030] Step 7.1 Project the point cloud and 3D bounding box after non-target elimination onto the flattened cylindrical surface according to equations (3) and (4) to obtain the projection coordinate x img and y img ,

[0031]

[0032]

[0033] Where h_res and v_res represent the horizontal resolution and vertical resolution of the lidar respectively;

[0034] In step 7.2, select the point with the smallest coordinate value among all the projection points as the coordinate origin, update the projection point coordinates, and obtain the 2D projection frame of the 3D radar.

[0035] Preferably, step 8 comprises the following steps:

[0036] Step 8.1, calculate the IOU value of each 2D target object and 2D projection box, and construct an n×m IOU matrix, where n is the number of 2D target objects and m is the number of 2D projection images;

[0037] In step 8.2, traverse the IOU matrix. When any IOU value is greater than the set IOU threshold t, the point cloud corresponding to the projection point in the 2D projection box is output as the target point cloud, and the target category of the 2D target object is output as the 3D target category.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention projects 3D lidar data onto a 2D image without using a checkerboard by finding the regions of interest of the radar and camera, and fuses the radar data and camera data based on the idea of ​​intersection over union (IoU). The present invention adopts a support vector machine algorithm with better performance in nonlinear conditions to classify the 3D point cloud of the target object, further improving the accuracy of 3D point cloud detection of the target object and the precision of fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Flowchart of the present invention.

[0041] Figure 2 It is the region of interest extracted by the three-dimensional laser radar in the present invention.

[0042] Figure 3 Schematic diagram of projecting point cloud into 2D image. DETAILED DESCRIPTION

[0043] The present invention provides a 3D target detection method that integrates a three-dimensional laser radar and a monocular camera. The software platform for implementing the method of the present invention is the Robot Operating System (ROS); the autonomous driving platform uses a four-wheel wire-controlled chassis; the monocular camera uses a Logitech C310 high-definition camera with a resolution of 720p, and the three-dimensional laser radar uses the RS-LiDAR-16 from RoboSense. The radar is a 16-line laser radar with a scanning frequency of 5 to 20 Hz, capable of 360° scanning in the horizontal direction, with a horizontal angle resolution of 0.1° to 0.4°, a vertical viewing angle of 30° (i.e., -15° to 15°), a vertical angle resolution of 2°, and can generate nearly 300,000 points per second in single-echo mode; the computer on the autonomous driving platform uses an Intel i7-10750H processor, 16GB of RAM, an NVIDIA RTX 2070 Max-Q (8GB, DDR6) graphics card, and an Ubuntu 18.04+ROS Melodic operating system.

[0044] Figure 1 A flow chart of a 3D target detection method integrating a 3D laser radar and a monocular camera is shown in FIG. Figure 1 ,The 3D object detection method includes the following 8 steps.

[0045] Step 1: Monocular camera calibration.

[0046] In the present invention, after the camera leaves the factory, the lens will have unavoidable distortion, and the camera needs to be calibrated to correct the lens distortion to achieve the best shooting effect.

[0047] Specifically, in step 1 of the present invention, the camera is calibrated using the checkerboard calibration method proposed by Zhang Zhengyou: a 9×12 checkerboard with a grid size of 6.5 cm × 6.5 cm is used as the calibration plate. During the calibration process, the checkerboard is placed within the camera's field of view. Using the calibration tools provided by OpenCV, the calibration plate is repeatedly placed flat, tilted left, right, forward, and backward in front of the camera to ensure that it covers all positions in the image. The intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients of the monocular camera are obtained.

[0048] Step 2: Acquire camera images and detect 2D target objects and target categories.

[0049] Step 2 of the present invention specifically includes the following two sub-steps:

[0050] Step 2.1, use the calibrated monocular camera to acquire a frame of camera image in real time;

[0051] In step 2.2, the target is selected on the camera image through the target detection network to obtain the 2D target object and target category.

[0052] In the present invention, the monocular camera is calibrated through step 1 to obtain the intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient of the monocular camera. Based on the intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient, each frame image output by the monocular camera is corrected to obtain a corrected camera image.

[0053] In step 2.2 of the present invention, the generation process of the target detection network is: obtaining several images containing targets, and marking rectangular target boxes on these images to determine the area where the target is located, and marking the target category for the target selected by the target box; using the images marked with target boxes and target categories as sample data sets; using the YOLO network model in the deep learning framework Darknet as the basic model, and training the parameters in the basic model based on the sample data set, and finally using the trained YOLO network model as the target detection network in the present invention to execute the content of step 2.2, so that the target box of the selected target is rectangular, and the rectangular target box, that is, the 2D target object, and the rectangular area selected on the camera image is the 2D target image, and the target category corresponding to the 2D target image can be obtained at the same time.

[0054] Step 3: Obtain the point cloud of the 3D lidar and preprocess it.

[0055] In this invention, the point cloud obtained from the 3D LiDAR cannot be used directly for target object detection. The point cloud needs to be preprocessed to obtain effective information. Specifically, it includes the following three sub-steps:

[0056] Step 3.1: Establish a 3D LiDAR coordinate system with the LiDAR's installation location as the coordinate origin. The LiDAR coordinate system uses a right-handed coordinate system, with the X and Y axes at 90 degrees on the horizontal plane. The X axis points straight ahead and aligns with the autonomous driving platform's forward direction, and the Z axis is perpendicular to the horizontal plane.

[0057] Step 3.2: Set the 3D laser radar region of interest (ROI) and extract the scan points within the ROI to form a point cloud P = {p(x, y, z) | -R max <x<R max , -R min <y<R min}, where p(x, y, z) is the coordinate of the point cloud in the 3D lidar coordinate system, R max is the radius of interest on both sides of the 3D laser radar, R min is the radius of interest on the left and right sides of the laser radar; here, the R max The value range is [10,50], R min The value range is [3, 10] and the unit is meter.

[0058] Step 3.3: De-noise the coordinate information of the point cloud in the 3D laser radar coordinate system to obtain the de-noised point cloud P. F 3D LiDAR data (coordinate information) has some measurement noise, so it is necessary to reduce the noise of the 3D LiDAR data of the point cloud. Here, the median filter method is used to filter the point cloud on the Z axis to improve the accuracy of the next step of ground fitting. Specifically:

[0059] Set the neighborhood radius to R and search for each scan point p j (x j ,y j , z j ) in the neighborhood of point set P j ={(x, y, z)|(xx j ) 2 +(yy j ) 2 ≤R 2}, where j is the index of the point cloud, j = 1, 2, 3..., and the point set P j Any point in is a scanning point in the point cloud; here, the value range of R is [2, 10], and the unit is centimeter;

[0060] Calculate each point set P separately through the median filtering algorithm j The median value z on the Z axis m and take the median Z m Replace the corresponding scanning point p j (x j ,yj , z j ) coordinate value z on the Z axis j , z j =Z m =mid{Z|(xx j ) 2 +(yy j ) 2 ≤R 2}, where mid{} is the median filter function.

[0061] Step 4: perform ground culling on the preprocessed point cloud.

[0062] In the present invention, although redundant noise point clouds are eliminated in step 3, the point clouds in the ground area cannot be completely eliminated. If the ground point clouds are not segmented, the accuracy of target classification using the point clouds in the subsequent use will be affected. Therefore, it is necessary to segment the ground and non-ground point clouds. The present invention adopts the classic random sampling consensus method (RANSAC) to fit and eliminate the noise-reduced point cloud P in step 3. F The useless ground point cloud includes the following five sub-steps:

[0063] Step 4.1, initialize i = 0, randomly select three points in the preprocessed point cloud as the initial plane subset points, and construct the plane;

[0064] Step 4.2, determine whether the remaining points belong to the plane based on the distance between the remaining points and the plane;

[0065] In step 4.3, when the number of points belonging to the plane is greater than the preset threshold, the plane is considered to be the ground point cloud plane and step 4.4 is executed; otherwise, step 4.4 is executed directly; the preset threshold value here is generally in the range of [800, 1400];

[0066] Step 4.4: Let i = i + 1 and determine if i > c, where c is the number of iterations. If so, remove the points contained in the ground point cloud plane from the point cloud. Otherwise, proceed to step 4.5. The value range of c is [10, 150].

[0067] In step 4.5, randomly select three different points in the preprocessed point cloud, construct a new plane, and return to step 4.2.

[0068] Step 5: Use the Euclidean clustering algorithm to cluster the point cloud after ground removal to obtain the clustered point cloud and the 3D bounding boxes corresponding to each type of point cloud.

[0069] In step 4, after removing the ground, the point cloud obtained still has some outliers. In order to further eliminate the influence of outliers and object boundary points on the overall feature point detection, clustering processing is required. The category information of the clustered point cloud will be more obvious, preparing for the subsequent use of the support vector machine algorithm to classify the target object. The specific method is as follows:

[0070] Step 5.1: Construct a KD tree based on point cloud coordinate information. The number of points contained in the leaf node is N. p ;

[0071] Step 5.2, set the search distance d and the minimum number of cluster points C min and the maximum number of cluster points C max , d∈[5, 20], unit is centimeter, C min ∈[50,100], C max ∈[200000,500000];

[0072] Step 5.3: Based on the search distance d, point cloud clustering is achieved using the Euclidean clustering algorithm.

[0073] Step 5.4: Determine whether the number of point clouds in the current cluster is less than the minimum number of cluster points C. min , if yes, return to step 5.3, otherwise, go to step 5.5;

[0074] Step 5.5: Determine whether the number of point clouds in the current cluster is greater than the maximum number of cluster points C. max , if so, directly output the point cloud cluster, otherwise return to step 5.3.

[0075] In step 5 of the present invention, a KD tree based on point spatial location is used to represent point cloud coordinate information, facilitating subsequent searches for neighboring points. When the number of clustered point clouds is less than the minimum number of cluster points, the point cloud cluster is not output. When the number of clustered point clouds is greater than the maximum number of cluster points, the cluster is not expanded. The output point cloud cluster includes several scan points, the total number of which is greater than the maximum number of cluster points, and the scan points belong to the point cloud after ground removal.

[0076] In step 5 of the present invention, while removing discrete points through clustering, the point cloud after the removal of discrete points can also be divided into multiple classes consisting of similar objects. Each class of point cloud has a corresponding 3D bounding box, and the 3D bounding box is also represented by three-dimensional coordinate information.

[0077] Step 6: Classify the clustered point cloud and remove non-target objects.

[0078] The clustering method in step 5 obtains a series of 3D minimum cuboids. In traditional classification, the spatial coordinates of the clustered point cloud can be projected onto the XOZ plane, and the cuboid becomes a circumscribed rectangle and a minimum circumscribed rectangle. For people and cars, the length of the car's rectangle is greater than the width; and for people, the width of the people's rectangle is greater than the length. Therefore, a threshold can be set based on human experience. If the length:width is greater than this threshold, it is identified as a "car", otherwise it is identified as a "person" to achieve the classification of the target object. However, since the aspect ratio is set based on human experience, in complex scenes, redundant point clouds cannot be eliminated completely, and the accuracy of the target object 3D point cloud recognition is significantly reduced, which will eventually affect the results of the camera data fusion in subsequent steps. In order to improve the accuracy of 3D target object detection and improve the accuracy of fusion, the present invention adopts a support vector machine algorithm with better performance under nonlinear conditions to classify the target object 3D point cloud. The specific steps are as follows:

[0079] Step 6.1, extract m features of n dimensions from the clustered point cloud;

[0080] Step 6.2: construct a feature vector f=(f1, f2, ..., f m );

[0081] Step 6.3: Based on the feature vector, the scanned points are classified using a point cloud classification model based on a support vector machine to obtain the target category of the point cloud;

[0082] Step 6.4: Eliminate the scanning points that do not belong to the target in the clustered point cloud, and obtain the point cloud P after non-target elimination. T .

[0083] In step 6.3 of the present invention, the construction of the point cloud classification model based on the support vector machine includes the following steps:

[0084] Construct a point data sample set, where each point sample data in the point data sample set includes a corresponding feature vector and sample category;

[0085] The number of positive and negative samples is divided into 1:1, and all feature vectors are normalized; the kernel function is set to Gaussian radial basis kernel function;

[0086] Based on the point data sample set, a multi-classifier is constructed using the support vector machine algorithm. This multi-classifier can identify target categories such as cars, people, trees, houses, etc. The classifier is re-learned and trained in each new iteration to obtain the multi-classifier with the optimal classification effect and classify the point cloud data.

[0087] In step 6.4 of the present invention, the scanning points identified as trees and houses can be eliminated, and only the scanning points of cars and people are obtained to form the point cloud P after non-target elimination. T .

[0088] Step 7: Point cloud P after non-target elimination T The 3D bounding box obtained in step 5 is projected to obtain a 2D projection box.

[0089] In the present invention, the 3D point cloud of the target object detected by the 3D laser radar cannot be directly fused with the 2D image detected by the camera, so the present invention adopts the projection method to point cloud P T Project it onto the 2D image and then perform data fusion. The specific method is as follows:

[0090] Step 7.1, remove the non-target point cloud P T The 3D bounding box obtained in step 5 is projected onto the flattened cylindrical surface according to equations (3) and (4), and the projection coordinates x are obtained. img and y img ,

[0091]

[0092]

[0093] Where h_res and v_res represent the horizontal resolution and vertical resolution of the lidar, respectively. Here, the value of h_res is 0.1° and the value of v_res is 2°.

[0094] In step 7.1 of the present invention, since multiple categories are obtained through clustering, each category corresponds to a 3D bounding box, so there are multiple 3D bounding boxes.

[0095] In step 7.2, select the point with the smallest coordinate value among all the projection points as the coordinate origin, update the projection point coordinates, and obtain the 2D projection frame of the 3D radar.

[0096] In step 7.2 of the present invention, the minimum value of the obtained projection points is set to (0, 0) to prevent most points from falling on the negative half axes of the x-axis and the y-axis.

[0097] Step 8: Based on the 2D target object and the 2D projection frame, output the fused 3D target point cloud.

[0098] Specifically, it includes the following sub-steps:

[0099] Step 8.1, calculate the IOU value between each 2D target object and the 2D projection box, and construct an n×m IOU matrix, where n is the number of 2D target objects and m is the number of 2D projection boxes;

[0100] In step 8.2, traverse the IOU matrix. When any IOU value is greater than the set IOU threshold t, the point cloud corresponding to the projected point within the 2D projection box is output as the target point cloud, and the target category of the 2D target object is output as the 3D target category. The value of t here is 0.8.

Claims

1. A 3D target detection method integrating a 3D laser radar and a monocular camera, characterized in that: The method comprises the following steps: Step 1: Monocular camera calibration; Step 2: Obtain camera images and detect 2D target objects and target categories; Step 3: Obtain and pre-process the point cloud of the 3D laser radar; Step 4: perform ground removal on the pre-processed point cloud; Step 5: Cluster the point cloud after ground removal to obtain the clustered point cloud and the 3D bounding boxes corresponding to each type of point cloud; Step 6, classifying the clustered point cloud and eliminating non-targets; Step 6 includes the following steps: Step 6.1, extract m features of n dimensions from the clustered point cloud; Step 6.2, constructing a feature vector based on the set of feature values ​​extracted from each scanning point; Step 6.3: Based on the feature vector, the scanned points are classified using a point cloud classification model based on a support vector machine to obtain the target category of the point cloud; Step 7, obtaining a 2D projection frame based on the point cloud after non-target elimination and the 3D bounding box; Step 7 includes the following steps: Step 7.1 Project the point cloud and 3D bounding box after non-target elimination onto the flattened cylindrical surface according to equations (3) and (4) to obtain the projection coordinate x img and y img , （3） （4） Where h_res and v_res represent the horizontal resolution and vertical resolution of the lidar respectively; Step 7.2, select the point with the smallest coordinate value among all the projection points as the coordinate origin, update the projection point coordinates, and obtain the 2D projection frame of the 3D radar Step 8: outputting a fused 3D target point cloud based on the 2D target object and the 2D projection frame; Step 8 includes the following steps: Step 8.1, calculate the IOU value of each 2D target object and 2D projection box, and construct an n×m IOU matrix, where n is the number of 2D target objects and m is the number of 2D projection images; In step 8.2, traverse the IOU matrix. When any IOU value is greater than the set IOU threshold t, the point cloud corresponding to the projection point in the 2D projection box is output as the target point cloud, and the target category of the 2D target object is output as the 3D target category.

2. The 3D target detection method integrating a 3D laser radar and a monocular camera according to claim 1, wherein: The monocular camera is calibrated using Zhang Zhengyou's checkerboard calibration method.

3. The 3D target detection method integrating a 3D laser radar and a monocular camera according to claim 1, wherein: The step 2 comprises the following steps: Step 2.1, use the calibrated monocular camera to obtain the camera image, In step 2.2, the target is selected on the camera image through the target detection network to obtain the 2D target object and target category.

4. The 3D target detection method integrating a 3D laser radar and a monocular camera according to claim 1, wherein: The step 3 comprises the following steps: Step 3.1: Use the installation location of the 3D LiDAR as the coordinate origin and use the right-handed coordinate system to establish the 3D LiDAR coordinate system. Step 3.2: Set the 3D LiDAR region of interest (ROI) and extract the scan points within the ROI to form a point cloud P = {p(x, y, z) | p∈ROI}, where p(x, y, z) is the coordinate of the scan point in the 3D LiDAR coordinate system. Step 3.3: perform data denoising on the coordinate information of the point cloud in the 3D lidar coordinate system.

5. The 3D target detection method integrating a 3D laser radar and a monocular camera according to claim 4, wherein: The region of interest ROI is -R max <x<R max , and -R min <y<R min , R max R is the radius of the front and rear sides of the 3D laser radar. min is the radius of the left and right sides of the 3D laser radar, the x-axis is the forward direction of the 3D laser radar, and the x-axis and the y-axis are 90° in the horizontal plane.

6. The 3D target detection method integrating a 3D laser radar and a monocular camera according to claim 4, wherein: The data denoising comprises the following steps: Set the neighborhood radius R and search for each scan point p j (x j ,y j , z j ) in the neighborhood of point set P j ={(x, y, z)|(xx j ) 2 + (yy j ) 2≤R 2 }, where j is the index of the point cloud, j = 1, 2, 3..., P j ∈P; calculate each point set P separately through the median filtering algorithm j The median Zm on the Z axis and the median Z m Replace the corresponding scanning point p j (x j ,y j , z j ) coordinate value z on the Z axis j .

7. The 3D target detection method integrating a 3D laser radar and a monocular camera according to claim 1, wherein: The step 4 comprises the following steps: Step 4.1, initialize i = 0, randomly select three scanning points in the preprocessed point cloud as the initial plane subset points, and construct the plane; Step 4.2, determining whether the remaining scan points belong to the plane based on the distances between the remaining scan points and the plane; Step 4.3: When the number of scan points belonging to a plane is greater than the preset threshold, the plane is considered a ground point cloud plane and step 4.4 is executed; otherwise, step 4.4 is executed directly. Step 4.4: Let i = i + 1 and determine whether i > c, where c is the maximum number of iterations. If so, remove the scan points contained in the ground point cloud plane from the point cloud. Otherwise, proceed to step 4.

5. In step 4.5, randomly select three different scanning points in the preprocessed point cloud, construct a new plane, and return to step 4.2.

Images