Target detection method based on density filtering and multi-point color feature recognition

Abstract

The invention relates to the field of automatic driving target detection, and discloses a target detection method based on density filtering and multi-point color feature recognition, which comprises the following steps: carrying out point cloud filtering based on density features; filtering a point cloud target frame based on color features; and target fusion based on IOU. According to the point cloud filtering method based on the density, a plane is fitted through point clouds in voxels, density evaluation parameters are calculated, dust raising point clouds are generally discrete, the density evaluation parameters are large, ground point clouds, vehicle point clouds and other object point clouds are generally compact, and the density evaluation parameters are small, so that most dust raising point clouds can be filtered out through a threshold value; and the interference of dust raising factors is inhibited. According to a point cloud filtering result, color xyz coordinates are provided, a color coordinate system is constructed, rules are set on the color coordinate system to remove a point cloud target frame formed by flying dust clustering, a point cloud target frame of a vehicle and other targets is reserved, and the detection precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving target detection, and particularly to a target detection method based on density filtering and multi-point color feature recognition. Background Art

[0002] The accuracy of the autonomous driving perception system determines the performance of its downstream decision-making system and control system, and has an important impact on the safe driving of autonomous vehicles. LiDAR is the main sensor of the perception system, and target detection is the main task of the perception system. Under clear weather conditions, there are many point cloud target detection algorithms such as pvrcnn, pointpillar, and second. However, under dusty conditions, the performance of these target detection algorithms is generally low. Under dusty conditions, the main target detection methods include the LIOR method, the DROR method, and the LIDROR method, etc. These methods are all based on outlier analysis, combined with the characteristics of reflection intensity, to filter the point cloud. However, these methods are all effective in slightly dusty scenarios and cannot be applied to moderately dusty environments. Currently, there is no effective technical solution to solve the problem of low accuracy of laser point cloud target detection in moderately dusty scenarios. Summary of the Invention

[0003] To solve the above problems existing in the prior art, the present invention aims to design a target detection method based on density filtering and multi-point color feature recognition that can improve the accuracy of laser point cloud target detection in moderately dusty scenarios.

[0004] To achieve the above object, the technical solution of the present invention is as follows: A target detection method based on density filtering and multi-point color feature recognition, comprising the following steps:

[0005] A: Point cloud filtering based on density features

[0006] A1: Point cloud voxelization

[0007] Using the octree segmentation algorithm, the point cloud is segmented into n voxels of the same size, and the side length of each voxel is L;

[0008] A2: Let the voxel serial number i = 1;

[0009] A3: Search for neighboring point clouds

[0010] With R as the search radius and the center point Ci of the i-th voxel as the center of the sphere, a sphere is formed to search for all the point clouds within the neighborhood of the sphere.

[0011] A4: Fit a plane

[0012] Perform a plane fitting on the point cloud searched in step A2, use SVD singular value decomposition on the array composed of the point cloud coordinates, obtain the normal vector λi, and the plane perpendicular to λi is the fitting plane Pi.

[0013] A5: Calculate the density evaluation parameter

[0014] Calculate the sum of the distances from all points within the sphere to the fitting plane Pi as the density evaluation parameter DPi;

[0015] A6: If i = n, go to step A7; otherwise, let i = i + 1 and go to step A3;

[0016] A7: Calculate the global normal vector

[0017] The global normal vector is the mean value λN of the normal vectors of all voxels, calculated using the formula λN = (Σλi) / n, where Σλi is the sum of the normal vectors of all voxels.

[0018] A8: Let the voxel number i = 1;

[0019] A9: Threshold filtering

[0020] Set the dust segmentation threshold DP_thod according to experience. The point clouds within the voxels where DPi is lower than DP_thod are all classified as non-dust regions. The non-dust regions include object point clouds and ground point clouds, and the point clouds within the voxels not lower than DP_thod are filtered out. Since the ground point clouds are generally flat and their voxels are not filtered out, set the ground segmentation threshold GP_thod according to experience. The point clouds within the voxels lower than GP_thod are removed, and the point clouds within the voxels not lower than GP_thod are retained. The object point clouds include the point clouds of people, vehicles, and buildings.

[0021] After screening by the above method, there are still ground point clouds corresponding to rough ground in the retained point clouds that are not filtered out. However, the normal vectors of the ground point clouds within the voxels generally point in a certain direction, which is basically the same as the direction of the normal vectors of the large-scale point clouds in the off-road scenario. Set the normal vector threshold λ_thod. If the angle between the voxel normal vector λi and the global normal vector λN is less than the normal vector threshold λ_thod, it proves that the voxel normal vector is consistent with the global normal vector, and then the point cloud within this voxel is removed; otherwise, the point cloud within this voxel is retained for subsequent processing.

[0022] A10: Clustering

[0023] Use the Euclidean clustering method to cluster the point clouds filtered in step A9, and cluster out the 3D cube bounding box of the point clouds, which is the point cloud target box.

[0024] A11: If i = n, go to step B; otherwise, let i = i + 1 and go to step A9;

[0025] B: Filter the point cloud target box based on color features

[0026] Convert the color image to a grayscale image, calculate the center point coordinates of the six planes of the point cloud target box respectively, and calculate the pixel values of the grayscale image coordinates corresponding to the six center point coordinates through the 3D to 2D projection formula. After projecting to the image plane, the center coordinate groups of the three unoccluded planes are selected to form a color xyz coordinate. Among them, the pixels corresponding to the coordinates projected on the two planes corresponding to the longest axis of the point cloud target box are the color x coordinates, the pixels corresponding to the coordinates projected on the upper and lower planes are the color z coordinates, and the pixels corresponding to the last remaining plane are the color y coordinates.

[0027] Among the color xyz coordinates thus formed, for the point cloud target box formed by clustering the unfiltered dust point clouds, the pixel grayscale values corresponding to the three selected center points are relatively close and are distributed near the diagonal. In the object point cloud target, due to the different materials of each part of the vehicle, the pixel grayscale values corresponding to the three selected center points are quite different, not located near the diagonal, and are distributed in an annular belt area. Therefore, with the diagonal as the normal and the direction perpendicular to the diagonal as the coordinate plane, project the color xyz coordinates into the color polar coordinates. Set the maximum color screening threshold CF_thod1 and the minimum color screening threshold CF_thod2 according to experience. Retain the point cloud target boxes within the annular belt area surrounded by the two thresholds, and filter out the point cloud target boxes outside the annular belt area. The unfiltered point cloud target boxes are the filtering results of the point cloud target boxes.

[0028] C. Object fusion based on intersection over union (IOU)

[0029] Use yolov8 to detect the image to obtain the detection image target box. Project the filtered point cloud target box in step B onto the image plane, calculate the intersection over union (IoU) between the point cloud target box and the image target box. If there is no image target in yolov8, directly output the point cloud target box in step B as the final result. Otherwise, filter out the point cloud target boxes with an IoU less than 0.5, and output the remaining point cloud target boxes as the final result.

[0030] Furthermore, the voxel side length L in step A1 is obtained from experience, and its value range is 0.2 - 0.5 m.

[0031] Furthermore, the search radius R in step A3 is obtained from experience, and its value range is

[0032] Further, the dust segmentation threshold DP_thod, the ground segmentation threshold GP_thod, and the normal vector threshold λ_thod described in step A6 are all obtained by statistically analyzing the data of the first 100 frames of a certain scene. According to the data of the first 100 frames, the density evaluation parameters of each voxel in each frame and the angle between the normal vector and the global normal vector are respectively statistically analyzed. Dust voxels, object voxels, and ground voxels are manually selected, and the ranges of the density evaluation parameters of the dust voxels, object voxels, and ground voxels are statistically analyzed. The middle value of the overlapping section of their distribution ranges is taken as the threshold.

[0033] Further, the maximum color screening threshold CF_thod1 and the minimum color screening threshold CF_thod2 described in step B are also obtained from the data of the first 100 frames. The point cloud target box is manually marked as belonging to dust or an object. The color polar coordinates corresponding to all object point clouds are uniformly distributed in an annular region. The outer boundary of this annular region is the maximum color screening threshold CF_thod1, and the inner boundary is the minimum color screening threshold CF_thod2.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. The present invention proposes a density-based point cloud filtering method. By fitting a plane to the point cloud within a voxel and calculating the density evaluation parameters, since the dust point cloud is usually relatively discrete and its density evaluation parameters are large, while the ground point cloud and the point cloud of objects such as vehicles are usually relatively compact and their density evaluation parameters are small, most of the dust point clouds can be filtered out through the threshold, suppressing the interference of dust factors.

[0036] 2. For the point cloud filtering result, the point clouds that are not filtered out cleanly will also form a target box after clustering. Obviously, this is not the target box we need. Moreover, since the pixel values of dust in the image are relatively close, the present invention proposes to construct a color coordinate system with color xyz coordinates and set rules on the color coordinate system to remove the point cloud target box formed by dust clustering and retain the point cloud target box of targets such as vehicles, improving the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a schematic flowchart of the present invention.

[0038] Figure 2 is a schematic flowchart of point cloud filtering based on density features.

[0039] Figure 3 is a polar coordinate representation diagram of color features. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] The present invention will be further described below with reference to the drawings. As Figure 1As shown in the figure, a target detection method based on density filtering and multi-point color feature recognition includes the following steps:

[0041] A. Point cloud filtering based on density features

[0042] The steps of point cloud filtering based on density features are as Figure 2 shown. In step A9, the threshold of the dust density evaluation parameter is 0.7 - 5.8, the range of the object density evaluation parameter is 0.02 - 0.9, and the overlapping area is 0.7 - 0.9. Therefore, the DP_thod threshold is selected as 0.8, and the GP_thod and λ_thod are also obtained by this method.

[0043] B. Point cloud target box filtering based on color features

[0044] The steps of point cloud target box filtering based on color features are as Figure 3 shown. The image used is converted from a color image to a grayscale image.

[0045] C. Target fusion based on intersection over union

[0046] The content in the above steps is the same as the content of the invention, and will not be repeated here.

[0047] The above describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements fall within the scope of the present invention claimed.

Claims

1. A target detection method based on density filtering and multi-point color feature recognition, characterized in that: Including the following steps: A: Point cloud filtering based on density features A1: Point cloud voxelization Using the octree segmentation algorithm, the point cloud is segmented into n voxels of the same size, and the side length of each voxel is L; A2: Let the voxel serial number i = 1; A3: Search for neighboring point clouds Taking R as the search radius and the center point Ci of the i-th voxel as the center of the sphere to form a sphere, and searching for all point clouds within the neighborhood of the sphere; A4: Fit a plane Performing a fitting plane on the point cloud searched in step A2, using the SVD singular value decomposition of the array composed of the point cloud coordinates to obtain the normal vector λi, and the plane perpendicular to λi is the fitting plane Pi; A5: Calculate the density evaluation parameter Calculating the sum of the distances from all points within the sphere to the fitting plane Pi as the density evaluation parameter DPi; A6: If i = n, go to step A7; otherwise, let i = i + 1 and go to step A3; A7: Calculate the global normal vector The global normal vector is the mean value λN of the normal vectors of all voxels, calculated using the following formula λN = (Σλi) / n, where Σλi is the sum of the normal vectors of all voxels; A8: Let the voxel serial number i = 1; A9: Threshold filtering Setting the dust segmentation threshold DP_thod according to experience. The point cloud within the voxel where DPi is lower than DP_thod is classified as the non-dust area, and the non-dust area includes object point clouds and ground point clouds. The point cloud within the voxel not lower than DP_thod is filtered out; since the ground point cloud is generally flat and its voxels are not filtered out, therefore, setting the ground segmentation threshold GP_thod according to experience, the point cloud within the voxel lower than GP_thod is removed, and the point cloud within the voxel not lower than GP_thod is retained; the object point cloud includes the point clouds of people, vehicles, and buildings; After screening by the above method, there are still ground point clouds corresponding to rough ground in the retained point cloud that are not filtered out, but the normal vectors of the ground point clouds within the voxel generally point in a certain direction, which is basically the same as the normal vector direction of the large-scale point cloud in the off-road scenario. Setting the normal vector threshold λ_thod, if the angle between the voxel normal vector λi and the global normal vector λN is less than the normal vector threshold λ_thod, it proves that the voxel normal vector is consistent with the global normal vector, and then the point cloud within this voxel is removed, otherwise, the point cloud within this voxel is retained for subsequent processing; A10: Clustering Using the Euclidean clustering method to cluster the point cloud filtered in step A9, and clustering the 3D cube outer frame of the point cloud, which is the point cloud target frame; A11: If i = n, go to step B; otherwise, let i = i + 1 and go to step A9; B: Point cloud target frame filtering based on color features Convert the color image to a grayscale image, calculate the center point coordinates of the 6 planes of the point cloud target box respectively, and calculate the pixel values of the grayscale image coordinates corresponding to the 6 center point coordinates through the 3D to 2D projection formula; after selecting the projection to the image plane, the center coordinates of the three unoccluded planes form a color xyz coordinate. Among them, the pixels corresponding to the coordinates projected on the two planes corresponding to the longest axis of the point cloud target box are the color x coordinates, the pixels corresponding to the coordinates projected on the upper and lower planes are the color z coordinates, and the pixels corresponding to the last remaining plane are the color y coordinates; Among the color xyz coordinates thus formed, for the point cloud target box formed by the clustering of the dust point clouds that have not been filtered out cleanly, the pixel grayscale values corresponding to the three selected center points are relatively close and are distributed near the diagonal; in the object point cloud target, since the materials of different parts of the vehicle are different, the pixel grayscale values corresponding to the three selected center points are quite different, not located near the diagonal, and are distributed in an annular belt area; therefore, with the diagonal as the normal and the direction perpendicular to the diagonal as the coordinate plane, project the color xyz coordinates into the color polar coordinates; set the maximum color screening threshold CF_thod1 and the minimum color screening threshold CF_thod2 according to experience. Retain the point cloud target boxes within the annular belt area surrounded by the two thresholds, and filter out the point cloud target boxes outside the annular belt area. Then the unfiltered point cloud target boxes are the filtering results of the point cloud target boxes; C. Object fusion based on intersection over union (IOU) Use yolov8 to detect the image to obtain the detection image target box, project the filtered point cloud target box in step B to the image plane, calculate the intersection over union IoU between the point cloud target box and the image target box. If there is no image target in yolov8, directly output the point cloud target box in step B as the final result. Otherwise, filter out the point cloud target boxes with an intersection over union IoU less than 0.5, and output the remaining point cloud target boxes as the final result.

2. The object detection method based on density filtering and multi-point color feature recognition according to claim 1, characterized in that: The voxel side length L described in step A1 is obtained from experience, and its value range is 0.2 - 0.5 m.

3. The object detection method based on density filtering and multi-point color feature recognition according to claim 1, wherein: The search radius R described in step A3 is obtained empirically and its value range is 4. The object detection method based on density filtering and multi-point color feature recognition according to claim 1, characterized in that: The dust segmentation threshold DP_thod, the ground segmentation threshold GP_thod, and the normal vector threshold λ_thod described in step A6 are all obtained from the statistics of the first 100 frames of data in a certain scene. According to the first 100 frames of data, respectively count the density evaluation parameters of each voxel in each frame and the angle between the normal vector and the global normal vector. Manually select the dust voxels, object voxels, and ground voxels, and count the density evaluation parameter ranges of the dust voxels, object voxels, and ground voxels. Take the middle value of the overlapping section of their distribution ranges as the threshold.

5. The object detection method based on density filtering and multi-point color feature recognition according to claim 1, characterized in that: The maximum color screening threshold CF_thod1 and the minimum color screening threshold CF_thod2 described in step B are also obtained from the first 100 frames of data. Manually mark whether the point cloud target box belongs to dust or an object. The color polar coordinates corresponding to all object point clouds are distributed in an annular area. The outer boundary of this annular area is the maximum color screening threshold CF_thod1, and the inner boundary is the minimum color screening threshold CF_thod2.

Images