Road surface throwing object detection method based on laser radar
By combining LiDAR with voxelization and multi-frame fusion of point cloud data, along with 3D clustering and target detection algorithms, the accuracy and adaptability issues of debris detection in high-speed scenarios have been solved, achieving efficient debris detection and remote early warning under different lighting conditions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 浙江海康智联科技有限公司
- Filing Date
- 2023-06-21
- Publication Date
- 2026-04-10
AI Technical Summary
Existing road debris detection solutions are not very accurate in high-speed scenarios, especially in low-light conditions at night, and are easily affected by background interference and changes in lighting, making it difficult to meet the high efficiency requirements of traffic management.
A LiDAR-based method for detecting road debris is adopted. By voxelizing point cloud data, multi-frame fusion, and background modeling, combined with 3D clustering and target detection algorithms, the method distinguishes debris and fuses detection results, and updates the background model to improve detection accuracy and adaptability.
It improves the performance of spill detection, enabling effective detection under different lighting conditions, reducing false detections, improving the adaptability and accuracy of detection, and supporting remote early warning and rapid handling of abnormal events.
Smart Images
Figure CN121831784A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of road traffic detection, in particular to a road surface litter detection method based on laser radar. BACKGROUND
[0002] In the field of road traffic, especially in high-speed scenarios, if an object is dropped during the driving process of a vehicle, it will cause great hidden dangers to the driving safety of the rear vehicle, and even cause serious casualties and property losses. From the perspective of influencing the driving safety of the vehicle, the type of litter to be detected is a non-moving target with a certain volume, such as a cuboid target, while targets such as paper sheets and plastic bags belong to the category that do not need to be detected. Since there are many types of litter, existing detection schemes cannot achieve high accuracy, and it is difficult to meet the high efficiency requirements of traffic management, so it is of great significance for traffic management to develop a reliable litter detection scheme.
[0003] In the prior art, the mainstream is a scheme for detecting road surface litter based on camera video image acquisition, which generally adopts a Gaussian background modeling method. However, due to the variety of litter itself, and the problems of ineffective detection under low-illumination conditions at night, background interference, and large influence of illumination changes on detection effect, the performance of the detection scheme is poor. SUMMARY
[0004] According to the problems in the background art, the present application provides a road surface litter detection method based on laser radar to solve the problems, and the present application will be further described.
[0005] A road surface litter detection method based on laser radar, comprising:
[0006] Background modeling, input laser radar point cloud data, after single-frame point cloud voxelization calculation and multi-frame point cloud voxel fusion, grid frequency filtering is performed, and the weighted average of the obtained point cloud is obtained Point cloud background voxel;
[0007] Litter detection, difference according to voxel features and background voxel features, obtain new point cloud features, adopt 3D clustering method to obtain 3D target contour, through frame-by-frame point cloud detection to obtain multi-frame detection result, based on the detection result, gradually fuse the detection area, and output the final litter detection result according to the appearance probability after counting the appearance probability histogram;
[0008] Further, the litter detection further comprises background modeling update. If the litter is detected, and the application platform judges that it is not litter, the point cloud background modeling update is started; if it is judged as litter, the ROI area of the detected litter is updated with the average result of the first number of frames of voxels in the detection history as the new point cloud background.
[0009] Furthermore, background modeling includes the following steps in sequence: The input is LiDAR point cloud data, which must include at least four dimensions: (x, y, z, intensity). Single-frame point cloud voxelization calculation divides the image into an M*N grid, with each grid having pixel sizes of dx and dy. For 3D point clouds, a Z-axis division is added, with each grid having physical sizes of dx, dy, and dz, thus dividing the entire 3D point cloud into an M*N*P 3D grid. Multi-frame point cloud voxel fusion: The point cloud voxelization result of one frame is extracted at equal intervals, and the result of the first number of frames is fused to obtain four-dimensional data. Point cloud background modeling: Based on the result of fusing multiple frame point cloud voxels, a threshold Thres is used for grid frequency filtering. The filtered point clouds are then weighted and averaged to obtain the point cloud background voxels. Furthermore, the detection of spilled material includes the following steps in sequence: For single-frame debris detection, a point cloud 3D target detection algorithm is used to detect traffic targets. If the detected traffic target's position coincides with the debris, it is judged as a false detection. By inputting the 3D voxel features of the point cloud, vector_cur, the voxel features and background voxel features are subtracted to obtain new point cloud features. A 3D clustering method is used to obtain the outline of the 3D target; vector_obj=Max(0,(vector_cur-vector_back)); Multi-frame fusion discrimination is performed by detecting point clouds frame by frame to obtain detection results for multiple frames. Based on the detection results, the detection areas are gradually fused. After statistically analyzing the target occurrence probability histogram, the final debris detection result is output according to the occurrence probability. Beneficial effects: Compared with the prior art, the present invention is based on a voxel-based multi-frame background modeling method, which integrates multi-frame discrimination information during the identification of spilled objects, effectively improving the performance of spilled object detection; and the method of updating the background modeling content of the algorithm based on the manually judged spilled object detection results effectively improves the adaptability of the spilled object detection scheme to the deployment scenario. Attached Figure Description Figure 1 : Flowchart of the spill detection process of this invention; Figure 2 Single-frame projectile detection process. Detailed Implementation The following detailed description of a specific embodiment of the present invention is based on the accompanying drawings. A method for detecting road debris based on lidar is disclosed. This method utilizes a road debris detection system comprising three parts: a lidar debris detection module, an image capture module, and a platform push module. The image capture module is configured to capture images of the debris, and the platform push module is configured to transmit the captured images to the lidar debris detection module. The lidar debris detection module assists traffic managers in identification and early warning. The lidar-based road debris detection method consists of three main modules: background modeling, debris detection, and background model updating. (See attached reference.) Figure 1 The detailed implementation steps are as follows. Step S1, background modeling, includes the following steps: S11, Input LiDAR point cloud data, the point cloud data includes at least four dimensions (x, y, z, intensity); S12, Single-frame point cloud voxelization calculation. Taking a two-dimensional image as an example, the two-dimensional image can be divided into an M*N grid, with the pixel size of each grid being dx and dy. Similarly, for a three-dimensional point cloud, the division along the Z-axis is added, and the physical size of each grid is set to dx, dy, and dz. In this way, the entire three-dimensional point cloud can be divided into an M*N*P three-dimensional grid. In this embodiment, in order to improve the detection accuracy, (dx, dy, dz) is preferably set to (0.1, 0.1, 0.1). S13, multi-frame point cloud voxel fusion. Taking the point cloud acquisition frame rate as 10 Hz as an example, the point cloud voxelization result is extracted at equal intervals of 5 frames, and 10 frames of results are fused. The fused four-dimensional data includes three-dimensional mesh and one-dimensional frequency. S14, point cloud background modeling: based on the result of fusing multiple point cloud voxels, a threshold Thres is used for grid frequency filtering. The threshold is preferably set to 3. The filtered point cloud is weighted and averaged to obtain the point cloud background voxels. Step S2, spill detection, includes the following steps: S21, Single-frame debris detection, including debris detection and point cloud-based traffic target detection methods. Even on highways, stationary traffic targets such as vehicles may exist, potentially leading to false detections of debris. Therefore, this invention also employs a point cloud 3D target detection algorithm to detect common traffic targets, as detailed in the appendix. Figure 2 This includes, but is not limited to, deep learning detection algorithms such as CenterPoint and PointPillar. Detection targets include vehicles, pedestrians, non-motorized vehicles, traffic cones, etc. If the location of the detected traffic target coincides with the location of the spilled object, it is considered a false detection. For projectile detection, the input point cloud 3D voxel features vector_cur are used. The voxel features and background voxel features are subtracted to obtain new point cloud features. Based on these features, 3D clustering methods such as DBSCAN are used to obtain the outline of the 3D target; vector_obj = Max(0,(vector_cur-vector_back)). S22, Multi-frame fusion and discrimination: By detecting point clouds frame by frame, detection results of multiple frames can be obtained. Based on the detection results, the detection areas are gradually fused. After statistically analyzing the target occurrence probability histogram, the final debris detection result is output according to the occurrence probability. Detailed steps include: S221, In the event of a single-frame projectile target, the data buffer records the detection results of the first frame, including the center point position and size; S222, in the point cloud detection results of the 1+Nth frame, N is recommended to be 49, for a total of 50 frames. Match the center point of the detection result of each frame with the first frame to check whether the center point of the new result is included in the historical 3D ROI. If it is, increment the target occurrence count by 1; otherwise, list it as a new target. S223, Calculate the histogram of the detection results of the first 50 frames. If the probability of the appearance of the projectile target is greater than the threshold Thres2, it is recommended to set Thres2 to 35. Then it is considered that the projectile target has been detected. S224, the data buffer clears the detection result of the first frame and repeatedly adds the detection result of subsequent frames to ensure that there are 50 frames of results in the buffer; S225, If the target result is larger than the set spill size threshold, the spill event information is reported. S3, background modeling updated. Point cloud background modeling update: If a projectile is detected, the system will push it to the application platform. If the application platform determines that it is not a projectile, it will start the point cloud background modeling update to avoid repeated detection. The point cloud background modeling update will update the voxel average result of the detection history of 50 frames in the ROI area where the projectile was detected, and use it as the new point cloud background. This invention can effectively detect debris on the road surface and capture images using a camera. The images are then sent to a platform for remote confirmation and warning. Upon confirmation, the platform can further push the warning information to advance warning signs on highways, elevated roads, and other road sections, enabling rapid handling of such abnormal events. Compared to traditional image-based debris detection methods, this invention has the following advantages: It can detect spilled materials during both daytime and low-light nighttime conditions; A custom-designed debris detection algorithm can effectively avoid interference from traffic targets, changes in lighting, and other factors. Supports manual identification of false positives for spilled materials, updates the background modeling content of the algorithm, and improves algorithm performance. The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for detecting road debris based on lidar, characterized in that, include: Background modeling involves inputting LiDAR point cloud data, which is then processed through single-frame point cloud voxelization calculation, multi-frame point cloud voxel fusion, and grid frequency filtering. The resulting point cloud is then weighted and averaged to obtain the point cloud background voxels. For debris detection, new point cloud features are obtained by differentiating voxel features and background voxel features. The outline of the 3D target is obtained by using a 3D clustering method. The detection results of multiple frames are obtained by frame-by-frame point cloud detection. Based on the detection results, the detection areas are gradually merged. After statistically analyzing the target occurrence probability histogram, the final debris detection result is output according to the occurrence probability.
2. The method for detecting road spills according to claim 1, characterized in that, The process of detecting spilled material also includes a background modeling and updating step. If spilled material is detected, but the application platform determines that it is not spilled material, then the point cloud background modeling and updating will be initiated. If the object is identified as a spill, the voxel average of the first number of frames in the detection history of the ROI region where the spill was detected is updated as the new point cloud background.
3. The method for detecting road spills according to claim 2, characterized in that, The background modeling steps include the following steps in sequence: The input is LiDAR point cloud data, which must include at least four dimensions: (x, y, z, intensity). Single-frame point cloud voxelization calculation divides the image into an M*N grid, with each grid having pixel sizes of dx and dy. For 3D point clouds, a Z-axis division is added, with each grid having physical sizes of dx, dy, and dz, thus dividing the entire 3D point cloud into an M*N*P 3D grid. Multi-frame point cloud voxel fusion: The point cloud voxelization result of one frame is extracted at equal intervals, and the result of the first number of frames is fused to obtain four-dimensional data. Point cloud background modeling: Based on the result of fusing multiple frame point cloud voxels, a threshold Thres is used for grid frequency filtering. The filtered point clouds are then weighted and averaged to obtain the point cloud background voxels.
4. The method for detecting road spills according to claim 3, characterized in that, The detection of spilled material includes the following steps: For single-frame debris detection, a point cloud 3D target detection algorithm is used to detect traffic targets. If the detected traffic target's position coincides with the debris, it is judged as a false detection. By inputting the 3D voxel features of the point cloud, vector_cur, the voxel features and background voxel features are subtracted to obtain new point cloud features. A 3D clustering method is used to obtain the outline of the 3D target; vector_obj=Max(0,(vector_cur-vector_back)); Multi-frame fusion discrimination is performed by detecting point clouds frame by frame to obtain detection results for multiple frames. Based on the detection results, the detection areas are gradually fused. After statistically analyzing the target occurrence probability histogram, the final debris detection result is output according to the occurrence probability.
5. The method for detecting road spills according to claim 4, characterized in that, The multi-frame fusion and discrimination step includes the following steps: Given the presence of a single-frame projectile target, the data buffer records the detection results of the first frame, including the center point position and size; In the point cloud detection results of frames 1+N, the center point of the detection result of each frame is matched with the first frame to check whether the center point of the new result is included in the historical 3D ROI. If it is, the target occurrence count is incremented by 1; otherwise, it is listed as a new target. Analyze the histogram of detection results for the first 1+N frames. If the probability of a projectile target appearing is greater than the threshold Thres2, it is considered that a projectile target has been detected. The data buffer clears the detection result of the first frame and repeatedly adds the detection result of subsequent frames to ensure that there are 1+N frames of results in the buffer; If the target result is larger than the set size threshold for the spilled material, the spilled material event information is reported.