Target detection system and method based on 4D millimeter wave radar sensor

Abstract

The invention discloses a target detection system and method based on a 4D millimeter wave radar sensor. The system comprises a radar data acquisition module, a preprocessing module, a ground filtering module, a clustering module, a bounding box calculation module, a data enhancement module, a deep learning detection module and a fusion tracking module. The method combines the advantages of the traditional clustering and deep learning methods, not only improves the detection precision, but also has good real-time performance. The Patchwork ground filtering module, the DBSCAN clustering module and the Center Point deep learning detection module work cooperatively, a high-robustness and real-time target detection system is achieved, and compared with a single method, the high-robustness and real-time target detection system has higher detection accuracy and wider adaptability.

Description

Technical Field

[0001] The present invention relates to the technical field of radar sensing and target detection, and particularly to a system and method for target detection based on a 4D millimeter-wave radar sensor. Background Art

[0002] Traditional 4D millimeter-wave radar methods have technical problems such as low accuracy and poor adaptability when detecting targets in complex environments such as autonomous driving and security monitoring. The present invention aims to solve the problems of accuracy and efficiency of current 4D millimeter-wave radar in target detection in complex scenarios. By combining traditional clustering techniques and deep learning algorithms, the present invention improves the recognition accuracy and robustness of 4D millimeter-wave radar for objects, and particularly shows significant advantages when detecting long-distance targets or low-reflection objects. Summary of the Invention

[0003] The purpose of the present invention is to provide a system and method for target detection based on a 4D millimeter-wave radar sensor, which can enhance the detection effect on the basis of effectively utilizing the speed, position and reflection information of the 4D radar, and provide an efficient and accurate perception solution for autonomous driving, security monitoring, etc.

[0004] A system for target detection based on a 4D millimeter-wave radar sensor provided by the present application, the system comprising: A radar data acquisition module, configured to acquire point cloud data of a 4D millimeter-wave radar, the point cloud data including distance, speed, azimuth and reflection intensity information of a target object; A preprocessing module, configured to preprocess the point cloud data, extract a region of interest (ROI), remove noise, and suppress multipath clutter; A ground filtering module, configured to remove ground point cloud data by using the Patchwork method to more accurately identify targets; A clustering module, configured to cluster the preprocessed point cloud data and generate point cloud clusters by using the DBSCAN algorithm; An bounding box calculation module, configured to calculate a 3D bounding box of a target according to the point cloud clusters generated by clustering to obtain target information, the target information including the size and position of the target; A data enhancement module, configured to perform data enhancement operations on sparse point cloud data; A deep learning detection module, which uses the CenterPoint network to detect the point cloud data to obtain a deep learning detection result; A fusion tracking module, configured to perform post-fusion on the target information and the deep learning detection result to generate a target data set including target object category, position and speed information; An output module, configured to output a detection result and send the information of the target object to an upper-layer control module.

[0005] As a further technical solution of the present invention, the preprocessing module includes: An ROI extraction unit for selecting points of the region of interest within a specified x, y, z range through a pass-through filter; A denoising unit for removing outliers using statistical filtering and radius filtering; A multipath suppression unit for suppressing multipath clutter generated due to the electromagnetic scattering characteristics of the radar.

[0006] As a further technical solution of the present invention, the ground filtering module uses the Patchwork method to perform plane fitting on the ground to remove ground point clouds.

[0007] As a further technical solution of the present invention, the data augmentation module includes random flipping, random rotation, and random scaling operations to enhance the generalization ability of the model to point cloud data.

[0008] As a further technical solution of the present invention, the CenterPoint network of the deep learning detection module includes: An input preprocessing module for performing voxelization processing on the input point cloud data and extracting features; A feature extraction network for extracting voxel features on multi-scale features; A center point detection head module for generating a center point heat map, regressing the size and direction offset of the 3D bounding box, and speed prediction; A post-processing module for extracting the final detection result through non-maximum suppression.

[0009] As a further technical solution of the present invention, the fusion tracking module fuses the target position in the clustering result and the category information detected by deep learning through Hungarian algorithm matching.

[0010] Another object of the present invention is to provide a method for target detection based on a 4D millimeter-wave radar sensor, and the method includes the following steps: Collect point cloud data of the 4D millimeter-wave radar, and the point cloud data includes distance, speed, azimuth, and reflection intensity information of the target object; Preprocess the point cloud data, extract the region of interest (ROI), remove noise, and suppress multipath clutter; Use the Patchwork method to remove ground point cloud data to more accurately identify the target; Cluster the preprocessed point cloud data, and use the DBSCAN algorithm to generate point cloud clusters; Calculate the 3D bounding box of the target according to the point cloud clusters generated by clustering to obtain target information, and the target information includes the size and position of the target; Perform data augmentation operations on sparse point cloud data; Use the CenterPoint network to detect the point cloud data and obtain the deep learning detection results; Perform post-fusion on the target information and the deep learning detection results to generate a target dataset containing the target object category, position, and speed information; Output the detection results and send the information of the target object to the upper control module.

[0011] The beneficial effects achieved by the present invention: The advantage of the present invention lies in combining the advantages of traditional clustering and deep learning methods, which not only improves the detection accuracy but also has good real-time performance. The Patchwork ground filtering, DBSCAN clustering, and CenterPoint deep learning detection modules work together to achieve a highly robust and real-time target detection system, which has higher detection accuracy and wider adaptability compared to single methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic flow chart of a method for target detection based on a 4D millimeter-wave radar sensor of the present invention.

[0013] Figure 2 It is a preprocessing flow chart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0014] The following describes the technical solutions of the present invention in detail with reference to specific drawings.

[0015] As Figure 1-2 shown, a system for target detection based on a 4D millimeter-wave radar sensor provided in this embodiment includes: A radar data acquisition module for acquiring the point cloud data of the 4D millimeter-wave radar. The point cloud data contains the distance, speed, azimuth, and reflection intensity information of the target object; the acquired data is organized in the PointCloud2 message format and published to the preprocessing module through an appropriate interface. Each frame of point cloud data contains multiple echo information, and the real-time performance and detection stability are ensured through frame rate control.

[0016] A preprocessing module for preprocessing the point cloud data, extracting the region of interest (ROI), removing noise, and suppressing multipath clutter; A ground filtering module for removing the ground point cloud data using the Patchwork method to more accurately identify the target; The clustering module is used to cluster the preprocessed point cloud data and generate point cloud clusters using the DBSCAN algorithm. DBSCAN does not require a preset number of clusters by setting the radius threshold and the number of neighboring points, and can exclude noise points. By detecting the density of the clustering results, close point clouds are clustered into one category for subsequent 3D bounding box calculation; DBSCAN is suitable for environments with dense or sparse point clouds and has good robustness to external points. By properly adjusting the radius and neighboring point parameters, effective detection of targets at different distances can be achieved; The bounding box calculation module is used to calculate the 3D bounding box of the target based on the point cloud cluster generated by clustering to obtain the target information, which includes the size and position of the target; 3D bounding box: For each point cloud cluster, the system fits the 3D bounding box through a search algorithm. The bounding box closest to the rectangle is selected to enclose the clustered point cloud to accurately describe the size and position of the target; it also includes convex hull calculation: the Graham scanning method is used to calculate the convex hull of the point cloud cluster, and then determine the boundary of the target. This convex hull description method is suitable for irregular objects, and can effectively reduce the calculation complexity, which is suitable for real-time processing requirements; Data enhancement module, used to perform data enhancement operations on sparse point cloud data; The deep learning detection module uses the CenterPoint network to detect point cloud data and obtain deep learning detection results; The fusion tracking module is used to post-fuse the target information with the deep learning detection results to generate a target data set containing the target object category, position and speed information; The output module is used to output the detection results and send the target object information to the upper control module; the detection results include the position (3D coordinates), category, speed and other information of each target. The system transmits the detection data to the upper control module for vehicle control or path planning. The display interface supports the visualization of the target, including 3D bounding boxes, speed arrows and category labels, providing real-time feedback to the driver or the autonomous driving system.

[0017] In this embodiment, the preprocessing module includes: ROI extraction unit, used to select the point cloud of the area of ​​interest within the specified x, y, z range through a pass-through filter; use the pass-through filter to filter the input point cloud. Set the ROI area (such as the distance range in the x, y, z directions) and extract the valid point cloud data therein. The ROI setting value is based on the requirements of the application scenario, such as the area in front and on both sides of the vehicle that needs to be paid attention to during autonomous driving; The denoising unit uses statistical filtering and radius filtering to remove outliers. Statistical filtering uses the mean and variance of neighboring points as criteria to exclude points that deviate too much from the mean. Radius filtering sets a threshold for the number of neighboring points to exclude noise points in low-density areas. The multipath suppression unit is used to suppress the multipath clutter generated by the radar electromagnetic scattering characteristics; a specific algorithm is used to suppress the false point clouds caused by the multipath effect. By calculating the movement consistency of points in adjacent frames and excluding the abnormal points caused by reflection, the authenticity and reliability of the target are improved.

[0018] In this embodiment, the ground filtering module uses the Patchwork method to perform plane fitting on the ground to remove the ground point cloud. The Patchwork ground removal algorithm is adopted. This algorithm is based on the principle of recursive partitioning and can accurately distinguish the ground and target objects in a sparse point cloud environment. This method is applicable to unstructured terrains, effectively removing the ground points and thus reducing false detections.

[0019] In this embodiment, the data augmentation module includes random flipping, random rotation, and random scaling operations to enhance the model's generalization ability for point cloud data.

[0020] Random flipping: Randomly flip the point cloud along a specific axis to increase the model's adaptability to different perspectives.

[0021] Random rotation: Randomly rotate the point cloud to enhance the model's ability to be invariant to rotation.

[0022] Random scaling: Adjust the scale of the point cloud to help the model identify targets of different sizes.

[0023] In this embodiment, the CenterPoint network of the deep learning detection module includes: The input preprocessing module is used to perform voxelization processing on the input point cloud data and extract features; specifically: voxelize the point cloud collected by the radar, divide the point cloud into multiple voxels, calculate features such as the geometric center and intensity for each voxel, and generate a sparse feature map to reduce the computational burden; The feature extraction network is used to extract voxel features on multi-scale features; specifically: use a ResNet or VGG backbone network to extract multi-scale features, and use a Feature Pyramid Network (FPN) to fuse feature maps of different resolutions to improve the detection effect of small objects; The center point detection head module is used to generate a center point heat map, regress the 3D bounding box size and direction offset, and perform speed prediction; specifically: by generating a center point heat map, predict the center position of the target, and combine the scale and offset to regress and output the size and position of the 3D bounding box. At the same time, a speed regression module is introduced to predict the speed information of the target; The post-processing module extracts the final detection results through non-maximum suppression; specifically: use non-maximum suppression (NMS) to remove redundant detection boxes to ensure the accuracy of the final output results.

[0024] In this embodiment, the fusion tracking module performs matching through the Hungarian algorithm to fuse the target positions in the clustering results with the category information detected by deep learning. By adopting a post-fusion strategy, the deep learning detection results are matched and registered with the targets obtained by clustering to obtain more accurate detection results. The matching method can select the Hungarian matching algorithm to judge the same target through information such as position and speed, and output a complete target data set including information such as category, position, and speed.

[0025] As Figure 1-2 shown, another object of the present invention is to provide a method for target detection based on a 4D millimeter-wave radar sensor, and the method includes the following steps: Collect the point cloud data of the 4D millimeter-wave radar, and the point cloud data includes the distance, speed, azimuth, and reflection intensity information of the target object; Preprocess the point cloud data, extract the region of interest (ROI), remove noise, and suppress multipath clutter; Use the Patchwork method to remove the ground point cloud data to more accurately identify the target; Cluster the preprocessed point cloud data, and use the DBSCAN algorithm to generate point cloud clusters; Calculate the 3D bounding box of the target according to the point cloud clusters generated by clustering to obtain target information, and the target information includes the target size and position; Perform data augmentation operations on the sparse point cloud data; Use the CenterPoint network to detect the point cloud data to obtain the deep learning detection results; Perform post-fusion on the target information and the deep learning detection results to generate a target data set including the category, position, and speed information of the target object; Output the detection results and send the information of the target object to the upper control module.

[0026] The 4D millimeter-wave radar target detection method provided by the present invention, which combines clustering and deep learning (RadarPillars), can effectively solve the technical problems such as low accuracy and poor adaptability existing in the traditional single method when detecting targets in complex environments such as autonomous driving and security monitoring. This method realizes the following remarkable technical effects through the fusion of technical solutions: 1. The detection accuracy is significantly improved: The present invention combines the classification capabilities of traditional clustering and centerpoint deep learning, and calculates the speed information and category information of the target respectively. Compared with using deep learning or clustering methods alone, the fused solution can improve the detection accuracy by about 10%-15%. In the test scenario, the detection accuracy of the present invention reaches more than 95%, which can effectively detect and distinguish target categories such as pedestrians and vehicles, and significantly improve the target recognition accuracy in complex scenarios.

[0027] 2. Enhanced real-time performance: By utilizing the efficient speed calculation of traditional clustering, the present invention has higher computational efficiency in target area division and speed estimation than deep learning alone. Test results show that within a range of 50 meters, the detection delay of the present invention is controlled within 50 milliseconds, meeting the high real-time requirements of autonomous driving and security monitoring, and is suitable for high-speed response application scenarios.

[0028] 3. Strong multi-target detection capability: The present invention can effectively meet the needs of multi-target detection, using the time and space information of 4D radar, as well as the radial velocity and reflection intensity of point cloud data, to cluster multiple target areas, and accurately classify them in combination with deep learning methods. Actual tests show that this method can effectively detect up to 10 or more targets in the same scene, which not only improves the detection capability, but also enhances the adaptability of the system in complex traffic environments.

[0029] 4. Strong scene adaptability and improved anti-interference ability: For harsh environments such as rain, fog, and night, the present invention relies on the anti-interference characteristics of 4D millimeter-wave radar to achieve strong scene adaptability. Test data shows that in rainy and foggy weather, the detection accuracy of the present invention is more than 20% higher than that of the traditional single method. At the same time, the robustness of the RadarPillars deep learning model when the ambient light changes also effectively improves the system's anti-interference ability in complex environments.

[0030] 5. Social and economic benefits: The present invention can be widely used in the fields of autonomous driving, security monitoring, intelligent transportation, etc., and can provide high-precision, low-latency target detection services in complex and changeable scenarios. Compared with traditional solutions, the present invention greatly improves detection accuracy and real-time performance without significantly increasing resource consumption, significantly reduces the system's false detection rate and missed detection rate, and reduces subsequent operating costs. In the field of autonomous driving, the invention can reduce the risk of traffic accidents caused by recognition errors, and has high social and economic benefits.

[0031] It should be noted that, in this article, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0032] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be included in the patent protection scope of the present invention by the same token.

Claims

1. A system for target detection based on 4D millimeter wave radar sensor, characterized in that: The system comprises: A radar data acquisition module is used to collect point cloud data of a 4D millimeter-wave radar, wherein the point cloud data includes information on the distance, speed, direction, and reflection intensity of a target object; The preprocessing module is used to preprocess the point cloud data, extract the area of ​​interest, remove noise, and suppress multipath clutter; The ground filtering module is used to remove ground point cloud data using the Patchwork method in order to more accurately identify the target; Clustering module, used to cluster the preprocessed point cloud data and generate point cloud clusters using the DBSCAN algorithm; A bounding box calculation module is used to calculate a 3D bounding box of a target according to the point cloud clusters generated by clustering, and obtain target information, wherein the target information includes the size and position of the target; Data enhancement module, used to perform data enhancement operations on sparse point cloud data; The deep learning detection module uses the CenterPoint network to detect point cloud data and obtain deep learning detection results; The fusion tracking module is used to post-fuse the target information with the deep learning detection results to generate a target data set containing the target object category, position and speed information; The output module is used to output the detection results and send the information of the target object to the upper control module.

2. The system for target detection based on 4D millimeter wave radar sensor according to claim 1, characterized in that: The preprocessing module comprises: ROI extraction unit, used to select the point cloud of the region of interest within the specified x, y, z range through a pass-through filter; Denoising unit, which uses statistical filtering and radius filtering to remove outliers; The multipath suppression unit is used to suppress the multipath clutter generated by the electromagnetic scattering characteristics of the radar.

3. The system for target detection based on 4D millimeter wave radar sensor according to claim 1, characterized in that: The ground filtering module uses the Patchwork method to perform plane fitting on the ground to remove the ground point cloud.

4. The system for target detection based on 4D millimeter wave radar sensor according to claim 1, characterized in that: The data enhancement module includes random flipping, random rotation and random scaling operations to enhance the generalization ability of the model for point cloud data.

5. The system for target detection based on 4D millimeter wave radar sensor according to claim 1, characterized in that: The CenterPoint network of the deep learning detection module includes: The input preprocessing module is used to voxelize the input point cloud data and extract features; Feature extraction network, used to extract voxel features on multi-scale features; The center point detection head module is used to generate the center point heat map, regress the 3D bounding box size and direction offset, and predict the speed; The post-processing module extracts the final detection results through non-maximum suppression.

6. The system for target detection based on 4D millimeter wave radar sensor according to claim 1, characterized in that: The fusion tracking module fuses the target position in the clustering result and the category information detected by deep learning through Hungarian algorithm matching.

7. A method for target detection based on a 4D millimeter wave radar sensor, characterized in that: The method comprises the following steps: Collecting point cloud data of 4D millimeter wave radar, wherein the point cloud data includes distance, speed, direction and reflection intensity information of the target object; Preprocess the point cloud data to extract the area of ​​interest, remove noise, and suppress multipath clutter; Patchwork method is used to remove ground point cloud data in order to identify the target more accurately; Cluster the preprocessed point cloud data and generate point cloud clusters using the DBSCAN algorithm; Calculate the 3D bounding box of the target according to the point cloud clusters generated by clustering to obtain target information, wherein the target information includes the size and position of the target; Perform data augmentation operations on sparse point cloud data; The CenterPoint network is used to detect point cloud data and obtain deep learning detection results; The target information is post-fused with the deep learning detection results to generate a target dataset containing the target object category, position and speed information; Output the detection results and send the target object information to the upper control module.

Images