3D target detection method based on millimeter wave radar

Abstract

The present invention proposes a method for three-dimensional target detection using only millimeter-wave radar. A variety of algorithms are designed for the sparse point cloud of millimeter-wave radar. In order to remove irrelevant environmental information, a background filtering algorithm based on background subtraction is proposed. By frame-by-frame matching and differential point clouds, background information is removed while retaining the maximum number of valid points. Due to physical limitations such as occlusion, it is impossible to directly obtain the target center point from the existing point cloud. The present invention proposes a target center generation algorithm based on Hough voting, which expands the valid data and helps to generate a more accurate initial bounding box. In order to utilize feature information of different scales and capture feature structures of different granularities, a multi-scale feature fusion network based on the attention mechanism is proposed to extract global and local features respectively, and comprehensively utilize feature information at different levels to enhance the feature expression of the target and improve the target detection effect.

Description

Technical Field

[0001] The present invention relates to the fields of autonomous driving and target detection, and is a 3D target detection method based on millimeter wave radar. Background Art

[0002] Due to its small size, easy installation, strong penetration and other advantages, millimeter-wave radar has become a powerful sensing solution in autonomous driving target detection methods. Due to cameras and lidars, previous millimeter-wave radar target detection algorithms did not make full use of sparse point clouds, and ignored their distribution characteristics, so that they did not pay attention to the commonalities and differences between foreground points and background points. At the same time, the neglect of the statistical properties of point clouds also made it insufficiently consider the important role of the center of the object. In addition, previous anchor-based methods may generate too many initial bounding boxes, which are greatly different from the actual target position, increasing the model calculation amount and bringing the risk of false detection.

[0003] The present invention comprehensively considers the distribution characteristics and statistical properties of point clouds, and designs for the sparse point clouds of millimeter-wave radar to improve the target detection effect. First, a background filtering algorithm based on background subtraction is designed to eliminate irrelevant environmental information. Secondly, according to the idea of ​​Hough voting, the center position of the target is estimated, so as to limit the position of the target, so as to achieve the purpose of expanding data and generating a more accurate initial bounding box. Finally, a multi-scale feature fusion method is designed, and the attention mechanism is introduced to fully learn the global and local features, so as to perform more accurate target detection. Summary of the invention

[0004] The purpose of the present invention is to provide a 3D target detection method based on millimeter wave radar, which includes the following steps:

[0005] Background filtering algorithm based on background subtraction: Through frame-by-frame matching and differential point cloud, more background information is excluded while retaining the maximum number of valid points. First, the number of points collected by the millimeter-wave radar in multiple frames of each scene is calculated, and the frame with the least number of points in the current scene is obtained. The background point threshold is set to filter the background points. If the number of points contained in the frame with the least number of points in the current scene is less than the threshold, it is considered to be background information, and repeated background points are filtered frame by frame.

[0006] Target center generation algorithm based on Hough voting: Use the idea of ​​Hough voting to estimate the target center of mass, and perform statistical analysis on the votes in the Hough space based on the distribution characteristics of the points. First, set up a specific Hough space for voting, then create a vote for each point in the point cloud, so that the grid point closest to it gets the vote, perform voting peak detection on the grid point, and perform inverse mapping to the point cloud space, so that the center position of the object can be estimated.

[0007] Multi-scale feature fusion method based on attention mechanism: Use self-attention mechanism to fuse global and local features to obtain a more comprehensive feature representation. Use feature extraction network to extract global features. Use multi-scale information local grouping strategy to extract local features, use the farthest point sampling algorithm for point cloud, and group the point cloud with the sample as the center, and divide it into multiple smaller areas containing local information, so as to extract multi-scale features. Fusion of global features and local features, using self-attention mechanism to make it focus on the most important information of the target.

[0008] Beneficial effects of the invention: The invention provides a 3D target detection method based on millimeter wave radar, and designs multiple algorithms for the sparse point cloud of millimeter wave radar to improve the target detection performance. The background filtering module based on background subtraction can effectively remove environmental information without losing foreground information; the target center generation module based on Hough voting can expand the effective data and help generate a more accurate initial bounding box; the multi-scale feature fusion module based on the attention mechanism comprehensively utilizes information at different levels, fully improves the utilization efficiency of features, and improves the performance of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is the overall flow chart of the present invention;

[0010] Figure 2 This is a feature fusion network model diagram of the present invention. DETAILED DESCRIPTION

[0011] The following describes the embodiments of the present invention in conjunction with the accompanying drawings.

[0012] like Figure 1 As shown, the 3D target detection method based on millimeter wave radar of the present invention includes three modules: a background filtering module based on background subtraction, a target center generation module based on Hough voting, and a multi-scale feature fusion module based on attention mechanism.

[0013] In the background filtering module based on background subtraction, the point cloud is matched and differentiated frame by frame, and more irrelevant background points are filtered out while retaining the maximum number of valid points. The specific operation is to calculate the number of points collected by the millimeter-wave radar in multiple frames of the current scene, and compare the minimum number of points with the set background point threshold. The point cloud with a number of points less than the background point threshold is regarded as background information, and repeated background information filtering is performed frame by frame.

[0014] Specifically, for the mth scene collected by the radar, the n-frame point cloud set It can be expressed as the following formula: It represents the nth frame point cloud data of the mth scene.

[0015]

[0016] For point cloud matching and filtering between radar data frames, we first need to determine the number of points contained in each frame of point cloud data, so that:

[0017]

[0018] The count function is used to calculate the number of radar points in the frame. From the above formula, we can get the point set P m , expressed as the following formula, where It represents the number of points in the nth frame of the mth scene.

[0019]

[0020] The minimum number of points in this scenario is:

[0021]

[0022] Set a threshold for the number of background information points. When the number of points in the frame with the least number of points in this scene is less than the set threshold, it is considered to be the background information in the current scene, and duplicate background points are filtered out frame by frame. If it is less than the set threshold, other data in this scene will be compared and filtered to remove background information. The details are as follows:

[0023]

[0024] The compare function is used to compare two frames of data and filter out duplicate points.

[0025] In the target center generation module based on Hough voting, the idea of ​​Hough voting is used to estimate the center position of the target. Specifically, a Hough space for voting is first created so that each point in the point cloud votes for the discrete grid point closest to it in the Hough space. The voting peak is detected on the grid points in the Hough space and inversely mapped back to the point cloud space, thereby estimating the center position of the target.

[0026] like Figure 2 As shown in the figure, in the multi-scale feature fusion module based on the attention mechanism, the global features and local features of the point cloud are processed separately. The multi-scale local area grouping strategy is used to adopt the farthest point sampling algorithm for the point cloud, and the point cloud is locally grouped with the sampled sample as the center to obtain multiple smaller areas containing local information, so as to obtain local features through feature extraction, fuse them with global features, and use the attention mechanism to learn important information in the features.

[0027] The points in the initial bounding box are pooled with features of the region of interest to obtain a single representative feature of each anchor box, and then a mapping of the representative features of the anchor box to the confidence of each box is obtained. The network will learn to select the corresponding high-confidence anchor box and use the confidence value to perform non-maximum suppression (NMS) sampling to remove redundant boxes that overlap highly with the high-confidence box. Since the size of the initial bounding box uses a predefined fixed value, the bounding box needs to be further refined to obtain the final bounding box prediction.

[0028] The bounding box classification problem in the network of this patent is a binary classification problem using cross entropy loss, and the loss function is as follows:

[0029]

[0030] Among them, y i is the ground truth, p i is the predicted confidence value. The refinement of the bounding box is a regression problem, for which the Smooth-L1 loss is used as follows:

[0031]

[0032] Where r and r' are the true value and regression refinement value of each parameter respectively.

Claims

1. A three-dimensional target detection method based on millimeter wave radar, characterized in that: The following steps are involved: Step 1: Filter the background information of the point cloud data collected by the millimeter wave radar; Step 2: Estimate the target center point position of the target in the point cloud data; Step 3: Extract and fuse features through the designed target detection network model, and train and verify the network model; Step 4: Apply the trained network model to 3D object detection; The background information filtering method described in step 1 excludes more background information while retaining the maximum number of valid points through frame-by-frame matching and differential point clouds; first, the number of points collected by the millimeter-wave radar in multiple frames of each scene is calculated, and the frame with the least number of points in the current scene is obtained by counting; Set the background point threshold to filter the background points. If the number of points contained in the frame with the least number of points in the current scene is less than the threshold, it is considered as background information, and the repeated background points are filtered frame by frame. For the mth scene collected by the radar, the n-frame point cloud set It is expressed as the following formula: It represents the nth frame point cloud data of the mth scene; For point cloud matching and filtering between radar data frames, we first need to determine the number of points contained in each frame of point cloud data, so that: The count function is used to calculate the number of radar points in the frame, and the point set is obtained from the above formula. , expressed as the following formula, where It represents the number of points in the nth frame in the mth scene; The minimum number of points in this scenario is: Set a threshold for the number of background information points. When the number of points in the frame with the least number of points in this scene is less than the set threshold, it is considered to be the background information in the current scene, and the repeated background points are filtered out frame by frame. If it is less than the set threshold, other data in this scene will be compared and filtered, as shown below: Among them, the compare function is used to compare two frames of data and filter out duplicate points; The target center point position estimation method described in step 2 uses the idea of ​​Hough voting to estimate the target center of mass, and performs statistical analysis on the votes in the Hough space according to the distribution characteristics of the points; first, a Hough space is set up for voting, and then a vote is created for each point in the point cloud, so that the grid point closest to it obtains the vote, and the voting peak is detected for the grid point and inversely mapped back to the point cloud space to estimate the center position of the object; The target detection network model described in step 3 uses the self-attention mechanism to fuse global and local features to obtain a more comprehensive feature representation; uses a feature extraction network to extract global features; uses a multi-scale information local grouping strategy to extract local features, uses the farthest point sampling algorithm for the point cloud, and groups the point cloud with the sampled point as the center, dividing it into multiple smaller areas containing local information, thereby realizing multi-scale feature extraction; fuses the global features with the local features, and uses the self-attention mechanism to make it focus on the most important information of the target; The points in the initial bounding box are pooled with features of the region of interest to obtain a single representative feature of each anchor box, and then a mapping of the representative features of the anchor box to the confidence of each box is obtained; the network will learn to select the corresponding high-confidence anchor box and use the confidence value to perform non-maximum suppression NMS sampling on it to remove redundant boxes that are highly overlapped with the high-confidence box.

Images