4D millimeter wave radar point cloud segmentation method based on automatic driving scene

Abstract

The invention provides a 4D millimeter wave radar point cloud segmentation method based on an automatic driving scene, and the method comprises the steps: collecting 4D millimeter wave radar data, and obtaining the point cloud data of continuous frames; selecting three frames of point cloud data, performing speed screening on the selected point cloud data, removing static point cloud to obtain filtered point cloud data, and performing coordinate conversion on the filtered point cloud data to convert the point cloud in a 4D millimeter wave radar coordinate system into the point cloud in a pixel coordinate system; mapping the point cloud under the pixel coordinate system into an image, carrying out edge detection, merging similar edge regions, and calculating a circumscribed matrix for the merged edge regions; calculating and generating to-be-clustered point clouds for the circumscribed rectangles, the filtered point clouds and the point clouds under the pixel coordinate system, and calculating by adopting a principal component analysis method to obtain feature vectors of the circumscribed rectangles; and carrying out 4D millimeter wave radar point cloud segmentation on the to-be-clustered point cloud and the feature vector of the external rectangle by adopting a DBSCAN clustering algorithm based on an adaptive direction ellipse neighborhood. The problem that multiple targets are too close and cannot be distinguished can be solved, and the accuracy of point cloud segmentation is improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of data processing, image processing, etc., and particularly relates to a 4D millimeter-wave radar point cloud segmentation method based on an autonomous driving scenario. Background Art

[0002] In recent years, autonomous driving technology has become popular, and various means of transportation tend to be intelligent. Due to the limited energy of drivers, modern vehicles are integrated with various sensors to assist drivers in perceiving road conditions. However, the road conditions in highway scenarios are complex, and the number of means of transportation such as cars and bicycles has increased sharply. Currently, sensors such as high-definition cameras and lidar integrated in mainstream intelligent vehicles are difficult to accurately and quickly respond to road conditions, especially in low visibility situations such as at night and in rainy and foggy weather, which can easily cause traffic accidents. Compared with sensors such as cameras and lidar, millimeter-wave radar has the advantages of long detection range and high reliability, and can work in low-light and low-visibility scenarios; in addition, millimeter-wave radar has stronger fog-penetrating ability than lidar and can work normally without being affected by bad weather such as fog, rain, and snow. On this premise, 4D millimeter-wave radar can directly receive information such as the distance, speed, and angle of a target, facilitating the screening and classification of targets.

[0003] Due to the sparse characteristics of 4D millimeter-wave radar point clouds and the interference of noise in complex road conditions, current target detection algorithms based on 4D millimeter-wave radar are mainly deep learning algorithms. And deep learning algorithms rely heavily on large and high-quality data sets. However, 4D millimeter-wave radar point clouds are sparse, and it is impossible to intuitively segment and classify point clouds through contours or certain parameters. Therefore, the point cloud segmentation of 4D millimeter-wave radar is also a difficult point that needs to be overcome in the research of deep learning and the process of autonomous driving.

[0004] Deep learning algorithms have extremely high requirements for the quality of the data set of 4D millimeter-wave radar point clouds. Therefore, the segmentation of radar point clouds is extremely important. Density Based Spatial Clustering of Applications with Noise (DBSCAN) is a commonly used clustering method, and its main idea is to divide data according to the density of the data. Guan proposed a method for clustering vehicles using DBSCAN based on lidar point clouds. To apply DBSCAN to the segmentation of 4D millimeter-wave point clouds, He proposed an improvement to the DBSCAN algorithm, that is, changing the circular neighborhood in the DBSCAN clustering algorithm to an elliptical neighborhood, which can solve the problem that multiple targets are too close to be distinguished and improve the accuracy of point cloud segmentation. However, due to the different directions of target movement in highway scenarios and the uncertainty of the point cloud distribution direction, the direction of the elliptical neighborhood often cannot fit well with measured targets such as cars, resulting in unsatisfactory point cloud segmentation effects in actual scenarios. Summary of the Invention

[0005] The object of the present invention is to provide a 4D millimeter-wave radar point cloud segmentation method in an autonomous driving scenario to adapt to millimeter-wave radar targets moving in different directions, solve the problem that multiple targets are too close to be effectively segmented, and improve the accuracy of segmenting point clouds by the 4D millimeter-wave radar.

[0006] The technical solution for achieving the object of the present invention is: a 4D millimeter-wave radar point cloud segmentation method in an autonomous driving scenario, including five steps of 4D millimeter-wave radar installation and data acquisition, 4D millimeter-wave radar data conversion, point cloud image detection, calculation of feature vectors, and 4D millimeter-wave radar point cloud segmentation, which are specifically as follows:

[0007] Step 1, install the 4D millimeter-wave radar on an autonomous driving vehicle, collect 4D millimeter-wave radar data, and obtain point cloud data of consecutive frames.

[0008] Step 2, select three frames of point cloud data from the point cloud data of consecutive frames in Step 1, perform speed screening on the selected point cloud data, eliminate static point clouds to obtain filtered point cloud data, and perform coordinate conversion on the filtered point cloud data to convert the point cloud in the 4D millimeter-wave radar coordinate system into the point cloud in the pixel coordinate system.

[0009] Step 3, map the point cloud in the pixel coordinate system in Step 2 to an image, perform edge detection, merge adjacent edge regions, and calculate the circumscribed matrix for the merged edge regions.

[0010] Step 4, for the circumscribed rectangle obtained in Step 3 and the filtered point cloud and the point cloud in the pixel coordinate system in Step 2, calculate and generate the point cloud to be clustered, and use the principal component analysis method to calculate the feature vector of the circumscribed rectangle.

[0011] Step 5, use the DBSCAN clustering algorithm based on an adaptive direction elliptical neighborhood for the point cloud to be clustered and the feature vector of the circumscribed rectangle obtained in Step 4 to perform point cloud segmentation of the 4D millimeter-wave radar.

[0012] Further, in Step 1, install the 4D millimeter-wave radar on an autonomous driving vehicle, collect 4D millimeter-wave radar data, and obtain point cloud data of consecutive frames. The specific method is as follows:

[0013] Step 1.1, fix the 4D millimeter-wave radar at the front of the autonomous driving vehicle, 60 cm above the ground, and the 4D millimeter-wave radar is facing the forward direction of the vehicle head.

[0014] Step 1.2, turn on the 4D millimeter-wave radar drive.

[0015] Step 1.3, use a buffer queue to save the point cloud data of consecutive frames.

[0016] Further, in step two, select three frames of point cloud data from the continuous frame point cloud data in step one, perform velocity screening on the selected point cloud data, filter out static point clouds to obtain filtered point cloud data, and perform coordinate transformation on the filtered point cloud data to convert the point cloud in the 4D millimeter wave radar coordinate system into the point cloud in the pixel coordinate system. The specific method is as follows:

[0017] Step 2.1, take the first three frames of point cloud data in the buffer queue in step 1.3;

[0018] Step 2.2, perform velocity screening on the three frames of point cloud data in step 2.1, filter out the points with an absolute velocity less than 0.2 m / s in the point cloud, and filter out static point clouds to obtain filtered point cloud data;

[0019] Step 2.3, perform coordinate system transformation on the filtered point cloud data obtained in step 2.2. The conversion relationship between the 4D millimeter wave radar coordinate system and the world coordinate system is shown in the following formula:

[0020]

[0021] Among them, (X W , Y W , Z W ) represents the coordinate position of the 4D millimeter wave radar point cloud in the world coordinate system, (X R , Y R , Z R ) represents the coordinate position of the 4D millimeter wave radar point cloud in the 4D millimeter wave radar coordinate system, R is the rotation matrix from the 4D millimeter wave radar coordinate system to the world coordinate system, and T is the translation matrix from the 4D millimeter wave radar coordinate system to the world coordinate system;

[0022] The rotation matrix R and the translation matrix T are as follows:

[0023]

[0024] Among them, r 11 to r 33 are the rotation relationship coefficients between the world coordinate system and the 4D millimeter wave radar coordinate system, and t 1 to t 3 are the translation relationship coefficients between the world coordinate system and the 4D millimeter wave radar coordinate system;

[0025] Project the filtered point cloud data with oxy as the plane. The conversion between the world coordinate system and the pixel coordinate system is shown in the following formula:

[0026]

[0027] Among them, (X uv , Y uvrepresents the coordinate position of the 4D millimeter-wave radar point cloud in the pixel coordinate system, (X W , Y W ) represents the coordinate position of the projected 4D millimeter-wave radar point cloud in the world coordinate system, R 1 is the rotation matrix from the world coordinate system to the pixel coordinate system, T 1 is the translation matrix from the world coordinate system to the pixel coordinate system.

[0028] Furthermore, in step three, map the point cloud in the pixel coordinate system obtained in step two into the image, perform edge detection, merge adjacent edge regions, and calculate the circumscribed matrix for the merged edge regions. The specific method is as follows:

[0029] Step 3.1, determine the parameters, including the width W, height H of the image, and the point cloud display radius r, and display the point cloud in the pixel coordinate system as white solid circles in an image with a black background to obtain a point cloud image;

[0030] Step 3.2, perform edge detection on the point cloud image using the Canny operator to obtain an edge contour;

[0031] Step 3.3, given a merging threshold t edge , merge the edge contours whose minimum distance between each other is less than the merging threshold to obtain an image detection contour;

[0032] Step 3.4, take the minimum circumscribed rectangle for the image detection contour, set a threshold t area for the area of the minimum circumscribed rectangle to exclude the influence of noise, and complete the preliminary segmentation of the point cloud to obtain multiple circumscribed matrices.

[0033] Furthermore, in step four, for the circumscribed rectangle obtained in step three and the filtered point cloud and the point cloud in the pixel coordinate system obtained in step two, calculate and generate the point cloud to be clustered, and use the principal component analysis method to calculate the eigenvector of the circumscribed rectangle. The specific method is as follows:

[0034] Step 4.1, select a circumscribed rectangle, convert the coordinates of the circumscribed matrix into the coordinates in the 4D millimeter-wave radar coordinate system, take the point cloud of the filtered point cloud in step two within the circumscribed matrix to generate multiple groups of point clouds to be clustered, and at the same time sample the white pixel points in the circumscribed rectangle to generate sampling points;

[0035] Step 4.2, convert the sampling points in the pixel coordinate system into the point cloud in the 4D millimeter-wave radar coordinate system through the transformation matrix in step two, perform principal component analysis on the converted point cloud to obtain the eigenvector of the sampling point as follows:

[0036]

[0037] Among them, e1 is the projection coefficient in the x-axis direction, and e2 is the projection coefficient in the y-axis direction;

[0038] Repeat step 4.1 until all circumscribed rectangles are processed to obtain the feature vectors of each circumscribed rectangle.

[0039] Furthermore, in step five, the point cloud to be clustered obtained in step four and the feature vectors of the circumscribed rectangles are segmented by using the DBSCAN clustering algorithm based on the adaptive direction elliptical neighborhood for the 4D millimeter-wave radar. The specific method is as follows:

[0040] Step 5.1, determine the clustering parameters, including the major axis ε a and the minor axis ε b , and the core point number threshold MinPts;

[0041] Step 5.2, for the point cloud to be clustered and the corresponding feature vectors of the circumscribed rectangles in step four Select any unvisited point p in the point cloud to be clustered, and calculate the focal coordinates of the elliptical neighborhood with an adaptive direction as follows:

[0042]

[0043] Among them, (x c , y c , z c ) are the coordinates of point p, c is the focal length in the elliptical neighborhood, F lx is the x-axis coordinate of the left focus in the elliptical neighborhood with an adaptive direction, F ly is the y-axis coordinate of the left focus in the elliptical neighborhood with an adaptive direction, F rx is the x-axis coordinate of the right focus in the elliptical neighborhood with an adaptive direction, F ry is the y-axis coordinate of the right focus in the elliptical neighborhood with an adaptive direction;

[0044] Step 5.3, calculate the number of point clouds within the elliptical neighborhood with an adaptive direction of point p as follows:

[0045]

[0046] Among them, (x i , y i , z i ) are the unvisited point cloud information in the point cloud to be clustered, t z is the threshold of the z-axis constraint, MF 1 is the distance from any point in the neighborhood to the left focus, MF 2 is the distance from any point in the neighborhood to the right focus, MF is the sum of MF 1 and MF 2 , and if MF is less than or equal to twice the major axis ε a then it is judged that (xi , y i , z i ) within the elliptical neighborhood in the adaptive direction;

[0047] Step 5.4, for the number of point clouds in the neighborhood calculated in Step 5.3, if the number of point clouds in the neighborhood of point p is greater than or equal to the set core point number threshold MinPts, classify these point clouds into one category. If the number of point clouds in the neighborhood of point p is less than the set core point number threshold MinPts, then mark point p as a noise point;

[0048] Step 5.5, loop and repeat Steps 5.2 to 5.4 until all the point clouds to be clustered are accessed, completing the 4D millimeter-wave radar point cloud segmentation.

[0049] A 4D millimeter-wave radar point cloud segmentation system based on an autonomous driving scenario, implementing the 4D millimeter-wave radar point cloud segmentation method based on an autonomous driving scenario, to achieve 4D millimeter-wave radar point cloud segmentation based on an autonomous driving scenario.

[0050] A computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the 4D millimeter-wave radar point cloud segmentation method based on an autonomous driving scenario, to achieve 4D millimeter-wave radar point cloud segmentation based on an autonomous driving scenario.

[0051] A computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the 4D millimeter-wave radar point cloud segmentation method based on an autonomous driving scenario, to achieve 4D millimeter-wave radar point cloud segmentation based on an autonomous driving scenario.

[0052] Compared with the prior art, the remarkable advantages of the present invention are as follows: 1) By converting the 4D millimeter-wave radar point cloud into a point cloud in the pixel coordinate system, and then transforming it into image processing, using an edge detection algorithm to screen the effective area, effectively suppressing environmental noise and extracting the point cloud in the area where the target is located. 2) For the point cloud in the area where the target is extracted, mapping and restoring it to the 4D millimeter-wave radar coordinate system, and obtaining the eigenvector with the largest distribution using the point cloud distribution situation. This eigenvector can be approximately represented as the target motion direction. 3) Using the DBSCAN algorithm based on an elliptical neighborhood in the adaptive direction to segment the 4D millimeter-wave radar point cloud with different motion directions, it can solve the problem that multiple targets are too close to be distinguished, improve the accuracy of point cloud segmentation, and at the same time can perform parallel processing on the point clouds in different areas where the target is extracted, improving the algorithm processing efficiency. Description of the Drawings

[0053] Figure 1This is a flowchart of the 4D millimeter-wave radar point cloud segmentation method based on the autonomous driving scenario of the present invention.

[0054] Figure 2 This is a schematic diagram of coordinate transformation.

[0055] Figure 3 This is a flowchart of the DBSCAN algorithm based on an adaptive-direction elliptical neighborhood.

[0056] Figure 4 This is the effect diagram of the present invention after image processing of the 4D millimeter-wave radar point cloud.

[0057] Figure 5 This is the effect diagram of the present invention for calculating feature vectors.

[0058] Figure 6 This is the scene diagram of the data collection of the present invention.

[0059] Figure 7 This is the comparison diagram between the method proposed by the present invention and the clustering effect using the original elliptical neighborhood DBSCAN. Detailed implementation manners

[0060] The following further elaborates on the present application in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than limiting the invention. After reading the present invention, various equivalent modifications made by those skilled in the art to the present invention fall within the scope defined by the appended claims of the present application.

[0061] A 4D millimeter-wave radar point cloud segmentation method based on the autonomous driving scenario includes five steps: installation and data collection of the 4D millimeter-wave radar, data conversion of the 4D millimeter-wave radar, point cloud image detection, calculation of feature vectors, and segmentation of the 4D millimeter-wave radar point cloud. The overall flowchart is as Figure 1 shown as follows:

[0062] Step 1: Install the 4D millimeter-wave radar on the autonomous driving vehicle and collect the 4D millimeter-wave radar data to obtain continuous-frame point cloud data. The specific method is as follows:

[0063] Step 1.1: Fix the 4D millimeter-wave radar at the front of the autonomous driving vehicle, 60 cm above the ground, and the 4D millimeter-wave radar is facing the forward direction of the vehicle head.

[0064] Step 1.2: Turn on the 4D millimeter-wave radar drive.

[0065] Step 1.3: Use a buffer queue to save the continuous-frame point cloud data.

[0066] Step 2: Select three frames of point cloud data from the continuous frame point cloud data in Step 1, perform velocity screening on the selected point cloud data, eliminate static point clouds to obtain filtered point cloud data, and perform coordinate transformation on the filtered point cloud data to convert the point cloud in the 4D millimeter-wave radar coordinate system into the point cloud in the pixel coordinate system. The specific method is as follows:

[0067] Step 2.1: Take the first three frames of point cloud data in the buffer queue in Step 1.3;

[0068] Step 2.2: Perform velocity screening on the three frames of point cloud data in Step 2.1, filter the points in the point cloud with an absolute velocity less than 0.2 m / s, and eliminate the static point clouds to obtain filtered point cloud data;

[0069] Step 2.3: Perform coordinate system transformation on the filtered point cloud data obtained in Step 2.2. The conversion relationship between the 4D millimeter-wave radar coordinate system and the world coordinate system is shown in the following formula:

[0070]

[0071] Among them, (X W , Y W , Z W ) represents the coordinate position of the 4D millimeter-wave radar point cloud in the world coordinate system, (X R , Y R , Z R ) represents the coordinate position of the 4D millimeter-wave radar point cloud in the 4D millimeter-wave radar coordinate system, R is the rotation matrix from the 4D millimeter-wave radar coordinate system to the world coordinate system, and T is the translation matrix from the 4D millimeter-wave radar coordinate system to the world coordinate system;

[0072] The rotation matrix R and the translation matrix T are as follows:

[0073]

[0074] Among them, r 11 to r 33 are the rotation relationship coefficients between the world coordinate system and the 4D millimeter-wave radar coordinate system, and t 1 to t 3 are the translation relationship coefficients between the world coordinate system and the 4D millimeter-wave radar coordinate system;

[0075] Project the filtered point cloud data onto the oxy plane. The conversion between the world coordinate system and the pixel coordinate system is shown in the following formula:

[0076]

[0077] Among them, (X uv , Y uv(X W , Y W ) represents the coordinate position of the 4D millimeter-wave radar point cloud in the pixel coordinate system, and (R 1 ) represents the coordinate position of the projected 4D millimeter-wave radar point cloud in the world coordinate system. R 1 is the rotation matrix from the world coordinate system to the pixel coordinate system, and T

[0078] Step 3: Map the point cloud in the pixel coordinate system obtained in Step 2 to the image, perform edge detection, merge adjacent edge regions, and calculate the circumscribed matrix for the merged edge regions. The specific method is as follows:

[0079] Step 3.1: Determine the parameters, including the width W, height H of the image, and the point cloud display radius r. Display the point cloud in the pixel coordinate system as white solid circles in an image with a black background to obtain a point cloud image;

[0080] Step 3.2: Perform edge detection on the point cloud image using the Canny operator to obtain an edge contour;

[0081] Step 3.3: Given a merging threshold t edge , merge the edge contours whose minimum distance between each other is less than the merging threshold to obtain an image detection contour;

[0082] Step 3.4: Take the minimum circumscribed rectangle for the image detection contour, set a threshold t area for the area of the minimum circumscribed rectangle to exclude the influence of noise, complete the preliminary segmentation of the point cloud, and obtain multiple circumscribed matrices.

[0083] Step 4: For the circumscribed rectangle obtained in Step 3 and the filtered point cloud and the point cloud in the pixel coordinate system obtained in Step 2, calculate and generate the point cloud to be clustered, and use the principal component analysis method to calculate the eigenvector of the circumscribed rectangle. The specific method is as follows:

[0084] Step 4.1: Select a circumscribed rectangle, convert the coordinates of the circumscribed matrix to the coordinates in the 4D millimeter-wave radar coordinate system, take the point cloud of the filtered point cloud obtained in Step 2 within the circumscribed matrix to generate multiple groups of point clouds to be clustered, and at the same time sample the white pixel points in the circumscribed rectangle to generate sampling points;

[0085] Step 4.2: Convert the sampling points in the pixel coordinate system to the point cloud in the 4D millimeter-wave radar coordinate system through the transformation matrix obtained in Step 2, perform principal component analysis on the converted point cloud, and obtain the eigenvector of the sampling point as follows:

[0086]

[0087] Among them, e1 is the projection coefficient in the x-axis direction, and e2 is the projection coefficient in the y-axis direction;

[0088] Repeat step 4.1 until all circumscribed rectangles are processed to obtain the feature vectors of each circumscribed rectangle.

[0089] Step Five: For the point cloud to be clustered and the feature vectors of the circumscribed rectangles obtained in Step Four, use the DBSCAN clustering algorithm based on an adaptive-direction elliptical neighborhood to perform point cloud segmentation of the 4D millimeter-wave radar. The specific method is as follows:

[0090] Step 5.1: Determine the clustering parameters, including the major axis ε a and the minor axis ε b , and the core point quantity threshold MinPts;

[0091] Step 5.2: For the point cloud to be clustered and the feature vectors of the corresponding circumscribed rectangles in Step Four Select any unvisited point p in the point cloud to be clustered and mark p as visited. Calculate the focal coordinates of the elliptical neighborhood in the adaptive direction as follows:

[0092]

[0093] Among them, (x c , y c , z c ) is the point cloud information of the current search center p, c is the focal length in the elliptical neighborhood, F lx is the x-axis coordinate of the left focus in the adaptive-direction ellipse, F ly is the y-axis coordinate of the left focus in the adaptive-direction ellipse, F rx is the x-axis coordinate of the right focus in the adaptive-direction ellipse, F ry is the y-axis coordinate of the right focus in the adaptive-direction ellipse;

[0094] Step 5.3: Calculate the number of point clouds within the elliptical neighborhood in the adaptive direction of point p as follows:

[0095]

[0096] Among them, (x i , y i , z i ) is the unvisited point cloud information in the point cloud to be clustered, t z is the threshold of the z-axis constraint, MF 1 is the distance from any point in the neighborhood to the left focus, MF 2 is the distance from any point in the neighborhood to the right focus, MF is the sum of MF 1 and MF 2 , and MF needs to be less than or equal to twice the major axis ε aIt can be determined that (x c , y c , z c ) is within the elliptical neighborhood in this adaptive direction. The steps 5.3 are repeatedly performed on the point cloud to be clustered to obtain the number of point clouds within the neighborhood;

[0097] Step 5.4: For the number of point clouds within the neighborhood calculated in step 5.3, if the number of point clouds within the neighborhood of point p is greater than or equal to the set core point number threshold MinPts, these point clouds are classified into one category. If the number of point clouds within the neighborhood of point p is less than the set core point number threshold MinPts, then point p is marked as a noise point;

[0098] Step 5.5: Repeatedly perform steps 5.2 to 5.4 until all the point clouds to be clustered are accessed, and complete the 4D millimeter-wave radar point cloud segmentation.

[0099] The present invention also proposes a 4D millimeter-wave radar point cloud segmentation system based on an autonomous driving scenario, which implements the 4D millimeter-wave radar point cloud segmentation method based on the autonomous driving scenario to achieve the 4D millimeter-wave radar point cloud segmentation based on the autonomous driving scenario.

[0100] A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the 4D millimeter-wave radar point cloud segmentation method based on the autonomous driving scenario to achieve the 4D millimeter-wave radar point cloud segmentation based on the autonomous driving scenario.

[0101] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it implements the 4D millimeter-wave radar point cloud segmentation method based on the autonomous driving scenario to achieve the 4D millimeter-wave radar point cloud segmentation based on the autonomous driving scenario.

[0102] Embodiment

[0103] In order to verify the effectiveness of the solution of the present invention, historical data of a certain section of Nanjing South University of Science and Technology Road on November 27, 2023 is selected for the following experiments.

[0104] 1) 4D millimeter-wave radar data conversion

[0105] Convert the 4D millimeter-wave radar coordinate system into the world coordinate system. The coordinate origin remains unchanged. Therefore, the rotation matrix and translation matrix from the 4D millimeter-wave radar coordinate system to the world coordinate system are:

[0106]

[0107] The practical meaning is that the x-axis in the world coordinate system is the reverse of the y-axis in the 4D millimeter-wave radar coordinate system; the y-axis in the world coordinate system is the x-axis in the 4D millimeter-wave radar coordinate system, and the z-axis in the world coordinate system is the z-axis in the 4D millimeter-wave radar coordinate system. There is no relative translation when converting the 4D millimeter-wave radar coordinate system to the world coordinate system, so the translation matrices are all 0;

[0108] The rotation matrix and translation matrix for converting the world coordinate system to the pixel coordinate system are:

[0109]

[0110] The practical meaning is that when converting the Cartesian coordinate system to the center of the image, the u-axis of the pixel coordinate system is the x-axis of the Cartesian coordinate system and is translated by half of the image width along the u-axis, and the v-axis of the pixel coordinate system is the reverse of the y-axis of the Cartesian coordinate system and is translated by half of the image width along the v-axis. A schematic diagram of the conversion relationship between the coordinate systems is as Figure 2 shown.

[0111] 2) Point cloud image detection

[0112] It is determined that the size of the generated image is 400×400, that is, W = 400, H = 400, and the radius r of the point cloud display is 3. The point cloud in the pixel coordinate system is displayed as a white solid circle in an image with a black background to obtain a point cloud image. The Canny operator is used to perform edge detection on the point cloud image to obtain an edge detection contour. Set the display setting merge threshold t edge = 5, perform merging processing on the detection results, set the area threshold t area = 90 of the minimum bounding rectangle, eliminate the noise edges, and the image processing part is as Figure 4 shown.

[0113] 3) Calculate the feature vector

[0114] Select a bounding rectangle, convert the coordinates of the bounding rectangle to the coordinates in the 4D millimeter-wave radar coordinate system, and select the point cloud within the bounding rectangle from the filtered point cloud to generate multiple groups of point clouds to be clustered. Since the radar point cloud has aggregation, the distribution of the 4D millimeter-wave radar point cloud within the bounding rectangle is uneven, which will seriously affect the result of the principal component analysis, resulting in the feature vector being unable to express the movement direction of the target. To eliminate this influence, sample the pixel points within the bounding rectangle, separate the point cloud from the background, sample the white pixel points in the bounding matrix to generate sampled points, convert the sampled points to the point cloud in the 4D millimeter-wave radar coordinate system, perform principal component analysis on the converted point cloud, and obtain the feature vector of the sampled points The obtained feature vector and the schematic diagram of the 4D millimeter-wave radar point cloud within the bounding rectangle are as Figure 5As shown in the figure. Since three-frame 4D millimeter-wave radar point cloud data is used as input and the point cloud has extensibility in the motion direction, the calculated eigenvector can approximate the motion direction of the target.

[0115] 4) 4D millimeter-wave radar point cloud segmentation

[0116] Set the major axis ε of the elliptical neighborhood parameter a = 5 and the minor axis ε b = 3, the core point number threshold MinPts = 3, and the height threshold t z = 3. For targets with less than 10 points in the category point cloud, they are defaulted to noise caused by miscellaneous points or reflections. The scene graph of the collected data is as shown in Figure 6 the figure, which contains four moving vehicle targets. The comparison effect between the elliptical neighborhood DBSCAN clustering with the same parameters and the improved adaptive direction elliptical neighborhood DBSCAN clustering is as shown in Figure 7 the figure.

[0117] In summary, the present invention uses coordinate transformation, image processing, and principal component analysis methods to complete the segmentation of 4D millimeter-wave radar point clouds, solving the problems of no clustering direction and inability to distinguish multiple targets that are too close before.

[0118] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combinations of these technical features do not conflict, they should be considered as within the scope described in this specification.

[0119] The above-described embodiments only represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A 4D millimeter wave radar point cloud segmentation method based on an autonomous driving scenario, characterized in that: It includes five steps: 4D millimeter wave radar installation and data collection, 4D millimeter wave radar data conversion, point cloud image detection, feature vector calculation and 4D millimeter wave radar point cloud segmentation, as follows: Step 1: Install the 4D millimeter-wave radar on the autonomous driving car, collect 4D millimeter-wave radar data, and obtain point cloud data of continuous frames; Step 2: Select three frames of point cloud data from the point cloud data of the continuous frames in step 1, perform speed screening on the selected point cloud data, remove the static point cloud to obtain filtered point cloud data, and perform coordinate conversion on the filtered point cloud data to convert the point cloud in the 4D millimeter wave radar coordinate system into the point cloud in the pixel coordinate system; Step 3, mapping the point cloud in the pixel coordinate system in step 2 to the image, performing edge detection, merging similar edge regions, and calculating the circumscribed matrix for the merged edge regions; Step 4: For the bounding rectangle obtained in step 3 and the point cloud filtered in step 2 and the point cloud in the pixel coordinate system, a point cloud to be clustered is generated, and a principal component analysis method is used to calculate a feature vector of the bounding rectangle; Step 5: The point cloud to be clustered obtained in step 4 and the feature vector of the circumscribed rectangle are used to perform point cloud segmentation of the 4D millimeter wave radar using the DBSCAN clustering algorithm based on adaptive directional elliptical neighborhood.

2. According to claim 1, a 4D millimeter wave radar point cloud segmentation method in an autonomous driving scenario is characterized in that: Step 1: Install the 4D millimeter-wave radar on the autonomous driving car, collect 4D millimeter-wave radar data, and obtain point cloud data of continuous frames. The specific method is as follows: Step 1.1, fix the 4D millimeter-wave radar at the front of the self-driving car, 60 cm from the ground, and face the direction of the car's forward movement; Step 1.2, turn on the 4D millimeter wave radar driver; Step 1.3, use the buffer queue to save the point cloud data of consecutive frames.

3. The 4D millimeter wave radar point cloud segmentation method based on the autonomous driving scene according to claim 1 is characterized in that: Step 2: Select three frames of point cloud data from the continuous frames of point cloud data in step 1, perform speed screening on the selected point cloud data, remove static point clouds to obtain filtered point cloud data, and perform coordinate conversion on the filtered point cloud data to convert the point cloud in the 4D millimeter wave radar coordinate system into the point cloud in the pixel coordinate system. The specific method is as follows: Step 2.1, get the first three frames of point cloud data in the buffer queue in step 1.3; Step 2.2, perform speed screening on the three frames of point cloud data in step 2.1, filter the points in the point cloud with an absolute speed less than 0.2 m / s, and remove the static point cloud to obtain filtered point cloud data; Step 2.3, transform the filtered point cloud data obtained in step 2.2 into a coordinate system. The transformation relationship between the 4D millimeter wave radar coordinate system and the world coordinate system is shown in the following formula: Among them, (X W ,Y W ,Z W ) represents the coordinate position of the 4D millimeter wave radar point cloud in the world coordinate system, (X R ,Y R ,Z R ) represents the coordinate position of the 4D millimeter-wave radar point cloud in the 4D millimeter-wave radar coordinate system, R is the rotation matrix from the 4D millimeter-wave radar coordinate system to the world coordinate system, and T is the translation matrix from the 4D millimeter-wave radar coordinate system to the world coordinate system; The rotation matrix R and translation matrix T are as follows: Among them, r 11 to r 33 is the rotation relationship coefficient between the world coordinate system and the 4D millimeter wave radar coordinate system, and t1 to t3 are the translation relationship coefficients between the world coordinate system and the 4D millimeter wave radar coordinate system; The filtered point cloud data is projected on the oxy plane, and the conversion between the world coordinate system and the pixel coordinate system is shown in the following formula: Among them, (X uv ,Y uv ) represents the coordinate position of the 4D millimeter wave radar point cloud in the pixel coordinate system, (X W ,Y W ) represents the coordinate position of the projected 4D millimeter-wave radar point cloud in the world coordinate system, R1 is the rotation matrix from the world coordinate system to the pixel coordinate system, and T1 is the translation matrix from the world coordinate system to the pixel coordinate system.

4. The 4D millimeter wave radar point cloud segmentation method based on the autonomous driving scene according to claim 1 is characterized in that: Step 3: Map the point cloud in the pixel coordinate system in step 2 to the image, perform edge detection, merge similar edge areas, and calculate the external matrix for the merged edge areas. The specific method is as follows: Step 3.1, determine the parameters, including the width W, height H and point cloud display radius r of the image, and display the point cloud in the pixel coordinate system as a white solid circle in the image with a black background to obtain a point cloud image; Step 3.2, use the Canny operator to perform edge detection on the point cloud image to obtain the edge contour; Step 3.3, given the merging threshold t edge , merge the edge contours whose minimum distance between each edge contour is less than the merging threshold to obtain the image detection contour; Step 3.4, take the minimum bounding rectangle for the image detection contour, and set the threshold t for the area of ​​the minimum bounding rectangle area Eliminate the influence of noise, complete the preliminary segmentation of the point cloud, and obtain multiple external matrices.

5. The 4D millimeter wave radar point cloud segmentation method based on the autonomous driving scene according to claim 1 is characterized in that: Step 4: For the bounding rectangle obtained in step 3 and the point cloud filtered in step 2 and the point cloud in the pixel coordinate system, calculate and generate the point cloud to be clustered, and use the principal component analysis method to calculate the feature vector of the bounding rectangle. The specific method is as follows: Step 4.1, select a bounding rectangle, convert the bounding matrix coordinates into coordinates in the 4D millimeter wave radar coordinate system, take the point cloud of the filtered point cloud in step 2 within the bounding matrix, generate multiple groups of point clouds to be clustered, and sample the white pixels in the bounding rectangle to generate sampling points; Step 4.2: Convert the sampling point in the pixel coordinate system into a point cloud in the 4D millimeter wave radar coordinate system through the conversion matrix in step 2, perform principal component analysis on the converted point cloud, and obtain the feature vector of the sampling point As shown below: Among them, e1 is the projection coefficient in the x-axis direction, and e2 is the projection coefficient in the y-axis direction; Repeat step 4.1 until all bounding rectangles are processed and the feature vector of each bounding rectangle is obtained.

6. The 4D millimeter wave radar point cloud segmentation method based on the autonomous driving scene according to claim 1 is characterized in that: Step 5: The point cloud to be clustered obtained in step 4 and the feature vector of the circumscribed rectangle are used to perform point cloud segmentation of the 4D millimeter wave radar using the DBSCAN clustering algorithm based on adaptive directional elliptical neighborhood. The specific method is as follows: Step 5.1, determine the clustering parameters, including the major axis ε of the ellipse neighborhood a and the minor axis ε b , core point number threshold MinPts; Step 5.2: The feature vectors of the point cloud to be clustered and the corresponding bounding rectangle in step 4 are Select any unvisited point p in the point cloud to be clustered, and calculate the focal coordinates of the ellipse neighborhood in the adaptive direction as follows: Among them, (x c ,y c ,z c ) is the coordinate of point p, c is the focal length in the ellipse neighborhood, F lx is the x-axis coordinate of the left focus in the adaptive direction ellipse, F ly is the y-axis coordinate of the left focus in the adaptive direction ellipse, F rx is the x-axis coordinate of the right focus in the adaptive direction ellipse, F ry is the y-axis coordinate of the right focus in the adaptive direction ellipse; Step 5.3, calculate the number of point clouds within the elliptical neighborhood of the adaptive direction of point p, as shown below: Among them, (x i ,y i ,z i ) is the point cloud information that has not been visited in the point cloud to be clustered, t z is the threshold of the z-axis constraint, MF1 is the distance from any point in the neighborhood to the left focus, MF2 is the distance from any point in the neighborhood to the right focus, MF is the sum of MF1 and MF2, and MF is less than or equal to twice the major axis ε a That is, judge (x i ,y i ,z i ) is within the elliptical neighborhood of the adaptive direction; Step 5.4: for the number of point clouds in the neighborhood calculated in step 5.3, if the number of point clouds in the neighborhood of point p is greater than or equal to the set core point number threshold MinPts, these point clouds are classified into one category; if the number of point clouds in the neighborhood of point p is less than the set core point number threshold MinPts, point p is marked as a noise point; Step 5.5, repeat steps 5.2 to 5.4 in a loop until all point clouds to be clustered are visited, and the 4D millimeter wave radar point cloud segmentation is completed.

7. A 4D millimeter wave radar point cloud segmentation system based on an autonomous driving scenario, characterized in that: Implement the 4D millimeter-wave radar point cloud segmentation method based on the autonomous driving scene as described in any one of claims 1-6 to achieve 4D millimeter-wave radar point cloud segmentation based on the autonomous driving scene.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for 4D millimeter-wave radar point cloud segmentation in an autonomous driving scenario according to any one of claims 1 to 6 is implemented to achieve 4D millimeter-wave radar point cloud segmentation in an autonomous driving scenario.

9. A computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for 4D millimeter-wave radar point cloud segmentation in an autonomous driving scenario according to any one of claims 1 to 6 is implemented to achieve 4D millimeter-wave radar point cloud segmentation in an autonomous driving scenario.

Images