A Machine Learning Method for Millimeter-Wave Radar Point Cloud Clustering and Multi-Target Recognition

Abstract

This invention discloses a machine learning method for millimeter-wave radar point cloud clustering and multi-target recognition. The method first projects the millimeter-wave radar point cloud onto a detection area to form a point cloud density distribution image. Then, a convolutional neural network is used to scan the detection area to homogenize the radar point cloud density distribution. Subsequently, the point cloud density distribution image is binarized. Finally, a connectivity analysis method is used to analyze the connectivity of the point cloud density distribution image, identifying each connected region as the same target and calculating the target's position, velocity, and other attributes. This invention can be used for clustering and target recognition of millimeter-wave radar point clouds, effectively identifying the target's spatial position, velocity, and other information. This invention uses a neural network to homogenize the spatial density distribution of the point cloud, without relying on extensive experience in selecting parameters or formulas, exhibiting high stability and applicability to various scenarios.

Description

Technical Field

[0001] This invention belongs to the field of radar data processing technology, and particularly relates to a clustering method for millimeter-wave radar point cloud data, especially a machine learning method for millimeter-wave radar point cloud clustering and multi-target recognition. Background Technology

[0002] Millimeter-wave radar is widely used in the transportation sector as a sensing device for autonomous vehicles or roadside applications, primarily for identifying and detecting targets such as vehicles and pedestrians within a designated area. Millimeter-wave radar offers advantages such as high cost-effectiveness, long detection range, and resistance to weather conditions. However, the sparse point clouds of millimeter-wave radar present challenges for point cloud clustering and target recognition. Traditional point cloud clustering algorithms, including K-means, DBSCAN (Density-based spatial clustering of application with noise), and OPTICS (ordering points to identify the clustering structure), rely on target quantity estimation, point cloud similarity formulas (such as Euclidean distance), and the selection of neighborhood radius parameters. For point cloud data in different scenarios, especially high-dimensional data, the estimation of target quantity, the selection of point cloud similarity formulas, and neighborhood radius parameters heavily depend on the experience of algorithm developers, introducing significant uncertainty into target recognition results. Therefore, new clustering methods are needed that rely less or no on experience in selecting parameters and formulas, enabling accurate and stable multi-target recognition in various complex scenarios. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of existing technologies by providing a clustering method that relies less on empirical formulas or parameter selection, can work stably in different scenarios, and is applicable to high-dimensional point cloud data, thereby improving the convenience and accuracy of target identification in point cloud clustering.

[0004] The technical solution to achieve the purpose of this invention is: a millimeter-wave radar point cloud clustering and multi-target recognition machine learning method, the method comprising the following steps:

[0005] Step 1: Acquire millimeter-wave radar point cloud data, select the detection area of ​​interest, and delete point clouds outside the detection area of ​​interest and point clouds with a velocity of 0.

[0006] Step 2: Convert the point cloud location information detected by the millimeter-wave radar from the radar's own polar coordinate system to the Cartesian coordinate system;

[0007] Step 3, Obtain point cloud density distribution image: Based on the location of the radar point cloud, project each radar point cloud onto the corresponding pixel in the detection area. The value in each pixel represents the point cloud density in the pixel, thus forming a point cloud density distribution image.

[0008] Step 4: Use a convolutional neural network to homogenize the point cloud density distribution image;

[0009] Step 5: Binarize the homogenized point cloud density distribution image;

[0010] Step 6: Perform connectivity analysis on the binarized point cloud density distribution map to identify targets;

[0011] Step 7: Traverse each connected component and average the attributes of all point clouds within each connected component as the attributes of the identified target. The attributes include position and velocity.

[0012] Furthermore, taking the lower left corner of the detection area of ​​interest as the origin, the lane line direction as the y-axis, and the direction perpendicular to the lane line to the right as the x-axis, for 3D millimeter-wave radar, the coordinate transformation formula in step 2 is:

[0013]

[0014] Where r and These are the point cloud distance information and azimuth information in the radar's own polar coordinate system, respectively. It is the angle between the radar normal angle and the lane line direction, x0 and y0 are the coordinates of the lower left corner of the detection area of ​​interest; (x, y) is the point cloud position information in the transformed Cartesian coordinate system.

[0015] Furthermore, for 4D millimeter-wave radar, the coordinate transformation formula in step 2 is:

[0016]

[0017] Where θ represents the elevation angle information of the radar point cloud, z represents the height coordinate, and z0 represents the ground height of the lower left corner of the detection area of ​​interest.

[0018] Furthermore, based on the location information of the point cloud, the density of the point cloud within each pixel is calculated to form a density distribution image. The specific method is as follows:

[0019] The detection region of interest is divided into rectangular grids, where each rectangular grid represents a pixel. The image coordinates (X,Y) corresponding to the point cloud (x,y) within this region are represented as follows:

[0020]

[0021] Where, ε x and εy These are the dimensions of the grid in the x and y directions, respectively, and floor() indicates rounding down;

[0022] Increment the pixel value at image coordinates (X,Y) by 1;

[0023] Repeat this step for each point cloud to obtain the density distribution image of the point cloud.

[0024] Furthermore, in step 4, the point cloud density distribution image is homogenized using a convolutional neural network. Specifically, the convolutional window of the convolutional neural network is used to scan the point cloud density distribution image, and the values ​​of zero-value pixels in the middle of pixels with a spacing distance less than a preset threshold and a density value that is not zero are changed to non-zero.

[0025] Furthermore, in step 4, the convolutional neural network adopts a single-layer convolutional neural network with a convolution window size of 3*3, each element having a value of 1, a convolution stride of 1, and 1 row and 1 column of 0 filling the top, bottom, left, and right sides of the convolution.

[0026] Furthermore, in step 5, the homogenized point cloud density distribution image is binarized. Specifically, a minimum point cloud density ε is set. If the point cloud density within a pixel is ≥ ε, the pixel value is 1; if the point cloud density within a pixel is < ε, the pixel value is 0.

[0027] Further, step 6 describes performing connectivity analysis on the binarized point cloud density distribution map, specifically: performing 4-neighborhood connectivity analysis on the binarized point cloud density distribution map, where the region with a pixel value of 0 has ID = 0, representing the background region; all connected non-zero pixel values ​​have the same ID and are identified as the same target; and all disconnected non-zero pixel values ​​have a different ID, representing different targets.

[0028] Furthermore, before step 7 is executed, the following judgment is also performed: extract all point clouds in each connected component. If the number of point clouds is greater than a set threshold, then step 7 is executed; otherwise, the connected component is discarded, and then step 7 is executed.

[0029] Compared with the prior art, the significant advantages of this invention are:

[0030] 1) It can be used for clustering and target recognition of millimeter-wave radar point clouds, and can effectively identify the spatial position, velocity and other information of the target.

[0031] 2) The spatial density distribution of point clouds is homogenized using neural networks. It does not rely on extensive experience to select parameters or formulas, resulting in high stability and applicability to various scenarios.

[0032] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0033] Figure 1 This is a flowchart of the millimeter-wave radar point cloud clustering and multi-target recognition machine learning method of the present invention.

[0034] Figure 2 (a) in the figure represents point cloud data measured by millimeter-wave radar. Figure 2 (b) in the image represents the filtered point cloud data. Figure 2 (c) in the figure represents the point cloud data after coordinate rotation.

[0035] Figure 3 This is a density distribution map of millimeter-wave radar point clouds.

[0036] Figure 4 This is a diagram showing the results of multi-target recognition obtained using the connectivity analysis method.

[0037] Figure 5 This is the radar point cloud extraction map corresponding to the connected domain. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0039] In one embodiment, combined Figure 1 A machine learning method for millimeter-wave radar point cloud clustering and multi-target recognition is provided, the method comprising the following steps:

[0040] Step 1: Acquire millimeter-wave radar point cloud data, select the detection area of ​​interest, and delete point clouds outside the detection area of ​​interest and point clouds with a velocity of 0.

[0041] Step 2: Convert the point cloud location information detected by the millimeter-wave radar from the radar's own polar coordinate system to the Cartesian coordinate system;

[0042] The origin is the bottom left corner of the area of ​​interest, the y-axis is the direction of the lane lines, and the x-axis is the direction perpendicular to the lane lines to the right.

[0043] For 3D millimeter-wave radar, the coordinate transformation formula is:

[0044]

[0045] Where r and These are the point cloud distance information and azimuth information in the radar's own polar coordinate system, respectively. It is the angle between the radar normal angle and the lane line direction, x0 and y0 are the coordinates of the lower left corner of the detection area of ​​interest; (x, y) is the point cloud position information in the transformed Cartesian coordinate system.

[0046] For 4D millimeter-wave radar, the coordinate transformation formula is:

[0047]

[0048] Where θ represents the elevation angle information of the radar point cloud, z represents the height coordinate, and z0 represents the ground height of the lower left corner of the detection area of ​​interest.

[0049] Step 3, Obtain the point cloud density distribution image: Based on the location of the radar point cloud, project each radar point cloud onto the corresponding pixel within the detection area. The value within each pixel represents the point cloud density within that pixel, thus forming the point cloud density distribution image; the specific process is as follows:

[0050] The detection region of interest is divided into rectangular grids, where each rectangular grid represents a pixel. The image coordinates (X,Y) corresponding to the point cloud (x,y) within this region are represented as follows:

[0051]

[0052] Where, ε x and ε y These are the dimensions of the grid in the x and y directions, respectively, and floor() indicates rounding down;

[0053] Increment the pixel value at image coordinates (X,Y) by 1;

[0054] Repeat this step for each point cloud to obtain the density distribution image of the point cloud.

[0055] Step 4: Use a convolutional neural network to homogenize the point cloud density distribution image. Specifically, use the convolutional window of the convolutional neural network to scan the point cloud density distribution image and change the values ​​of zero-value pixels in the middle of pixels with a spacing distance less than a preset threshold and a density value that is not zero to non-zero values.

[0056] Preferably, the convolutional neural network uses a single-layer convolutional neural network with a convolution window size of 3*3, each element having a value of 1, a convolution stride of 1, and 1 row and 1 column (filled with 0) on each side.

[0057] Step 5: Binarize the homogenized point cloud density distribution image. Specifically, set a minimum point cloud density ε. If the point cloud density within a pixel is ≥ ε, the pixel value is 1; if the point cloud density within a pixel is < ε, the pixel value is 0.

[0058] Step 6: Perform connectivity analysis on the binarized point cloud density distribution map to identify targets; specifically: perform 4-neighborhood connectivity analysis on the binarized point cloud density distribution map. Regions with a pixel value of 0 have ID=0, representing background regions; all connected non-zero pixel values ​​have the same ID and are identified as the same target; all disconnected non-zero pixel values ​​have a different ID, representing different targets.

[0059] Step 7: Extract all point clouds within each connected component. If the number of point clouds is greater than the set threshold, proceed to step 8; otherwise, discard the connected component and then proceed to step 8.

[0060] Step 8: Traverse each connected component and average the attributes of all point clouds within each connected component as the attributes of the identified target. The attributes include position, velocity, etc.

[0061] In one embodiment, a millimeter-wave radar point cloud clustering and multi-target recognition machine learning system is provided, including a data preprocessing module, a region convolution homogenization module, a connectivity analysis module, and a target information extraction module.

[0062] The data preprocessing module is used to filter, remove, and perform coordinate transformation on the point cloud data acquired by radar, as well as project it onto the region of interest to obtain a point cloud density distribution map of the region of interest. Data filtering includes selecting the detection region of interest, deleting point clouds outside the region of interest, and deleting point clouds with a velocity of 0 (usually corresponding to the background environment). Since the coordinate system used by radar point clouds is typically a polar coordinate system with its own position as the origin and the normal as the polar axis, coordinate transformation can convert the position and other information of the point cloud to a Cartesian coordinate system. The effect of data preprocessing is as follows: Figure 2 As shown.

[0063] Using the lower left corner of the area of ​​interest as the origin, the lane line direction as the y-axis, and the direction perpendicular to the lane line to the right as the x-axis, for 3D millimeter-wave radar, the coordinate transformation formula is:

[0064]

[0065] Where r and These are the point cloud distance information and azimuth information in the radar polar coordinate system, respectively. It is the angle between the radar normal angle and the lane line direction, and x0 and y0 are the coordinates of the lower left corner of the area of ​​interest.

[0066] If the radar is a 4D millimeter-wave radar that can provide elevation angle information, then the conversion formula is:

[0067]

[0068] Where θ represents the elevation angle information of the radar point cloud, z represents the altitude coordinate, and z0 represents the ground altitude at the lower left corner of the area of ​​interest.

[0069] Point cloud projection refers to obtaining a density distribution image of the point cloud within a region of interest based on the point cloud's coordinate information. The dimensions ε corresponding to the pixel coordinates in the x and y directions of the region of interest are defined. x and ε y Then the image coordinates (X,Y) corresponding to the point cloud (x,y) can be represented as:

[0070]

[0071] Here, `floor()` represents rounding down. Then, the pixel value at image coordinates (X, Y) is incremented by 1. This step is repeated for each point cloud instance, eventually yielding the density distribution image of the point cloud. For... Figure 2 The density distribution image of the obtained point cloud is as follows: Figure 3 As shown.

[0072] Region Convolutional Uniformization Module: Used to uniformize the point cloud density distribution image; specifically, the convolutional window of the convolutional neural network scans the point cloud density distribution map of the region of interest to uniformize the point cloud density distribution image, which can avoid being identified as two targets due to the segmentation of the point cloud of the same target.

[0073] In this embodiment, a single-layer convolutional neural network is used, with a convolution window size of 3*3, each element having a value of 1, a convolution stride of 1, and 1 row / column padded (filled with 0) on each side.

[0074] The connectivity analysis module first binarizes the point cloud density distribution image, setting pixel values ​​less than the minimum point cloud density ε to 0 and those greater than or equal to 1. Then, it performs 4-neighborhood connectivity analysis on the binarized image. The main idea of ​​4-neighborhood connectivity analysis is to assign the same ID to connected pixels with the same value, and different IDs to disconnected pixels, with background pixels assigned ID=0. Specifically, if a pixel has a point cloud density value of 0, it is assigned ID=0, representing the background. If a pixel has a non-zero density value and has not yet been assigned an ID, its 4-neighborhood of non-zero value pixels is searched, along with the 4-neighborhood of these non-zero value pixels, and so on, until they are all assigned the same new ID. The clustering method based on connectivity analysis is shown below. Figure 4 As shown.

[0075] The target information extraction module is used to calculate the average position and velocity of all point clouds within each connected component, which are then used as the actual position and velocity of the target represented by that connected component. For example... Figure 5As shown, connectivity analysis can obtain a set C of one or more pixels corresponding to each target. n =∪ i (X i ,Y i Extract the corresponding pixel set C. n All point cloud P within i ∈C n The position and velocity information of all point clouds are averaged as the position and velocity of the target.

[0076]

[0077]

[0078]

[0079]

[0080] in Let x be the x-coordinate of the target n. Let n be the y-coordinate of the target n. Let n be the velocity of the target n, and o n Other information for target n, where N is the number of point clouds.

[0081] This invention is a clustering method that relies less on empirical formulas or parameter selection, can work stably in different scenarios, and is applicable to high-dimensional point cloud data. It can improve the convenience and accuracy of target identification in point cloud clustering.

[0082] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention without departing from its spirit and scope should be included within the protection scope of the present invention.

Claims

1. A machine learning method for millimeter-wave radar point cloud clustering and multi-target recognition, characterized in that, The method includes the following steps: Step 1: Acquire millimeter-wave radar point cloud data, select the detection area of ​​interest, and delete point clouds outside the detection area of ​​interest and point clouds with a velocity of 0. Step 2: Convert the point cloud location information detected by the millimeter-wave radar from the radar's own polar coordinate system to the Cartesian coordinate system; Step 3, Obtain point cloud density distribution image: Based on the location of the radar point cloud, project each radar point cloud onto the corresponding pixel in the detection area. The value in each pixel represents the point cloud density in the pixel, thus forming a point cloud density distribution image. Step 4: Use a convolutional neural network to homogenize the point cloud density distribution image; Step 5: Binarize the homogenized point cloud density distribution image; Step 6: Perform connectivity analysis on the binarized point cloud density distribution map to identify targets; Step 7: Traverse each connected component and average the attributes of all point clouds within each connected component as the attributes of the identified target. The attributes include position and velocity. In step 4, the point cloud density distribution image is homogenized using a convolutional neural network. Specifically, the convolutional window of the convolutional neural network is used to scan the point cloud density distribution image, and the values ​​of zero-value pixels in the middle of pixels with a spacing distance less than a preset threshold and a density value that is not zero are changed to non-zero. In step 4, a single-layer convolutional neural network is used, with a convolution window size of 3*3, each element having a value of 1, a convolution stride of 1, and 1 row and 1 column of 0s on each side.

2. The millimeter-wave radar point cloud clustering and multi-target recognition machine learning method according to claim 1, characterized in that, Using the lower left corner of the detection area of ​​interest as the origin, the lane line direction as the y-axis, and the direction perpendicular to the lane line to the right as the x-axis, for 3D millimeter-wave radar, the coordinate transformation formula in step 2 is: Where r and These are the point cloud distance information and azimuth information in the radar's own polar coordinate system, respectively. It is the angle between the radar normal angle and the lane line direction, x0 and y0 are the coordinates of the lower left corner of the detection area of ​​interest; (x, y) is the point cloud position information in the transformed Cartesian coordinate system.

3. The millimeter-wave radar point cloud clustering and multi-target recognition machine learning method according to claim 2, characterized in that, For 4D millimeter-wave radar, the coordinate transformation formula in step 2 is: Where θ represents the elevation angle information of the radar point cloud, z represents the height coordinate, and z0 represents the ground height of the lower left corner of the detection area of ​​interest.

4. The millimeter-wave radar point cloud clustering and multi-target recognition machine learning method according to claim 1, characterized in that, In step 3, based on the location information of the point cloud, the density of the point cloud within each pixel is calculated to form a density distribution image. The specific method is as follows: The detection region of interest is divided into rectangular grids, where each rectangular grid represents a pixel. The image coordinates (X,Y) corresponding to the point cloud (x,y) within this region are represented as follows: Where, ε x and ε y These are the dimensions of the grid in the x and y directions, respectively, and floor() indicates rounding down; Increment the pixel value at image coordinates (X,Y) by 1; Repeat this step for each point cloud to obtain the density distribution image of the point cloud.

5. The millimeter-wave radar point cloud clustering and multi-target recognition machine learning method according to claim 1, characterized in that, In step 5, the homogenized point cloud density distribution image is binarized. Specifically, a minimum point cloud density ε is set. If the point cloud density within a pixel is ≥ ε, the pixel value is 1; if the point cloud density within a pixel is < ε, the pixel value is 0.

6. The millimeter-wave radar point cloud clustering and multi-target recognition machine learning method according to claim 1, characterized in that, Step 6 describes performing connectivity analysis on the binarized point cloud density distribution map. Specifically, this involves performing 4-neighborhood connectivity analysis on the binarized point cloud density distribution map. Regions with a pixel value of 0 have ID = 0, representing background regions. All connected non-zero pixel values ​​have the same ID and are identified as the same target. All disconnected non-zero pixel values ​​have a different ID, representing different targets.

7. The millimeter-wave radar point cloud clustering and multi-target recognition machine learning method according to claim 1, characterized in that, Before step 7 is executed, the following judgment is also performed: extract all point clouds in each connected component. If the number of point clouds is greater than a set threshold, then step 7 is executed; otherwise, the connected component is discarded, and then step 7 is executed.

Images